By Colin Woodward
As the cliché goes, the Civil War was a conflict that pitted “brother against brother.” But that was not true for most American families, and it certainly wasn’t true for the Lee family. All of Lee’s brothers supported the Confederacy. His brother Smith served as a Confederate naval officer. All of Lee’s sons, furthermore, served in the Confederate military. The war, nevertheless, took an enormous toll–even on those who survived it with their lives and limbs intact.
One did not have to lose a son, brother, or husband in battle to experience tragedy during the Civil War. All three of Robert E. Lee’s sons survived the war. But the Lee family suffered tragedy all the same. The Lees lost their home Arlington early in the war to Federal troops. After four years of horrendous fighting, Robert E. Lee lost the war and his citizenship.
Long before the guns at Appomattox fell silent, General Lee lost a sister and daughter to illness as well as several grandchildren. Lee also nearly lost his son, William Henry Fitzhugh, in 1863. Fitzhugh languished for months in a Union prisoner of war camp before being released.
Called Fitzhugh for short and often referred to as simply “F” in his father’s letters, Fitzhugh also went by the nickname “Rooney.” He was Robert E. Lee’s second son (Custis was the oldest) and he served as a cavalryman during the war. Rooney rose to the rank of major general. Custis was a brigadier general, though most of his time in the army was not spent in field command. Robert, Jr., began the war as a private and ended the war as a major in charge of troops defending Richmond.
Against the odds, all of Lee’s sons survived the war. Generals and other officers had a better chance of getting killed than privates did, but none of Lee’s sons suffered such a fate. Nevertheless, it was the rare officer who did not get killed, wounded, or captured, or suffer the loss of a loved one.
Fitzhugh was wounded in the fighting of June 9 at Brandy Station, the largest cavalry battle of the war. He was sent to Ashland, Virginia, to heal, but this did not stop Yankee troops from seizing him. Union officials eventually sent him to Fort Lafayette prison camp in New York, where he stayed until 1864.
This letter of June 1863 describes the circumstances surrounding Fitzhugh’s capture:
By then, the 26 year old Fitzhugh had already lost two young children to disease. While in captivity, he suffered another loss: his wife, Charlotte Georgiana Carter Wickham Lee, who was only 22. She died the day after Christmas in 1863. She is buried in Shockoe Cemetery in Richmond, where her grave has the epitaph, “How Sweet Her Memory.”
Ann Kinloch Lee Marshall, Unionist
Robert E. Lee survived the war. But as head of the South’s most aggressive army, he saw death all around him. Soldiers died by the thousands from battle and disease. But those behind the lines also suffered.
Lee’s older sister Ann died in 1864. Unlike Robert, she was supported the Union–the only one of Robert’s siblings to do so. She married Louis William Marshall, a judge and native of Maryland. When he resigned from the United States army, Robert E. Lee wrote to his sister Anne about his reasons for siding with Virginia in the secession crisis.
“With all my devotion to the Union and the feeling of loyalty and duty of an American citizen, I have not been able to make up my mind to raise my hand against my relatives, my children, my home…I know you will blame me; but you must think as kindly of me as you can, and believe that I have endeavored to do what I thought right.”
The war effectively ended their relationship. Anne died on 1864 February 20. Few of her letters survive and no photographs. Historians are not even sure where she died, though it most likely seems to have been Baltimore.
Annie Lee, the General’s Daughter
Probably the deepest loss General Lee suffered during the war was his daughter, Anne. She died of an illness in October of 1862 in North Carolina at the age of 23. She had moved to North Carolina to escape the ravages of the war in Virginia. Her death was a great blow to the family, but it happened at a time when Robert E. Lee had little time for mourning.
Over the course of the war, Robert E. Lee’s letters become increasingly fatalistic. The general never lost faith in the healing power of Christianity. In fact, the war strengthened his views that the dead were enjoying a better life in the hereafter.
The Lees also lost people close to them, such as William Orton Williams, who befriended Lee’s daughter Agnes. The controversial and perhaps mentally unstable Williams was not killed in battle but hanged in July 1863 for being a traitor. The incident, which happened a few days after Lee’s defeat at Gettysburg, further embittered General Lee against the Yankees.
By the end of the war, General Lee was a man without a home or a country. When he had heard of the death of A. P. Hill at Petersburg, he said, “He is now at rest, and we who are left are the ones to suffer.” That fairly well summed up the general’s view as well as that of many near and dear to him.