When it comes to the sons of Light Horse Harry Lee, Robert E. Lee certainly leads in terms of popularity. But it was not inevitable that Robert E. Lee emerged as the best known member of his family. For a while, it seemed that Sidney Smith Lee (1802-1869) would achieve the greatest military glory among his brothers. However, Robert’s brother has emerged as something of a footnote, rather than central player, in the Lee family story.
Who was Sidney Smith? It is telling that historians aren’t even in agreement concerning the spelling of his first name (some write it Sydney, others Sidney). Part of the reason why they can’t agree is because Sydney/Sidney went by Smith rather than his first name. Another reason he is not well known is that he wasn’t much of a writer. Robert E. Lee left thousands of letters behind. Smith, very few.
Smith was born 1802 September 2 in Camden, New Jersey, while his father was on business in Philadelphia and his mother was visiting a friend in Jersey. After graduating from the U.S. Naval Academy, at the age of 18, he became a midshipman in the United States navy.
From his youth, Smith had a reputation for falling out of contact. In 1822, Smith’s mother, Ann Carter Lee, complained about her son not writing very often. She wrote to Smith, who was then at sea, saying, “if you would accustom yourself to writing letters, habit would soon reconcile you to it, and you would be improved by it, and you certainly ought to write to your Mother, even if it be disagreeable to you to do so.”
Back in the days when people wrote letters often, it was not uncommon for mothers (or fathers) to complain about children not writing enough. Smith, however, was just as bad as writing to his brother, Robert, as he was his mother. Robert wrote to his brother Charles Carter to say: “And there is that vile Smith (as I am in a bad humour, he shall not escape) when he comes ashore I can’t see, and if I write to him he’ll not answer. Tell him that I see no good he does in the U. S. he had better go to sea again, that no body wants him here. What is he doing now?”
As is true of his more famous brother, Smith was known as a handsome man. As one of Robert E. Lee’s daughters said of Smith, “No one who ever saw him can forget his beautiful face, charming personality, and grace of manner, which, joined to a nobility of character and goodness of heart, attracted all who came into contact with him, and made him the most generally beloved and popular of men.”
Smith, then, was a “catch,” and in 1834, he married Anna Maria Mason, the daughter of Hon. John and Anna Maria Mason of “Clermont” in Fairfax County. After being wedded in Alexandria, the couple returned to Arlington, where much punch was consumed by the revelers, who drank from a bowl that once belonged to George Washington.
During the Mexican War, which both Robert and Sidney Smith Lee fought in, Robert was pleased to get a letter from his brother. “Your long & affectionate letter of the 14 Jany from Annapolis, was brought me by the train that arrived on the 1st Inst: from Vera Cruz,” Robert said on 1848 March 4. “It has given me great pleasure & I have read it over & over.” Robert in return replied with a long letter.
As the Civil War approached, Smith had an impressive resume. By then, he had fought Seminoles, Mexicans, and had been with Commodore Perry when he opened American markets in Asia. But his Civil War experiences proved lackluster.
As was true of his brother, Smith was not an ardent secessionist. Nevertheless, he “followed his state” after it left the Union and served as an officer in the CSA navy. Early in the war, he had admirers, including South Carolina diarist Mary Chesnut, who preferred Smith Lee to his army brother, Robert.
In December of 1861, Smith was sent to command in Norfolk. He was skeptical of armored ships, but nevertheless he helped construct the CSS Virginia, which was made by converting the USS Merrimack into a new kind of fighting machine: the ironclad. The Virginia had a brief and glorious history when it destroyed several enemy ships in March 1862 before its famous battle with the USS Monitor, which ended in a draw.
Smith Lee did not actually take part in the fight for Hampton Roads, and after fighting the Monitor, the Virginia was never again used in combat. Norfolk fell on 1862 May 10, and the Virginia was destroyed by Rebels the next day. Smith managed to remove many of the military stores in Norfolk before the Federals seized them. His actions helped the Confederates stay supplied amid General McClellan’s campaign for the capital. Smith had a chance to distinguish himself at the battle of Drewry’s Bluff a few days after Norfolk’s fall. The Confederacy won a victory at Drewry’s Bluff, but Smith Lee did nothing of note.
Afterward, Lee was sent to the Confederate naval academy in Charlotte (yes, the CSA had a naval academy). Smith did nothing exceptional for the rest of the war. His inability to win the kind of fame his brother enjoyed may have owed to bad luck and ill timing. Robert E. Lee certainly did not achieve great successes at his first battles. But it also seems that Smith lacked two important qualities that his brother possessed: energy and aggressiveness.
Smith’s love of the navy, though, apparently ran in the family. His son, Sidney Smith, Jr., saw action on various vessels during the war, including the Confederate ships Louisiana, Shenandoah, Rappahannock, Atlanta, and Georgia. He survived the war and lived until 1888. He never married.
After his famous brother’s surrender at Appomattox, Smith moved to Stafford County, where he was a struggling farmer. True to form, Smith once again fell out of touch with his brother for a long period. He did not see Robert again until May of 1869. General Lee, who was then president of Washington College, traveled to see Smith, who was faring poorly. On 1869 July 24, Smith died. Robert said the loss of Smith made for a “sad gap in our family . . . a grievous affliction to me which I must bear as well as I can.” Because he did little to document his life and career, Smith still remains a gap in the history of the Lees.
For further reading on Smith Lee, see Frank Cunningham, “A Seafaring Lee: Almost Forgotten Today, He Trained Two Navies,” Richmond Times-Dispatch, 1952 April 6, Section A.