The Grey Family

Who are the members of the Gray family? Why are they so important, not only to Arlington County, Virginia, but to the United States? The answers to these questions are rooted in the Civil War, when the lives of all African Americans teetered somewhere between emancipation and oppression. The family matriarch, an enslaved woman named Selina Gray, saved a group of artifacts associated with George Washington, an act that has earned her recognition as one of the nation’s first preservationists. After the war, Selina’s son, Harry W. Gray, built the first brick townhouse in Arlington, VA, which has been designated a National Historic Landmark and serves as a monument to African American achievements in Arlington County. Then, in the 1930s, Selina’s daughters provided the National Park Service with details necessary to restore Arlington House to its original state. Given their heritage, the Gray family’s contributions to the field of historic preservation are not only remarkable, they are heroic.

Life of the Gray Family at Arlington House

Selina Norris Gray was born and raised a slave at Arlington House, the Virginia plantation of George Washington Parke Custis. Custis, the adopted son of George Washington, also owned her parents, Leonard and Sally, along with X other individuals. He had inherited many of his slaves when Washington died in 1799, and set them to work in 1802 building his new home, which he intended to store and display a variety of objects associated with the nation’s first president. When Custis died in 1858, the Washington treasury passed to his only child, Mary, the wife of Robert E. Lee. Selina became very familiar with these artifacts because she served for many years as Mary Lee’s personal maid, a position that required her to tend Lee’s growing household. Selina also married and raised a family of her own at Arlington. 

While Selina worked in the main house, her husband, Thornton Gray, labored as both stable-hand and handyman. How Thornton came to be a slave on the Arlington estate is unclear; his mother, a woman of African and Native American parentage, was a slave at Mount Vernon, and several documents suggest that George Washington himself manumitted Thornton years earlier (Gillem, NPS Interview). Some historians have speculated that Thornton went as a slave to Arlington House so that he could stay with his family in Virginia, where the opportunities available to free blacks were limited, at best. This conjecture seems unlikely, as it suggests a conscious decision on Thornton’s part to remain enslaved. (African American Story)  Regardless of how and why he came to be there, it seems likely that his feelings for Selina Norris motivated him to stay.

Though married, Selina and Thornton Gray faced the harsh reality that Virginia law declared “slave unions” not legally binding; although some religious denominations recognized marriages between slaves, finding an ordained minister who would perform the ceremony was difficult. (African American Story) Mary Custis Lee, as a symbol of her affection for her maid, made the arrangements for Selina and Thornton’s wedding, which took place at Arlington House in the same room where she had married Robert E. Lee in 1831. According to Emma Syphax and Sarah Wilson, two of Selina and Thornton Gray’s daughters, their parents were married “by an Episcopal clergyman from Alexandria, whom Mrs. Lee had come over to perform the ceremony.” (Syphax and Wilson) Following their marriage, the Grays had eight children at Arlington House: Emma, Annice, Florence, Sarah, Ada, Selina, John, and Harry. All eight were also slaves. In Virginia, as in most southern states, a child’s status reflected the condition of the mother. Thus the entire Gray family appeared in the “Inventory of the Personal Estate of Major George W.P. Custis,” recorded at the Alexandria County Court House on September 11, 1858. (Inventory) Listed along with the mules, cattle, ploughs, wagons and carts, the Grays were classified in the same category as livestock and farming equipment--property--and were subject to the same treatment, good or bad, at the complete discretion of their owner. Many masters sold children away from their parents, for example. Fortunately, Selina and Thorton were allowed to keep their children and lived with them on the estate.

The Grays occupied two rooms in a substantial brick building located immediately south of the main house and to one side of a large courtyard. These betterthan average accommodations were assigned to them because Selina was Mary Lee’s personal maid and housekeeper, a position of high status on the plantation. In addition, it was necessary that Selina be available on short notice should Lee need her, and close proximity made Selina’s walk much shorter. The building the Gray’s occupied even came to be called “Selina’s House.” They lived in the west end of that structure. The middle part was the smokehouse, and the east end, the one nearest the main house, contained the meat house and a storeroom. A similar building stood on the north side of the courtyard and accommodated the kitchen and additional slave quarters.

Because of their architecture and high quality materials, these were extraordinary buildings for slaves to live in. Overall, the buildings had a classical Greek theme that complemented the main house. The rear facade of the south building included Greek pilasters with small rounded arches. (Arlington House: Official National Park Handbook, 45) On the front, small panels above the doors featured paintings – purportedly crafted by George Washington Parke Custis - of American Eagles and George Washington’s warhorse. After the war, the Gray family took over the entire south building and cut doors between the three portions. At the turn of century, the south building was returned to its original design; however, during another attempt to restore Arlington House in the 1930s, National Park Service historians endeavored to include the post-war changes made by the Gray family.

While the slave quarters underwent restoration, many of Selina and Thornton’s children were interviewed to insure authenticity. Two of them, Emma Gray Syphax and Sarah Gray Wilson, supplied important information, including the tiniest of details, such as the placement of their parents’ bed, which was located to the right of their front door. Mrs. Syphax and Mrs. Wilson stated that the children slept in a loft that had such a low ceiling that they could not stand up in it. They also carefully noted that the steps to the loft were placed in front of the window nearest the fireplace. (Syphax and Wilson) Much in the tradition of their mother, the Gray children had preserved their family’s treasured memoirs.

The daughters also shared memories about growing up as slaves of the Lee family. According to Mrs. Syphax and Mrs. Wilson, the Gray children were the only slave children allowed near the main house; the field slaves were not welcome there and rarely came around. They recalled that Mrs. Lee always treated their mother especially well and said that the Lees were good to them, too. Mrs. Syphax remembered that one of the Lee daughters, Miss Mildred,  who was the same age as she, taught her the “ABC’s.” (Syphax and Wilson) The Gray children studied everything from grammar to religion with the Lee children. In addition, they engaged in less serious childhood pastimes.

Annie Gray Baker and Ada Gray Thompson, two of Selina’s and Thornton’s other daughters, recalled much laughter and silliness inside Arlington House. For example, Miss Annie Lee, a very religious woman, read the Gray children bible verses, but afterwards she taught them many children’s songs like “Let Dogs Delight to Bark and Bite,” “Little Drops of Water, Little Grains of Sand,” and “Far, Far away.” (Baker and Thompson) Mrs. Baker conveyed the sense of ease that she and her siblings felt among the Lees. She noted that, “in the old days” she would freely mimic Miss Martha Williams, Robert E. Lee’s cousin and “a very pretty and attractive girl.” According to Mrs. Baker, Miss Martha would come down the stairs “in her large billowing dress” and say, “Good morning, Cousin Robbie” while “rolling her eyes” at him. Later, when the adults were no longer around, the Lee boys would beg Baker to imitate Williams’ simpering manner towards their father. (Baker and Thompson) Such comraderie suggests that the Lee children had perhaps a better relationship with the Grays than they did with some of their own relatives.

The closeness that existed between the Gray and Lee children may have reflected the bond shared by their mothers. Karen Byrne of the National Park Service suggests that, despite the inequities of their relationship, as the lives of Selina Gray and Mary Custis Lee became intertwined, the two women became good friends. Both had big families and thus both experienced “the joys and frustrations of motherhood.” In addition, their temperaments seemed complementary. Lee was very untidy, never on time, and a poor housekeeper. She relied on Selina to keep not only her house and family in order but her frame of mind. Over the years, mistress and slave developed a kind of mutual respect for one another. However, most importantly, the Civil War and the survival of Arlington House and its Washington heirlooms bound them together in a significant cause. (Byrne)

When her husband joined the Confederacy as commander of the military forces in 1861, Mary Lee realized that she and her children would have to leave Arlington House as it was so close to the Union capital. According to Selina’s daughters, Mrs. Baker and Mrs. Thompson, when Lee fled, she took with her only two slaves because the journey to Ravensworth [NB: define it, another Custis plantation?] had to be swift and inconspicuous. (Baker and Thompson) Similarly, since carrying a large quantity of personal effects was impossible, Lee shipped some of the family’s belongings, but had to leave behind many of the Washington artifacts and all of the household furniture. (Baker and Thompson) Consequently, before she left Lee gave her trusted maid, Selina Gray, the keys to Arlington House and thereby made a slave the official “head of the household.” (Byrne)

One can only imagine how Gray felt about the departure of her owner and the impending arrival of Union troops. She surely appreciated the opportunity that she and her family suddenly had for freedom. But as Karen Byrne asserts, Gray had grown up “steeped in the Washington apotheosis” and well understood the national significance of the Washington treasury. Moreover, Gray must have known that Mary Lee had “supreme confidence” in her abilities to safeguard the mansion, and she probably shared Lee’s belief that her guardianship would be temporary. (Byrne) For whatever reasons, Selina Gray stayed put and with her family protected Arlington House just as she promised she would.

Union forces occupied the Lee estate on May 23, 1861, and made Arlington House the headquarters for the Department of Northern Virginia. According to two of her daughters, Selina Gray greeted the commander, General Irwin McDowell, at the front door and, hoping to avoid any conflicts that might jeopardize the safety of the Washington heirlooms, graciously handed him the keys to the mansion. (Syphax and WIlson) During the next few months, Selina and her family watched helplessly as an encampment of thousands of federal troops surrounded the main house. The soldiers destroyed much of the property by constructing roads, removing trees, and building fortifications. Selina could do nothing about these activities, and so concentrated her efforts on saving the Lees’ belongings. In December, however, “she discovered that various items had disappeared.” (Byrne) Soon thereafter, she witnessed some of  McDowell’s men looting the house. Her daughters said that Gray tried to intervene and told the thieves “Never to touch any of the things, [that] they were Miss Mary’s things,” but the soldiers ignored her. (Syphax and Wilson) Later, Gray investigated and meticulously inventoried the supposedly secret areas of the house where she had hidden Mary Lee’s keepsakes. When she discovered that some of the Washington relics had also disappeared, she promptly provided a list of the missing objects to General McDowell and convinced him that the significance of the collection required his involvement. He first secured the attic and basement areas to prevent further theft, then had the remaining Lee heirlooms shipped to the Patent Office in Washington, DC for safekeeping. McDowell was upset and embarrassed that such crimes had been committed under his command, so he included a note with the boxes that stated: “I have, during the time I have been here, endeavoured to take the greatest care of this house and its furniture, and of the grounds . . . [T]his place is not a safe one for the preservation of anything that is known to have an historical interest small or great.” (Nelligan, 424-5) The note was significant because it established that McDowell, the man to whom the soldiers owed their complete obedience, believed that he could not protect the Washington relics. Had Selina Gray felt the same way, their fate would have been very different.

Six years after successfully executing the monumental task of safeguarding the Lee treasures, Selina and her family finally left Arlington House. Like other newly emancipated blacks in the area, they bought land and established a home of their own in what was called Convalescent Camp, a community located along present-day Shirlington Road in Green Valley. There the Grays farmed their fifteen-acre property and harvested produce that they sold at the corner of Seventh Street and Louisiana Avenue in Washington. They lived out the rest of their lives in freedom. (Gillem Interview)

Selina’s legacy has been too long overlooked. By the time of her death in 1907, she appeared to be just an ordinary black woman. By contrast, most early preservationists, like Ann Pamela Cunningham, who founded the Mount Vernon Ladies’ Association, were the wives of wealthy white men, and they had little interest in celebrating the contributions of a former slave. (Byrne, 22) Despite her origins (or perhaps because of them), Selina possessed many of the qualities essential to success, especially courage, confidence, and conviction. In the generations that followed, these traits would come to define other members of the Gray family as well. 

The Harry W. Gray House in Arlington, VA:
A National Historic Landmark

            The Gray family has another claim to historic preservation: Harry W. Gray, Selina and Thornton’s son, built a house in Arlington County that is now a National Historic Landmark. Born into slavery, Harry Gray grew up on the Custis-Lee estate, where he did odd jobs around the mansion. Most notably, he built the high masonry wall that surrounds what is now Arlington National Cemetery and thus became a skilled mason. Hi daughter, Martha Gray Gillem, contends that, during the Civil War, Harry was taught to read and write at a fifth grade level by the men of the Northern New York Volunteers. (Interview 2) Having a trade and being literate would serve him very well after emancipation, when he became a leader in Arlington County. For forty years, he worked as a clerk and messenger at the Department of the Interior. He also worked at the Blink West Brickyard and tended his own ten-acre farm. By working three jobs, Harry was able to fulfill his lifelong dream of building an elegant brick townhouse for his wife and children. (“Historic Arlington Days Oct. 9 & 10,” 8) Unlike stone, which was easy to find and inexpensive, fired clay brick was scarce and very expensive, making it a symbol of a family’s prosperity.            

Harry modeled his residence after the brick row houses that characterized the Foggy Bottom area of Washington, DC. Martha Gray Gillem said in a 1963 interview that her “Papa” bought bricks from the Blick-West Brickyard when he could afford them, and over many years, bit by bit, he built a two-story row house on his farm. (Arlington County Historic Notebook 1) “Since he built it like the city row houses . . . there are no windows on the sides and the house is narrow and tall,” Gillem explained. For posterity’s sake, Harry carved his name and the date into the last brick before placing it into position near the rear doorway. (Netherton and Ross 85) All told, the house cost about $1,800 to build, more than twice the amount he paid for the land it sat on. When the Gray family finally moved into the house on May 1, 1881, it was the only one of its kind for miles.

The house also boasted an elaborately designed landscape. Besides the house, the ten-acre property included an outhouse, a buggy shed, a barn, a pig house, a well, and a brick patio. Surrounding the house were apple, peach, pear, and cherry orchards, a cornfield, and a grazing field for livestock. Interspersed between all of these were flower gardens that must have splashed vibrant colors across the property. Plat maps even show a croquet field! Given Harry Gray’s exacting nature, such attention to detail is not surprising.

Above all his accomplishments, Harry took greatest pride in his family. He and his wife, Martha, a former slave once owned by former president James Madison, had four children: Thorton, Julia, Sara, and Martha. The Grays taught their offspring the value of determination and hard work. Harry especially fostered the belief that knowledge could open the doors of opportunity that racial inequities had otherwise closed to African Americans. Before he died, Harry saw Thornton become an attorney and all three daughters, teachers.

In his will, Harry Gray attempted to see that his own legacy—the brick townhouse—would be preserved. The document stated that all of his debts should be paid first and that everything that remained should pass to his wife Martha Hoard Gray. In keeping with state law, if she couldn’t pay the bills with the money left in his estate, then the will authorized Martha to sell some of the ten-acre property. When Gray’s widow died, the will stipulated further that his daughter, Martha Gray Gillem, should inherit the family home, all of its furniture, and two-and-a-half acres of land. The remaining seven-and-a-half acres were to be divided equally among the other children, who could do as they pleased with their share of the land. Under no circumstances, however, was the townhouse to be sold outside the family. (Gray Will) In her 1952 will, Martha Gray Gillem left the building to her son, Henry, but in 1975 it passed to her daughter, Cecelia Mitchell. Mitchell and her husband, George, sold it to Commander and Mrs. Richard Storwick in 1979.

By the late 1970s, Harry Gray’s impressive property had changed considerably. The main dwelling had been moved from its original location along what is now Columbia Pike, and the Mitchells had made extensive renovations to it. The once vibrant gardens had given way to the numerous houses of Harry Gray’s children and grandchildren. Other lots had been sold to family friends and some strangers, too. As the neighborhood—called Gray’s Subdivision—changed, concern grew that the brick townhouse might be resold, remodeled, or worse. A movement thus arose to preserve its historic integrity. In 1983, with the Storwicks full support and cooperation, the house became an historic landmark. This designation provides tax benefits to the property’s owners, who have changed several times since 1983, but it requires them to secure permission for any changes from Arlington County’s Historic Affairs and Landmark Review Board. With an historic marker standing guard in the front yard, the house is now a permanent symbol of the Gray family’s achievements. Harry Gray would undoubtedly be pleased.

Restoration of Arlington House 

            During major restorative efforts to Arlington House from 1929 to 1930, the Gray family made another important contribution to the history of Arlington County and the nation. Four of Selina and Thornton’s daughters provided crucial details about the house and its furnishings, and their input proved vital to the authenticity of the project. A series of interviews with Emma Gray Syphax and Sarah Gray Wilson in 1929 and with Annie Gray Baker and Ada Gray Thompson in 1930 enabled preservationists to restore the Custis-Lee estate to its Civil War era appearance. 

The recollections of Syphax and Wilson painted a vivid mental image of the North Building, which sat opposite the South Building where the Gray family lived. They recalled that while other slaves did live in the North Building, its primary function was for food preparation and cooking. The sisters indicated that it was called the “Summer Kitchen,” although meals were prepared there all year long. Summer kitchens were typical on southern plantations, where weather conditions during certain months prompted the temporary relocation of cooking facilities, with their heat and odors, away from the main dwelling. Both Syphax and Wilson remembered that the kitchen was large enough to accommodate the preparation of the most elaborate of feasts, and they knew every detail of the North Building’s interior, which included a large, open fireplace with a spit and an old-fashioned oven. (3)

In 1930, Baker and Thompson picked up where their sisters left off and provided information about the exterior of the North Building. They told of the vegetable garden that was situated just outside of the main house, next to the kitchen door. On the opposite side of the main house was a flower garden. The sisters recalled precisely which flowers and trees were planted there, and they noted the exact placement of each and every one: “The arbor in the center is red and pink honeysuckle with yellow jasmine over it. Along the sides of the garden were rows of roses. In the center were two large magnolia trees, white and pink.” (2) They also said that while the Union forces occupied Arlington House, fallen U.S. soldiers were brought to the estate and buried next to the flower garden.

The details provided by the Gray daughters about the layout of the main house were even more specific than their memories of the North Building. They remembered that, positioned directly beneath the north wing of the mansion, next to the wine cellar and the furnace rooms, there was a washroom where all of the Lee clothing and household textiles, such as curtains, tablecloths, napkins, sheets, and blankets, were laundered before being hung outside to dry. (Syphax and Wilson, 3) Under the south wing was the dairy house where they kept milk in a deep well in the center of the room. (Syphax and Wilson, 3) In addition, the sisters disclosed that one of the jobs required of the slaves who worked in the dairy house was to churn fresh butter every morning for the daily meals. Their knowledge of the workrooms, the areas of the mansion most frequented by the Lee slaves, should not be surprising. However, the sisters had had access to the Lee family’s living areas as well.

The sisters began by describing the public rooms, which were located on the first floor. Baker and Thompson remembered vividly that a pair of antlers hung in the main hall above the front door and that paintings of battles adorned the walls of the front room. (2) In the rooms to the left of the hall were fireplaces that provided heat to the rest of the house. According to Baker and Thompson, carrying firewood into the house was a constant and often tedious chore for the slaves. (3) They stated that to the right of the hall was the parlor, where guests were received. Just beyond the parlor was a bedroom and the walls throughout the house were painted a  “yellowish shade.” (3) Directly in front of the main entrance in the hall were steps leading upstairs. The women provided the exact layout of the second floor, which included the southwest chamber used by General and Mrs. Lee and the rooms of all of the Lee children. They even described the placement of the Lee children’s beds. (Syphax and Wilson, 5) By supplying these details and more, Selina’s daughters played as crucial a role in the preservation of Arlington House as their mother had years before.


The legacy that began with Selina Gray and continued through the lives of her children remains a source of inspiration. The transition from slavery to freedom was difficult for all African Americans, and those living near the nation’s capital faced special challenges. Like other newly emancipated slaves, the Grays continued to reside near their antebellum home. Unlike most freedmen, however, the members of this family had been owned by people of unusual historic significance--the Lees, the Custises, and the Washingtons. The Grays understood this fact, but they also understood their own place in history and acted accordingly to preserve it. In the process, they contributed to the distinctive black heritage of Arlington, Virginia.

Works Cited

“Arlington County Historical Notebook.”  Vertical File, Biography – Gray, Virginia Collection, Arlington County Central Library, Arlington, VA.

Baker, Annie  and Ada Thompson.  Personal  Statement.  5  March  1930.  Vertical File, Arlington House Library, Arlington, VA.

Benjamins,  Cynthia R.  “Detached Italianate-Victorian Townhouse.”  The Journal (20 Apr. 1990).

Byrne, Karen.  “The Remarkable Legacy of Selina Gray.” CRM 4 (1998): 20-22.

Department of Community Affairs Planning Division.  Historic District Plat, the Harry W. Gray House. Vertical file, Biography – Gray, Virginia Collection, Arlington County Central Library, Arlington, VA. 

Gillem, Martha Gray, George H. Mitchell, Jr. and Cecelia G. Mitchell.  “Deed  of Gray Property, Lot 50B of Gray’s Subdivision.” Arlington County, VA, Deed Book 1882, page 203, 13 Dec.  1976. Vertical File, Biography – Gray, Virginia Collection, Arlington County Central Library, Arlington, VA.

Gillem, Martha Gray.  Interview.  1963.  Vertical File, Biography – Gray, Virginia Collection, Arlington County Central Library, Arlington, VA.

Gillem, Martha Gray.  “Last Will and Testament of Martha Gray Gillem.”  23 Feb.  1952. Vertical File, Biography – Gray, Virginia Collection, Arlington County Central Library, Arlington, VA.

Gray, Harry W.  “Last Will and Testament of Harry W. Gray.”  No. 151.  1 Oct. 1913.  Vertical File, Biography – Gray, Virginia Collection, Arlington County Central Library, Arlington, VA.

Gray’s Subdivision and Harry W. Gray House.  Map.  Arlington County Historical District Notebook, Vertical File, Biography – Gray, The Virginia Collection, Arlington County Central Library, Arlington, VA.

Harry W. Gray Estate.  Map of gardens.  Arlington County Historical District Notebook, Vertical File, Biography – Gray,  The Virginia Collection, Arlington County Central Library, Arlington, VA.

“Harry W. Gray House.”  Arlington VA Historical Society  24 Nov. 2002

“Historic Arlington Days, Oct. 9& 10.”  Northern Virginia Sun  (30 Sept. 1982): 8. “Inventory of the Slaves at Arlington Belonging to the Estate of G.W.P. Custis.” Alexandria Court Records, Will Book (7 January 1858).  Vertical File, Biography – Gray, The Virginia Collection, Arlington County Central Library, Arlington, VA.

National Park Service.  “Welcome to the Robert E. Lee Memorial”  (28 Nov. 2000).  Arlington House History <> (24 Nov. 2002).

National Park Service.  The African American Story at Arlington House: The Gray Family.  Vertical File,Arlington House Library, Arlington, VA.

National Park Service. Arlington House, The Robert E. Lee Memorial: Official National Park Handbook.  Washington, DC: GPO, 1985.  Vertical File, Historical Buildings Arlington House, Virginia Collection, Arlington County Central Library, Arlington, VA.

Nelligan, Murray H.  The Story of the Robert E. Lee Memorial. Burke, VA:  Chatelaine Press, 2001.

Netherton, Nan and Ross.  Arlington County in Virginia: A Pictorial History.  Norfolk, VA:  The Donning Company, 1987.

Ramirez, Constance, Chairman, to Cdr. Richard Storwick (28 Mar.  1983). Vertical File, Biography – Gray, The Virginia Collection, Arlington County Central Library, Arlington, VA.

Silberfarb, Marcia, Historic Preservation Coordinator, to Cdr. and Mrs. Richard Storwick (29 Oct. 1987). Vertical File, Biography – Gray, The Virginia Collection, Arlington County Central Library,  Arlington County, VA. 

Syphax, Emma and Sarah Wilson.  Personal Statement (6  Dec.  1929).  Vertical File, Arlington House Library, Arlington, VA.

Life of the Gray Family at Arlington House

The Harry W. Gray House in Arlington, VA: A National Historic Landmark

Restoration of Arlington House 


Works Cited


Selina's House

Martha Gray Gillem