The daughter of Abraham Rodriguez Brandon and Sarah Esther Lopez Brandon, Sarah Rodriguez Brandon was born on the island of Barbados in 1799. Her family had grown wealthy from the sugar trade that dominated the Barbadian economy and they maintained ties—both religious and mercantile—throughout the Atlantic world, especially with New York. Most of the Brandon children would leave Barbados, and both Sarah and her brother Isaac would marry children of Isaac Moses of New York, she to Joshua in 1817, and her brother to Lavinia seven years later. Together Sarah and Joshua had ten children, including a set of twins. One of the twins, Isaac Moses, Jr. would serve as a high-ranking Union officer in the Civil War.