Benjamin was the last child to survive infancy born to Joshua Moses and Sarah Rodrigues Brandon Moses. His siblings included Isaac Jr., Israel, Lionel, Abraham, and Joseph. Benjamin’s father was a merchant and the son of German immigrant Isaac Moses, and his mother the daughter of Sephardic merchant Abraham Rodrigues Brandon.