Moses was, together with his twin brother Isaac Jr., the second of of nine children born to Joshua Moses and Sarah Rodriguez Brandon Moses, a group well represented by a series of daguerreotypes in the collections of the American Jewish Historical Society. Moses’ siblings included Israel, Lionel, Benjamin, and Joseph. Moses’ father was a merchant and the son of German immigrant Isaac Moses, and his mother the daughter of Sephardic Merchant Abraham Rodriguez Brandon.