The child of Mordecai and Leah Lazarus Cohen, David Daniel Cohen was born in Charleston. His father had arrived from Poland in 1788 and, initially working as a peddler, achieved great success in business, including real estate, agriculture, and slave trading. In 1830 David married Mary Hart, daughter of Nathan and Rachel Hart. Hers was also a prominent Charleston Jewish family, and among her siblings was Samuel Nathan Hart. Mary and David had six children whom they raised in Charleston.