David I Cohen was the eighth of ten children born to Israel and Judith Solomon Cohen. His father, a merchant from Bavaria, and his mother, born in Bristol, had settled in Richmond, Virginia, in the 1780s, and it was there that David was born. When he was only three, his father died, and five years later Judith moved the family to Baltimore. His brothers, led by the eldest, Jacob, achieved great success in finance, their firm, Jacob I. Cohen & Brothers, one of the only banks to survive the Panic of 1837. David and another brother, Benjamin, were among the founders of the Baltimore Stock Exchange.

In 1830 David married Harriet Cohen, daughter of Welsh Jews Jacob and Catherine Cohen in their hometown of Swansea, although how this match came to be is something of a mystery. They had seven children, including a daughter Bertha. They raised their family in Baltimore, where the Cohens were active in Jewish civic life.