Born and raised in Philadelphia, Henry Lazarus was one of two children born to Esther Lyons and Isaac Lazarus. In 1835 he married his first cousin, Merriem Moss, daughter of John Moss and Rebecca Lyons. They had one child, George Gaines Lazarus, born in 1836.

Depicted beside him Lazarus here is a figure recorded as Isaac P. Jacobs, about whom nothing is known.