Jacob Denver Gilliard was born in 1828, in Baltimore Maryland. His father, Nicholas a former slave and his mother Catherine Gilliard. Nicholas was a prominent figure in the Baltimore Methodist Community, even going as far as helping to purchase land for its church in the city. Nicholas Gilliard's father was named Jacob Gilliard as well. The grandfather Jacob helped to build the Gilliard dynasty within the Baltimore community. The family's money came from a blacksmith business started by the grandfather Jacob. It was from this steady stream of income that Jacob was able to buy the freedom of his two enslaved sons Jacob jr. and Nicholas; as well as helping to free another man Henry Williams. In the time after Jacob Sr. was paying off the price for his sons Jacob jr. attempted to flee, in return his grandfather was forced to put out an advertisement asking for slave catchers to return his son to him.
Within the Baltimore black community, one important pillar was the Methodist church. Freed blacks in Baltimore looked to the Methodist church. The Methodist church gave the people of Baltimore a way to lift themselves up and springboard them into a higher social status. Jacob Sr. himself went to weekly classes to help teach himself how to read and write.