In his lifetime, Ball was enslaved in Maryland, Georgia, and South Carolina, and also worked as a cook. He recalled Black people having their own gardens in secluded areas on the plantation where they grew corn, potatoes, pumpkins, and melons for themselves.