Born enslaved to rice planter Plowden Weston, Anthony was a millwright who amassed a small fortune rebuilding and repairing rice mills. Despite his own continued enslavement, Anthony and his wife purchased 20 people, including skilled artisans, to work for them.