James Latimer was born in the north of Ireland. In 1736, as a young man, Latimer immigrated with his family to Chester County, Pennsylvania. Later he entered the shipping business in Philadelphia, sailing twice a year to his native country. After his marriage in 1749, Latimer moved to Newport, Delaware, where he operated flour mills and shipped grain and produce to markets in Philadelphia and the West Indies.
Latimer served with a Delaware regiment during the French and Indian War and was a lieutenant colonel of militia during the Revolutionary War. He was a member of the council of safety for New Castle County in 1775 and for the state during the winter of 1776-77, when he provided aid to General George Washington's army encamped on the Delaware River. Latimer was later elected a justice of the court of common pleas and the orphan's court of New Castle. In 1787 he presided over Delaware's convention to ratify the U.S. Constitution. Latimer gained national prominence when Delaware became the first state to act on behalf of the document.