He was the son of Palatine German immigrants Catherine Petrie and Johann Jost Herchheimer (one of various spellings) from the German Electorate of the Palatinate living in German Flats in the Mohawk Valley in the Colony of New York, and the elder brother of Captain Johan Host Herkimer. He was of slender build, dark complexion, black hair, and not quite 6 feet tall. He could speak both German and Iroquois.
During the Attack on German Flatts in the French and Indian War, he was involved in its defense. He was made a captain in the militia on January 5, 1758 and he repelled a second attack on German Flatts in April 1758.
In July and August 1775 he headed the Tryon County Committee of Safety, and became Colonel of the district militia. After the split in which loyalist militia members from the area withdrew to Canada, he became a Brigadier General in the Tryon County militia by the Provincial Congress on September 5, 1776.
In June 1776, he led 380 men of the Tryon County militia to confront Joseph Brant at Unadilla. Herkimer requested that the Iroquois remain neutral while Brant said the Indians owed their loyalty to the King.
When he learned of the siege of Fort Stanwix to the west in late July 1777, he ordered the Tryon County militia to assemble at Fort Dayton. He then marched them out to Fort Stanwix, about 28 miles to the west. His force marching in column was ambushed on August 6 by a mixed force of British regulars, Tory militia, and Mohawk Indians in the Battle of Oriskany. Herkimer’s horse was shot, and he was seriously wounded. In spite of his injuries, he sat propped against a tree and lit his pipe. He rallied his men to avoid two panicked retreats, and when they withdrew they carried him home.
The brigade surgeon, William Petrie, dressed Herkimer’s wound in the field and placed him on a litter. The wound quickly became infected, but the decision to amputate the leg was delayed for about ten days after the battle. The operation was performed by an inexperienced surgeon, Robert Johnson, because William Petrie was wounded in this battle and was not available. The operation went poorly, the wound bled profusely, and Herkimer died of the injury on August 16 around the age of 49.
His nephew, John Herkimer, later became a U.S. Congressman.