The park was named after Fort Dummer, the first permanent white settlement in Vermont. Built on the frontier in 1724, it was initially the gateway to the early settlements along the banks of the Connecticut River. Forty-three English soldiers and twelve Mohawk Indians manned the fort in 1724 and 1725. Later, the fort protected what was then a Massachusetts colony from an invasion by the French and Indians. Made of sturdy white pine timber, stacked like a log cabin, Fort Dummer served its purpose well.

The park overlooks the site of Fort Dummer which was flooded when the Vernon Dam was built on the Connecticut River in 1908. This site can be seen from the northernmost scenic vista on the Sunrise Trail. It is now underwater near the lumber company located on the western bank of the river.