Part of James Madison Park occupies the place where the original plans for the city of Madison, made by James Duane Doty in 1836, provided for one end of a canal between Lakes Mendota and Monona. This canal was never dug, but a square of land on Lake Mendota bounded by Franklin, Hancock, and Gorham Streets remained. In the 1870's, the sixty-five foot long, one hundred fifty passenger steam yacht Mendota made daily trips around Lake Mendota from this location.
A little to the west, at the intersection of North Butler and North Hamilton Street, the Conklin & Sons ice house stored ice that was harvested from Lake Mendota. Though the city was willed $75,000 for a park on this site in 1916, that was not enough to buy the ice house property. (Apparently, the will had been drawn up after the old icehouse had burned down, and before a new one was built.) Eventually, the advance of mechanical refrigeration made icehouses obsolete. In 1939, the city purchased the Conklin property in order to develop it as a park providing lake access near the center of the city.5 "Conklin Park" was used regularly by Madison swimmers, but the park was considered too small to justify building a shelter.
The 1950's saw the purchase of additional land. In 1963, the park was given the name "James Madison Park," since the city of Madison had no public facilities specifically named after the fourth president. The city also made plans to purchase the remaining lakeshore between Butler Street and Lincoln School for the park. Federal aid made it possible to purchase much of the property in the 600 block of East Gorham St. during the 1960's.
In 1995, a lot was added to the park to improve access to the Bernard-Hoover Boathouse at 622 E. Gorham St. This building dates back to 1915, though Charles Bernard built the first boathouse at this site in 1855. In 1943, Harry Hoover bought the boathouse and operated it until 1968, when the city acquired the property. It was rehabilitated in 1992 and is now leased by the Mendota Rowing Club. It is on the National Register of Historic Places.
The Gates of Heaven Synagogue originally stood at 214 West Washington Avenue. It was built in 1863 and is one of the oldest surviving synagogue buildings in the country. It was designed by August Kutzbock, who also designed the second state capitol building, which burned in 1904. After the Ahavath Achim congregation sold the building in 1916, it was put to other uses, including as a funeral parlor, government storage facility, headquarters of the Women's Christian Temperance Union, a church, a dentist's office and a veterinary clinic. In 1970, it was threatened with demolition, and a Gates of Heaven Foundation was formed to save it and place it on the National Register of Historic Places. Jacking it up on 96 aircraft wheels and moving it to James Madison Park saved the building.
To the right of the synagogue, there is a monument to the 45,000 International Volunteers who fought for the Spanish Republic during the Spanish Civil War (1936-1939), including the 2800 Americans who served in the Abraham Lincoln Brigade. The monument includes the names of 37 Wisconsin volunteers. It was erected in 1999, and was only the second monument of its kind. (The first was in Seattle.)
The James Madison Park shelter was designed by Kenton Peters and erected in 1979.
The former Lincoln School building adjacent to James Madison Park was designed by the Madison architectural firm of Claude and Starck and built in 1915. It has been called the best remaining Prairie-style school building in Wisconsin. From 1964 through 1980, it housed the Madison Art Center and other arts groups. It is now a privately owned apartment building.