The park surrounding Fantaisie Palace, created by a succession of owners, combines design elements from various style periods: the rococo period, the sentimental landscape period and the mixed style period.

Three years after the death of his wife Wilhelmine, Margrave Friedrich began building a new palace, but did not live to see it completed. His daughter Elisabeth Friederike Sophie inherited the property in 1763 and had a late baroque garden laid out from 1763-1780. Remaining from this time are the pavilion, probably designed by Carl Philipp von Gontard, the Neptune Fountain with a group of figures by the Räntz brothers, as well as retaining walls, flights of steps and two original sandstone seats.

The 18th-century hedge rooms originally located east of the palace and the former kitchen garden with its fruit trees were reconstructed from 1993 to 1996. The cascade, of which only fragments remained, was reconstructed in its magnificent original form from 2003 to 2005.

In 1793 Duchess Friederike Dorothee Sophie von Württemberg inherited the palace. The duchess incorporated the surrounding countryside into the park by creating numerous attractive buildings and garden scenes using the existing sandstone rocks: a catacomb, the Pillar of Harmony and various seats. Her son Alexander Friedrich was an officer and unable to devote much time to his inheritance. The only features he added were the Borodino Rock and the Alexander Chapel.

It was not until his son Duke Alexander of Württemberg that Fantaisie reached new heights. The new owner used the palace as a summer residence and had it rebuilt in 1850/1852 in its present form. He decorated the north garden area in front of the palace with carpet beds, treillages and sculpture. South of the palace he laid out pleasure gardens with areas of lawn and flowerbeds, and turned the slope in front of the building into a vineyard. The most important alteration he made was the extension of the older garden with a landscape park containing many exotic trees and shrubs.