The road south from Oranjestad ends abruptly at Fort de Windt, where a couple of rusty cannons sit atop a cliffside stone wall. While there's not much else to this small 18th-century fort, you'll be rewarded with a fine view of St Kitts to the southeast. The most interesting geological feature in the area is the white cliffs to the east of Fort de Windt, a landmark readily visible from neighboring islands.The fort is just southeast of Oranjestad. To get there, take the road that runs past the old Dutch Reformed Church and follow it south, through a dry terrain of cacti and stray goats, to its end 3km (2mi) away.

Oranjestad, the island's capital and its only town, is a pleasant place with a fine sense of history. Located along the waterfront, Lower Town was the location of the original port town and still has some ruins from the colonial era, as well as the island's best beach and its harbor. Upper Town is Oranjestad's main commercial and residential area.Right in the center of town, Fort Oranje is an intact 17th century fort complete with cannons, triple bastions and a cobblestone courtyard. It's perched on the cliffside directly above Lower Town and offers a broad view of the waterfront below. The Sint Eustatius Museum, operated by the Sint Eustatius Historical Foundation, gives visitors a glimpse of upper-class colonial life on Statia and is one of the region's finest historical museums. It occupies the Simon Doncker House, a restored 18th-century Dutch merchant's home with collections of nautical artifacts, china and hand-blown bottles. The basement, formerly a wine cellar, houses the museum's pre-Columbian collection.The Honen Dalim, an abandoned synagogue that dates from 1739, is the second oldest in the western hemisphere. About half a kilometer east of the synagogue ruins is a Jewish cemetery with gravestones dating from 1742 to 1843. The thick stone walls of the old Dutch Reformed Church, built in 1755, remain perfectly intact, but the roof collapsed during a 1792 hurricane and the building has been open to the heavens ever since. You can climb the steep steps of the church tower for a good view of the surrounding area.Although high seas and hurricanes have taken their toll on Lower Town's historic waterfront, the remains of the old foundations from some of the 18th-century warehouses that once lined the coast can still be seen jutting into the water along the shore. Submerged sections of the old seawall that once protected the harborfront can be explored by donning a mask and snorkel.