A landmark overlooking the town of Vaison-la-Romaine, the count's castle was built by Raymond V, Count of Toulouse and lord of Comtat Venaissin in the 12th century. The monument originally consisted of a wooden tower erected in 1185 to assert the count's power. During the 13th century, it was transformed into a fortified building consisting of three bodies and a 42.6 ft (13 m) high square stone keep surrounding an interior courtyard. The count's castle of Vaison-la-Romaine became a papal possession in 1274 and remained one until 1791.