Rousay has a rich archaeological heritage. Along the west road from the pier there are four well-preserved, perhistoric burial cairns: Taversoe Tuick - an unusual two-storied cairn, Blackhammer and Knowe of Yarso - both stalled cairns, and Midhowe - the largest known stalled cairn, which is protected in its own special building. Close at hand is Midhowe Broch, one of three fortified dwellings built by the Picts along the shore of Eynhallow Sound, during the Iron Age.
Between Westness Farm and Midhowe, there is a Viking cemetery, the remains of a Viking farm, the remains of the parish church of St. Mary's and the deserted farm buildings of Skaill, where Earl Paul Hakonson was kidnapped.
Further north, the wild and open area of Quandal was cleared in the 19th Century to make way for sheep as part of the laird's agricultural improvements. Ruined farmsteads, turf dykes and the house of Tofts, which is probably the oldest two-storied house in Orkney, are still visible.
North of Quandal, on the Loch of Wasbister, lies Burrian, a crannog. This artificial island was created in the Iron Age as a protected loch dwelling. Brettaness, a similar structure located at the edge of the loch was excavated in 1974, showing evidence of Pictish occupation and the possibility of an early medieval chapel.
Westness House was the mansion house of the lairds of Rousay, the Traill family, who came to Orkney with the Stewart Earls. The original house was sacked and burned down by the Hanovarian Forces when the owner was accused of supporting the Jacobite Rebellion in 1745. He later built the present house.
Trumland House is the large house immediately visible from the pier, It was built in 1873 by General Traill-Burroughs, following his marriage. It was designed by David Bryce in the traditional Scottish baronial style. The gardens and woodland were planted at the same time. The house and gardens are currently in the process of being restored by the present owners. The gardens are open to the public.