The first castle at Totnes, beside the lowest fordable location on the River Dart, was a wooden structure erected soon after the Norman conquest, and this endured for around 100 years until replaced by a stronger, stone fort, probably under the command of William de Braose, the third Lord of Bramber. The fort had the traditional motte and bailey design, with a square keep on top of a conical hill of man-made origin, and a walled enclosure (the inner bailey) at the base, containing subsidiary buildings, all enclosed by a water-filled moat. Beyond, to the northwest, was the less-protected outer bailey, now the site of a park.
Totnes Castle was never particularly important and had only limited involvement in conflicts, so it was never significantly expanded, though the keep was replaced in the 1320s after the castle had passed to the de la Zouche family, with a sturdier design using the local sandstone and limestone. Hence the castle provides one of the best examples of the early Norman style of construction, being mostly complete and not subject to the major alterations that were carried out on most other fortifications from this period. About half of the moat remains, now dry, plus the majority of the curtain wall, and all of the walls of the later shell keep, which was much larger than the original, and circular rather than square in outline, with a diameter of 80 feet. Other remains are limited to the stone foundations of the earlier keep tower; there is no trace of the subsidiary buildings in the lower bailey.
The castle is high enough to be seen from many places in the centre of Totnes though is not so easy to reach, being surrounded by narrow streets, some one-way. The closest parking area is just few minutes walk away along North Street. The site is managed by English Heritage, open daily between April and October, and is perhaps a little overpriced, as the site is small and all can be seen half an hour or less. The castle is not particularly photogenic, and the nearby trees and buildings make it hard to obtain a good overall view of the site. Entry is up a short alley on the north side of the castle, past a fragment of the old town walls, and then the entrance booth, to the inner bailey, a grassy area are now used for picnics, beneath the shade of a few large, ancient trees. The bailey is enclosed by a mostly complete, curving wall about 500 feet long, with a modern opening on the north side, above the surviving section of the moat. The two south ends of the wall climb the motte and link with the keep. The south and east sides of the motte, which are crossed by a series of embankments, appear to be inaccessible to the public.
A stepped path ascends the north face of the motte to the gated doorway of the keep, allowing access to the circular interior, which contains the earlier tower foundations but is otherwise empty, though six corbels projecting from the wall to the northwest indicate the previous existence of an interior room. Either side of the doorway, inside the walls, are two short series of stone steps leading to the wall walk, which extends a full 360 degrees around the top of the keep, protected on the inside by modern railings. The battlemented walls survive to their full height, and the regular gaps allow commanding views across the River Dart valley. Some of the upper sections of the walls are pierced with arrowslits, some cross-shaped, others merely vertical. The only other feature inside the keep is an enclosed room, or garderobe, contained within the southern wall, illuminated by two arrowslits, one having been enlarged to make a small window.