When established, late in the 12th century, Neath Castle, or Castell Nedd, overlooked the River Neath just upstream of its tidal lower section, controlling the first practicable crossing point, and it was occupied for the next 500 years. The river has since changed course somewhat, flowing 300 feet further north, and the castle now has a rather incongruous setting right beside a Morrisons supermarket, built here in 1988 after removal of many closely-spaced cottages which had surrounded the site since the 1800s; the curtain wall during this time formed the rear boundary of many of the dwellings.
The castle has a D-shaped outline incorporating 300 feet of curtain wall that encloses a relatively small central courtyard, flanked by a single tower at the northeast corner, surviving only at ground floor level, and a pair of towers either side of the main entrance at the northwest corner, still the original three stories tall, at least for the external walls, as the inner walls are largely missing. Access to the courtyard is not currently permitted owing to unstable masonry so only the exterior can be viewed in detail. The frontage adjoins a small park, while the remainder faces modern residential streets.
The first fortification beside the river crossing was built by the Romans - the fort of Nedum, about which little is known, and its position was only established by excavations in the 1950s - the fort lies partly on the playing fields of the local grammar school, on the west side of the river. The site was reused by the Normans, under Richard de Granville, who constructed a timber motte and bailey castle here around 1130, at the same time as the foundation of Neath Abbey, half a mile west. This castle was joined a few decades later by the present structure, built on a slightly elevated site on the opposite (east) side of the river, the initial occupier being Gilbert de Clare, Earl of Gloucester, and this proved to be better located, resulting in the earlier castle being dismantled, around 1207. The new building was strengthened, using stone instead of timber, by Gilbert's son Richard following Welsh attacks early in the 13th century, then further enhanced in the early 1300s, after which the castle, while not involved in any significant military campaigns, served as the westernmost stronghold of the Lords of Glamorgan for several centuries. Its function gradually transferred from defense to civil governance, until abandonment in the 18th century after which the castle became ruinous. Since this time, the only significant repairs were carried out the 1960s.
The front of the castle, to the west, is accessed via a path from the Morrisons carpark, while two roads run alongside the other sides. Entrance was via an imposing arched gateway between two round towers, both rising to three stories; the northern tower has a basement. Below the entrance is another basement, with steps down at the far side and a wall across the centre; this is a relic from an earlier phase of the building, a former postern gate, as latterly, access was through a passageway above. The twin-towered gatehouse was built in the 14th century, replacing an earlier, single northwestern tower, and its outer walls are almost complete, lacking only the upper crenellations. A short outer wall section on one side of the entrance is thought to be a fragment of the original town walls, the remainder having been completely removed. The northeast tower was similar in size and shape to the others, though older, mid 13th century, and also had a basement, plus an unusually large latrine turret. This tower is now only about ten feet tall, similar to the majority of the curtain walls. Within the courtyard are low foundations from five chambers, uncovered during recent excavations, and also a circular cock pit, this a much more recent feature, installed in the 18th century. The curved section of the curtain walls are mostly intact while the straight portion is largely modern, and faces another walled area at lower elevation, once a Quaker burial ground. The courtyard was formerly entered via a breach in the south wall, through a 19th century gate, but this is now kept locked.