The two oldest buildings in Abergavenny, constructed shortly after the Norman invasion, are the castle and the Benedictine Priory, this latter established by Hameline de Ballon in 1087 and dedicated to St Mary. St Mary's Priory flourished for over four centuries without major incidents, while never becoming very prosperous, until closure in 1536 as part of the Dissolution of the Monasteries, after which the church was given to the parish, and it still stages regular services today. Although greatly modified, in particular during the 1800s, medieval stonework does survive in a few places, however the main point of interest is the fine collection of effigies and tombs, from the 14th to 16th centuries, mostly for members of the ap Thomas and Herbert families. One other building from the monastic era is the priory's tithe barn which has also been in continuous use since the 12th century, and currently houses a museum and exhibition venue.
The church and the barn have a south central location in Abergavenny, 700 feet north of the castle, and just east of the ancient town walls, of which virtually no trace remains. The church is open for regular visitors between 9 am and 4:30 pm, and there is usually a volunteer in residence. The nearest car park is along Castle Street.
The north side of the church adjoins the graveyard, in which are found a few minor wall fragments from some of the subsidiary monastic buildings, demolished soon after the Dissolution, while the south side links with the Priory Centre, a relatively modern building. The church itself consists of the nave and north aisle (the south aisle having long since been removed), leading to the crossing and choir, set between two chapels; the Lewis chapel on the north side and the Herbert chapel to the south. The chapels and north aisle incorporate the north and south transepts of the original, smaller church. Rising above the centre is a square tower, one of the few surviving medieval sections; almost all the visible stonework is from the Victorian alterations though the two chapels also have some late Norman components. The interior is relatively plain, and most of the windows contain clear glass, however the church is unusually large and impressive, and has been described as the 'Westminster Abbey of Wales' owing to its lofty ceilings and the collection of monuments. Stained-glass is present in four windows, north and south of the crossing, at the east end of the chancel (above the altar) and at the east end of the Lewis chapel; all are recent, the oldest installed in 1922. The architectural style is mostly Perpendicular Gothic. Besides the monuments, other ancient components are limited to a stone font - actually earlier than the church, probably mid 11th century, and the wooden stalls in the choir, which are from the 15th century.
Most of the ancient monuments are in the two chapels, the main exception being the wooden effigy of Baron John Hastings (died 1324), which is in the north transept. Below the east window of the Lewis chapel is the famous Jesse sculpture, just one part of a huge oak carving from the 15th century, depicting the father of King David. Close by is the tomb of Dr David Lewis (died 1584), the first principal of Jesus College in Oxford, and also two much older stone figures, one unidentified, the other of Eva de Braose (died 1255), daughter and coheiress of the Marcher Lord William de Braose. The Herbert chapel contains three ornate, alabaster tombs and effigies of notable couples; Sir William ap Thomas (died 1445; founder of Raglan Castle) and Lady Gwladys, Sir Richard Herbert of Coldbrook (died 1469) and Margaret, and William Baker (died 1648) and Joan. Also in this chapel are moments to Sir Lawrence de Hastings, Lord of Abergavenny (died 1348), and Sir Richard Herbert of Ewyas (died 1510), grandson of Sir William.