1106 sees the earliest recorded reference to a castle at Newcastle It was built by the Lord of Glamorgan himself, Robert FitzHamon. It was strategically built on a high bluff above the Ogmore Valley to guard the river below. An earlier earth work castle of ring work type, is thought to have been situated where the present castle sits. The Castle was probably rebuilt in stone during 1180s, when the king himself, Henry II, held the castle. The Castle boast of a complete Norman doorway which greets the visitor on their approach.
“It is late 12th-century, contemporary with the curtain wall. On the inside it is quite plain, but the outside is given fine decorative treatment. Once inside the curtain wall, the circuit of which is complete, the nature of the castle becomes apparent. It is a courtyard castle, roughly circular in plan, with two mural towers built into the curtain wall on the south and west sides. The curtain wall, which was built in straight sections, is impressive and stands to its full height on the west side.
The square mural towers were a new development in military planning when built, but were soon to be superseded by round towers. The south tower is the better preserved, standing in parts to three storeys high. It was much altered for domestic use in the 16th century, when Tudor windows and fireplaces were inserted. Only the ground floor of the west tower survives. Very fragmentary foundations of a detached building at the north end, and the more complete foundations of two buildings against the east curtain wall are visible.”
The Castle and lands were later given to the de Turberville family in 1217. The castle and lands of Newcastle became part of the the Margam Estate when it was bought by Sir Rice Mansel of Margam during 1536.