The rugged limestone hills rising sheer above the Tawe valley created an Alpine scene that sought sympathetic architecture.
Here it was that Captain Powell created his neo-gothic castle at Cae Bryn Melin Bach on the site above the river and close to his father's home.
Captain Powell used local limestone for the original building but the cost ran away with him and he was unable to complete the north wing. Turrets and four pyramids shaped the walls of the roof, and a recently completed road from Ystradgynlais to Brecon passed the door.
In 1843, the family moved in, and Captain Powell settled to the life of a country gentleman, becoming a county magistrate and a High Sheriff of Brecknock.
Although both he and his father married well, misfortune began to fall upon the family when cholera took his younger son in 1851. He suffered the loss of his wife and his daughter before he died in 1862.
Two years later, the eldest son was killed in a hunting accident on the Isle of Wight. Sarah, the eldest daughter, had married a Captain Allaway and the couple remained at the castle after the death of her brother, until tragedy struck once more when Captain Allaway died a few years later.