On 17 April 1555, the first furrows were opened and the first stone was laid by the first founding Bishop, Saint Francisco de Borja, Commissary General of Spain. The deed of foundation dates from 12 November 1555. The site chosen for the building was the north-western part of the city, bordering the old fortress. The plan of the College is shown as a hammer-shaped complex of buildings between the remains, fortress, walls and streets as well known in Plasencia as Avenida Juan Carlos I, Santa Ana. The work lasted until 1556. Masonry materials dominated in the College. The college was grouped around an inner courtyard with a double floor and openings arranged with paired columns in the Serlian style, which gave on to the dormitories, library, refectory, kitchen and cellars.

On the northwest side there was a large courtyard or vegetable garden, bounded by the fence and the Torre Lucía. The builder of the College was Father Bartolomé de Bustamante, while the master stonemasons and carpenters were residents of Plasencia, trained in the construction of the New Cathedral.

In its early years, Don Gutierre Vargas de Carvajal, Bishop of Plasencia, established a true educational centre in Plasencia. Jesuit rectors governed the school, proof of which is the extraordinary library in which books were kept which have been considered over the years to be unique pieces in Spain, due to their great cultural and scientific importance.

In 1767, Charles III expelled the Jesuits from Spain. Most of their colleges became charitable institutions. In the Colegio de Plasencia, a Hospital and Foundling Home of the Bishopric of Plasencia was established. In 1928, the Antigua Casa de la Compañía was used as a health centre until the construction of the new Psychiatric Hospital.