During the Hundred Years’ War, the Treaty of Brétigny offered Montauban to the English. Construction began on a fort on the banks of the Tarn, while the left bank remained French.
In 1664, Bishop Pierre de Bertier decided to build his new episcopal palace on the ruins of the old fort. After the French Revolution, the palace became the town hall and home to the first municipal museum and drawing school.