In the early Ming Dynasty, this palace was built to be the Residence of Chen Li and the Residence of Zhu Gaoxu. During the Qing Dynasty, the palace became the Office of Liangjiang Governor. The Emperors Kangxi and Qianlong also used it as their temporary imperial palace when they had tours around Jiangnan region.

During the ruling of the Taiping Heavenly Kingdom, Hong Xiuquan constructed the Heavenly King’s Palace on a large scale. This palace became the Office of Liangjiang Governor again after the failure of the Kingdom.