Nanxuan Academy was built based on the ancestral temple of Zhang Jun and Zhang Shi, two famous scholars in South Song Dynasty. Zhang Shi is known as Mr. Nanxuan. The academy adopts the architectural structure of a traditional academy and is a closed quadrangle courtyard.

Along the central axis of the academy is the ancestral hall of Zhang Jun and Zhang Shi (father and son), where there are the busts of the two scholars. The academy has a cultural exhibition room, lecture hall, conference hall, library, etc. The exhibition room, which is set up by Yuelu Academy, focuses on Zhang Jun's and Zhang Shi's life, showing the development of Nanxuan Thought and its influence on later generations.

Backed by the Luodai Mountain, the Nanxuan Academy is picturesquely situated on the bank of the Weishui River, presenting a splendid and wonderful scene.