The centralized system in ancient China originated from patriarchal cooperative agriculture in the pre-national era. The centralized system of government was established in the Xia, Shang, and Zhou “kingship power” era. In the “imperial power” era from the Qin Dynasty to the Qing Dynasty, the authoritarian centralized system with imperial power as the core was increasingly strengthened. The evolution from “kingship power” to “imperial power” was an inevitable trend of China’s historical development, and the centralized power system had also been strengthened. As a dominant ideology, the thought of “Great Unity” provided a theoretical cornerstone for the centralized system in ancient China; the centralized system in ancient China provided the institutional guarantee for the official status of the idea of “Great Unity.” The system of centralization has been continuous in Chinese history, perfectly adaptable to Chinese traditional society, and interchangeable with the idea of “Great Unity,” providing an ideological basis and institutional guarantee for the formation, continuation, and development of a unified multi-ethnic country in China. The thought of “Great Unity” and centralization are the common values of the Chinese nation, which can provide useful reference for the current road of modernization.