Altered three-dimensional architecture of chromatin influences various genomic regulators and subsequent gene expression in human cancer. However, knowledge of the topological rearrangement of genomic hierarchical layers in cancer is largely limited. Here, by taking advantage of in situ Hi-C, RNA-sequencing, and chromatin immunoprecipitation sequencing (ChIP-seq), we investigated structural reorganization and functional changes in chromosomal compartments, topologically associated domains (TADs), and CCCTC binding factor (CTCF)-mediated loops in gallbladder cancer (GBC) tissues and cell lines. We observed that the chromosomal compartment A/B switch was correlated with CTCF binding levels and gene expression changes. Increased inter-TAD interactions with weaker TAD boundaries were identified in cancer cell lines relative to normal controls. Furthermore, the chromatin short loops and cancer unique loops associated with chromatin remodeling and epithelial–mesenchymal transition activation were enriched in cancer compared with their control counterparts. Cancer-specific enhancer–promoter loops, which contain multiple transcription factor binding motifs, acted as a central element to regulate aberrant gene expression. Depletion of individual enhancers in each loop anchor that connects with promoters led to the inhibition of their corresponding gene expressions. Collectively, our data offer the landscape of hierarchical layers of cancer genome and functional alterations that contribute to the development of GBC.