Inadequate geometric accuracy of cameras is the main constraint to improving the precision of infrared horizon sensors with a large field of view (FOV). An enormous FOV with a blind area in the center greatly limits the accuracy and feasibility of traditional geometric calibration methods. A novel camera calibration method for infrared horizon sensors is presented and validated in this paper. Three infrared targets are used as control points. The camera is mounted on a rotary table. As the table rotates, these control points will be evenly distributed in the entire FOV. Compared with traditional methods that combine a collimator and a rotary table which cannot effectively cover a large FOV and require harsh experimental equipment, this method is easier to implement at a low cost. A corresponding three-step parameter estimation algorithm is proposed to avoid precisely measuring the positions of the camera and the control points. Experiments are implemented with 10 infrared horizon sensors to verify the effectiveness of the calibration method. The results show that the proposed method is highly stable, and that the calibration accuracy is at least 30% higher than those of existing methods.