This paper aims at a semi-dense visual odometry system that is accurate, robust, and able to run realtime on mobile devices, such as smartphones, AR glasses and small drones. The key contributions of our system include: 1) the modified pyramidal Lucas-Kanade algorithm which incorporates spatial and depth constraints for fast and accurate camera pose estimation; 2) adaptive image resizing based on inertial sensors for greatly accelerating tracking speed with little accuracy degradation; and 3) an ultrafast binary feature description based directly on intensities of a resized and smoothed image patch around each pixel that is sufficiently effective for relocalization. A quantitative evaluation on public datasets demonstrates that our system achieves better tracking accuracy and up to about 2X faster tracking speed comparing to the state-of-the-art monocular SLAM system: LSD-SLAM. For the relocalization task, our system is 2.0X∼4.6X faster than DBoW2 and achieves a similar accuracy.