This paper compares Frege’s philosophy of mathematics with a naturalistic and nominalistic philosophy of mathematics developed in Ye (2010a, 2010b, 2010c, 2011), and it defends the latter against the former. The paper focuses on Frege’s account of the applicability of mathematics in the sciences and his conceptual realism. It argues that the naturalistic and nominalistic approach fares better than the Fregean approach in terms of its logical accuracy and clarity in explaining the applicability of mathematics in the sciences, its ability to reveal the real issues in explaining human epistemic and semantic access to objects, its prospect for resolving internal difficulties and developing into a full-fledged theory with rich details, as well its consistency with other areas of our scientific knowledge. Trivial criticisms such as “Frege is against naturalism here and therefore he is wrong” will be avoided as the paper tries to evaluate the two approaches on a neutral ground by focusing on meta-theoretical features such as accuracy, richness of detail, prospects for resolving internal issues, and consistency with other knowledge. The arguments in this paper apply not merely to Frege’s philosophy. They apply as well to all philosophies that accept a Fregean account of the applicability of mathematics or accept conceptual realism. Some of these philosophies profess to endorse naturalism.