An early critic of anti-racist mathematics was then-current UK Prime Minister Margaret Thatcher, who said in her address to the Conservative Party Conference in October 1987:

"Children who need to be able to count and multiply are learning anti-racist mathematics, whatever that is."

Critics claim anti-racist mathematics takes relativism and postmodernism to extremes. These critics assert that mathematics and science, especially hard science fields such as physics, chemistry, and biology, accurately reflect universals, as opposed to being cultural interpretations or social constructions. For example, some critics say that Newton's formulations of the laws of gravity are neutral formulations that apply regardless of cultural, regional or ideological context. With this view, critics suggest that hard science is different from ideology because it is based on the scientific method, essentially an extremely cautious means of building a supportable, evidenced understanding of our world.

Critics also note that traditional mathematics is not essentially western. Middle-Eastern, Arab, Persian, and Indian mathematicians and astronomers from every era have made vast contributions to mathematics. The most concrete acknowledgement of this fact is the use of Hindu-Arabic numerals throughout mathematics. Indian mathematicians are also widely recognized as first developing the concept of the zero in the Old World.

Critics claim that the anti-racist approach to teaching mathematics reduces children's mathematical abilities on standardized tests. For example, controversy regarding anti-racist mathematics arose in January 2005 when critics blamed the plummeting mathematics scores at the Newton Public Schools in Newton, Massachusetts on its prominent anti-racist education principles. Critics believe these values were emphasized over the teaching of math skills. However, others dispute there is actual evidence to suggest Newton is practicing "anti-racist mathematics" and pointed out that Newton's eighth-graders demonstrated improved mathematical abilities on standardized tests.