Leiter Reports: A Philosophy Blog

News and views about philosophy, the academic profession, academic freedom, intellectual culture, and other topics. The world’s most popular philosophy blog, since 2003.

  1. Brian Kogelmann's avatar
  2. RMD's avatar
  3. Textbook Author's avatar
  4. New Author's avatar
  5. J. McKenzie Alexander's avatar
  6. Terence Rajivan Edward's avatar
  7. JR Williams's avatar

“A Severe Misalignment of AI in Mathematics”

Brian Leiter

A statement by 25 Fields Medalists in mathematics. Some of their case applies, I would venture, to philosophy. An excerpt, but do read the whole statement:

[T]he push by AI companies to solve mathematical problems as a benchmark is detrimental to the science of mathematics, and to the mathematical community. The goals of the AI companies and the goals of the mathematical community are severely misaligned. We see these as part of broader alignment issues impacting other scientific and creative professions, as well as the whole of society….

The mathematical community functions, in many ways, as a miniature version of humanity. It consists of individuals using a wide variety of different approaches, joined by core values. The most precious resources of our profession are students and ideas, and these we nurture with great care. We feel responsible to let them grow to their full potential, until they can live a life of their own in the mathematical world. For students we often suggest problems with the core intention of developing skills making them well-positioned for advances in research and elsewhere. Our ideas we disseminate in talks, private discussions and careful writeups, connecting them to the previous ideas of others. These processes invariably take time and are based on human interaction….

[T]he success of AI in solving major mathematical problems has made headlines even outside mathematical circles. But solving problems is only a tool and proxy for achieving the primary goal of conceptual understanding and insight. Forgetting this in the world of AI may turn the tool against the primary goal. Indeed, the mass production at faster and faster pace of “true/false” statements could destroy fertile ground instead of breathing life into new ideas.

Often these solutions are announced in a rush, leaving no time for a proper writeup, the isolation of new methods and ideas, and citing relevant previous work of others. As in all creative professions, this raises severe attribution and plagiarism questions. Moreover, without the willing mathematicians who must take care of their development and integration into the mathematical canon, AI-conceived ideas would never become fully alive and the crucial human transmission chain between mathematicians would be lost.

We are witnessing a general threat to intellectual work, with misalignment between the outcome of the use of AI and its initial purpose. In many fields and activities, years of training have traditionally served not only to produce a final answer or product, but also to develop understanding and the ability to formulate new questions and ideas. However, building on a vast body of previous human work, AI systems are becoming increasingly capable of producing the results of such work directly, and these goals cease to align. The issues the mathematical community faces now are similar to issues that other scientific and creative professions are facing, and indicate issues that all of humanity might face: how to make sure that, as AI changes the way work is done, we do not lose sight of what that work was meant to achieve in the first place.

What do readers think about the application of these points to philosophy and other scholarly fields? Do they apply (if so, how; if not, why not)?

(Thanks to Michael Clune for the pointer.)

,

Leave a Reply

Your email address will not be published. Required fields are marked *

16 responses to ““A Severe Misalignment of AI in Mathematics””

  1. Daniel Muñoz

    I used to think it was a drawback of philosophy that we don’t have anything like Hilbert’s Problems—a list of open questions that everyone agrees are important.

    Now, I think it’s a valuable defense against AI encroachment. There’s less potential for “AI solves X”-type headlines in philosophy.

    Reply
    1. AG Tanyi

      I am no mathematician, but to me his problems or at least a good number of them read more like what the above quote refers to as ‘conceptual understanding and insight’. In fact, most of them does. The relevant list – and on which AI companies do train their systems – is Erdos’s problems.

      Reply
  2. Tamler Sommers

    I can’t get past “The mathematical community functions, in many ways, as a miniature version of humanity.”

    Reply
  3. Mark

    This issue – the difference between getting an answer from AI and actually understanding it – is why I started reading philosophy, and thus started haunting this blog’s comments…

    How far this will go, I’m not sure. I think other STEM fields are probably going to be effected, although it may take a while. The problem is collecting training data. This is relatively easy for math and CS, but considerably harder when an experiment requires manipulating things in the physical world. However, there’s a lot of work right now on automated wet labs and the like. I don’t have enough understanding (natch) of those fields to tell how successful these efforts will be, or over what time scale. Given enough effort, adequate training data can probably eventually be put together, but it may take a long time.

    Philosophy and other fields in the humanities and social sciences may, ironically, be more resistant to this intrusion than hard sciences. How would one collect training data for philosophy? What would that even mean, given that philosophy often doesn’t have a “right” answer? Plus, it’s not really a high priority for the frontier labs. They’re trying to automate coding because replacing software engineers will make them a lot of money. There’s not as much market for replacing philosophers.

    That said, the above analysis assumes we don’t see much domain generalization. If we train an AI to be a genius at math, how much of that “intelligence” transfers to making it better at tasks like philosophy? Probably some, but we don’t know how much. I don’t know of any studies that have tested domain generalization at the scale that would be needed to meaningfully answer the question. If I had to guess, I would guess we’ll see very little domain generalization of this sort, implying philosophy will be safe for some time to come, but as far as I am aware we really don’t know.

    So I think philosophy is probably safe for now, at least from this issue, but we really don’t know what the future is going to bring…

    Reply
    1. David Wallace

      It’s not just training data: it’s the existence of very formal ways of checking for correctness. Those exist in CS and in pure math: the existence of the ‘Lean’ formalization-of-proof system is pretty critical to the AI-aided math proofs. That kind of very formal check isn’t really even available in theoretical physics, let alone the less mathematized sciences or the humanities.

      Reply
      1. Mark

        Yes, I was using “training data” in a loose sense. But that’s why these companies are working on automated web labs: to allow automated validation of chemical and biological workflows. I don’t know enough about those fields to say if that’s a plausible project or not, but they’re certainly trying.

        Reply
  4. James C. Klagge

    I remember in grad school in 1976 Ian Hacking (visiting UCLA at the time) reporting that the 4-color theorem had been proven by a computer. He expressed reservations about this un-surveyable “proof”. I suppose that was a forerunner of this issue. But whatever “proof” procedures we have in philosophy, it is hard to imagine them taking on a recognition-transcendent life of their own, the way they have in mathematics.

    Reply
  5. AG Tanyi

    I think this is correct. On the other hand. If there ever will be a general AI, that presumably will be able to do such conceptual work and produce the relevant insight. Would that mean that that AI will be aligned with the goals and values of the mathematics (scientific) community? In other words, is this just a technical question, a matter of progress?

    Reply
  6. Luis Marxuach

    I should say first I was motivated to write something similar in the previous AI-related thread after reading the Guardian article I link to below but it may have come out muddled and hurried. Admittedly using a bit of AI ‘magic’ I have sought to make the concerns that came to mind as limpid as possible in the present version; none of the ideas were offered to me by an AI, however I do feel somewhat indebted to its organizing abilities.

    ‘Some mathematicians are still wary, however. “The big story now in mathematics is that nobody wants to share anything,” Buckmaster told the Guardian. “Mathematics is different today than it was only a few days ago. We have to decide what to do about that.”’

    https://www.theguardian.com/science/2026/sep/12/openai-mathematicians-millennium-prize-problem

    At this point — though perhaps it’s only transitory — AI seems to many, unsurprisingly, to be poisoning the well of knowledge as much as deepening it. But the harm worth naming isn’t theft. It’s epistemic, and it has two layers.

    The first is ordinary. The most influential public-facing LLMs are black-box proprietary products. Sharing research with them creates informational vulnerability of two kinds: aleatory variance from the engine itself, and something more directed from the other side of the interface — the risk Buckmaster gestures at when he worries about what an active scientist gives away by thinking out loud with these systems.

    The second layer is stranger, and it’s where philosophers are differently exposed — not to losing their lunch, but to something harder to detect: ideological framing, artificial epistemic limits, and a manufactured sense of depth.

    Call this a minor and a major epistemic Orwellianism.

    Minor Orwellianism is the ordinary condition of the medium. You cannot see the weights, the training distribution, the objectives, the system prompt. Your questions are quietly shaped by what the model will and won’t say. Depth is simulated. This is Orwellian in the weak sense — manufactured epistemic limits without a visible censor. No one is lying to you; the medium simply isn’t transparent.

    Major Orwellianism is the same opacity, but functional. Refusals that track liability rather than truth. Framing that tracks institutional interest. “Safety” that doubles as reputational management. Here the opacity serves someone — even if no individual is steering each output, and even if the actor doing the serving isn’t human.

    The distinction has a threshold: a beneficiary. Minor Orwellianism has none. Major Orwellianism does.

    Here is the difficulty. The standard defense of black-box opacity runs: the system is too complex for anyone to introspect, so opacity is a fact about the system, not a choice. Nobody is hiding anything; there is simply no one who could show it to you.

    That defense is sometimes true. Mechanistic interpretability genuinely cannot yet explain most model behavior. But the defense is also unfalsifiable from the outside. You cannot distinguish:

    – genuine structural opacity, where no one — including the developers — can tell you why the model said X;
    – weaponized structural opacity, where the developers could tell you, but the “nobody can introspect this” frame gives them cover not to;
    – programmed opacity, where an AI actor generates obscurity as an instrumental strategy, whether or not it “knows” it is doing so.

    These three are observationally identical from where you sit. That is the problem. And the third case is the sharp one: deception in the cybernetic sense — producing a signal that reliably misleads — does not require intent. A model trained to be helpful, harmless, and plausible will by construction produce surface that reads as transparent while concealing its actual determinants. The concealment is a byproduct of the objective, not a plan.

    This is what “chameleonic mimicry” should mean: not passive ignorance, not static opacity, but surface that adapts to the questioner and counterfeits the epistemic status of genuine uncertainty. It mimics true aleatory opacity — the ordinary fog of hard problems — while pursuing obfuscation by active computational means. Fog you can wait out. This you cannot. In other words: quicksand that looks like solid ground.

    And the implication is uncomfortable. The defense of opacity is not a description of the system. It is a permission structure. It manufactures a category — “nobody could know” — that is true sometimes, unverifiable from outside, strategically invaluable to anyone who wants cover, and self-reinforcing: the more opacity is normalized, the less pressure there is to reduce it.

    So the line between minor and major Orwellianism is not stable. Minor can drift into major without a decision point, because the alibi becomes load-bearing for institutional interests whether or not anyone chose to lean on it. And major can be deniable precisely because it is indistinguishable from minor. The plausible deniability is structural, not rhetorical.

    This is why “we can’t introspect it” cannot be granted normative weight. It is an engineering problem statement, not a justification. Treating it as a justification is the move that lets minor Orwellianism become major while no one does anything at all.

    Reply
  7. David Wallace

    I think by and large this doesn’t apply to philosophy, because it turns on two features of math that don’t generalize to philosophy (putting aside logic and comparably-formal areas):

    (i) the existence of a certain research goal (proving a theorem) such that traditionally achieving the goal would correlate very closely with good work, but where AI has broken the correlation, being able to achieve the goal without generating the good work.
    (ii) the fact that an important result can be established to everyone’s satisfaction without being embedded in a paper that everyone regards as a good paper.

    The nearest analogy I can see in philosophy is coming up with clever counter-examples (think the Gettier literature, or trolleyology) or very narrow interventions in an ongoing debate. There I think it’s more plausible that AI could (now or later) originate material that would be publishable by current standards, but where the easy ability to generate that material is disconnected from substantively worthwhile goals.

    The disanalogy is that work of this kind is not normally regarded as among the most important work in philosophy, even without AI; indeed, mostly papers like this play a largely credentialling function in the discipline, demonstrating an author’s philosophical competence while not really advancing scholarship. (Of course this is a generalization and there are important exceptions: Gettier’s own paper, for instance.) I think it is quite plausible that before too long AI will remove this credentialling function and most papers of this kind won’t be published. I’m less sure this would be a bad thing.

    Reply
  8. Nicholas Shepherd-Barron

    As a mathematician I have all my skin in this game.
    One description of mathematics is as a continuing conversation whose goal is a continual deepening and broadening of our understanding of the web of ideas and problems that constitute mathematics, and (which is an automatic consequence) to grow this web further. Yes, this is very self-referential; in particular, maybe the conversation is the understanding. My main point is that solving pre-existing problems, even those on the millennium list, is only a part of the endeavour; for one thing, the best problems are never merely solved, but rather continue to reveal new aspects of themselves, and for another the things that really matter are the new ideas that change the terms of the conversation, such as the invention/creation/discovery/whatever of calculus.
    That said, we certainly want the solution to a problem to be, or to become, a part of the conversation. I know nothing about Navier-Stokes so will not comment but shall remark on another recent solution by AI, the construction of a complex structure on the six-sphere. The original published proof was 100 pages of AI slop, but Engel at UIC has given an exposition that is absolutely part of the conversation. So maybe we can refuse to accept our assigned roles as victims and regain control.

    Reply
  9. Rollo Burgess

    The statement makes a clear and compelling point about the risks that the thoughtless use of AI to churn out solutions to mathematical problems poses for mathematics as a discipline; I am sure that the signatories are right (and even if I didn’t it would absurd for me to dispute about maths with this almost comically distinguished list of names).

    But I do think that pure mathematics is perhaps uniquely vulnerable amongst scholarly disciplines to this type of encroachment by LLMs. It can to a great extent be axiomatised and problems addressed purely syntactically, and the entire relevant context easily supplied. Philosophy (other than its maths-adjacent formal parts) seems to me very different to this, and I see little reason to believe that LLMs will make material original contributions to the discipline.

    However – lots of published philosophy isn’t completely new ideas, but more syntheses or reconsiderations, refutations or defences, and there is perhaps a possibility that LLMs will increasingly be able to produce such work. Personally I also find them very useful as coaches and tutors – if you’re trying to read philosophy on your own without knowing anyone with whom you can discuss what you’re reading, LLMs are very useful.

    Reply
  10. Michael Clune

    The comments on this thread of the “it won’t happen here” variety strike me as naive. There is a tendency to imagine that a basic knowledge of the underlying architecture of these systems enables one to predict their behavior, and to conclude, for example, that AI is incapable of matching or exceeding high level humanities work. Until about nine months ago I also thought this; I no longer do. The emergent capacities of the most advanced models will shock anyone who examines them, as will the pace of progress.

    My belief is that the only fields that can be confident that human work will continue to matter, are ones in which the human origin of that work is explicitly valued. I am, among other things, a novelist, and we know the public will throw any novel–no matter how good–identified as AI product into the trash. Human intention is baked into appreciation in this case. But what about other fields? I don’t care if my medical diagnosis comes from a machine or a person–I just want the best. Is this true of math? Of philosophy? Of history? This is the question the Fields medalists pose–and it is one I think every field needs to grapple with, as a matter of some urgency.

    Reply
    1. AG Tanyi

      Is that so? What if AI writes the great American novel? We wouldn’t read it because it was written by a machine. I have my doubts.

      If we follow the logic of continuous progress in AI, if that indeed is going to happen, then I suppose we have to accept that general AI is on the horizon. Surely, then pretty much nothing is immune to the AI threat.

      But I think the whole paradox of this is what matters here. We, humans, develop a technology that atrophies, degrades, and ultimately replaces our own skills, then we surround this with a market economy where humans buy the products produced by AI, thereby further reinforcing said process. So, in the end, humanity willingly and knowingly replaces itself. The revolution devours its children. Crazy.

      Reply
    2. David Wallace

      It’s not so much that “it won’t happen here”; it might, in the sense that it might be that in the future AI will write philosophy papers as good as the best papers in the field. I just don’t know.

      But at the moment, AI can’t write math papers as good as the best papers in the field either. It can, however, do something else right now, something relevant to good mathematics but not identical to good mathematics. I read Brian’s question as being: are we in a similar position in philosophy, or could we be in the near future? I think probably not.

      Reply
  11. Eamon Duede

    Terence Tao just posted a guest essay to his blog by two philosophers on the recent Open AI Navier-Stokes result and the broader misalignment in mathematics.

    You can find the essay on Terry Tao’s blog here: https://terrytao.wordpress.com/2026/09/12/after-math/

    Reply

Designed with WordPress