He explains at CHE. An excerpt:

[O]ne of my main takeaways from gig work [for AI companies] over the last few months is just how hard it is to catch the most sophisticated frontier LLMs in philosophical blunders. When ChatGPT first emerged, a common pastime among my philosopher friends was posting screenshots on social media of its more egregious mistakes; prompts involving logical complexity would tend to produce answers that were both confident and comically incoherent. Now, finding errors in the responses of frontier models takes hours of expert paid labor, and the errors you do find — which typically require misleading prompts — are much less blatant than they used to be.

More striking, though, is what happens when you stop trying to trip the models up. When I straightforwardly prompt a frontier LLM with a technically sophisticated and deeply unobvious question — “What challenges are there to simultaneously holding Ramsey-style analytic functionalism about mental categories and endorsing a theistic argument from design?” — the model typically sees what I’m getting at and answers better than I’d expect from my graduate students, of whom I think very highly. I picked that example because it involves two ideas that are not typically discussed in the same context, and so you know the model isn’t just straightforwardly summarizing information from its training data: It has to reason [sic] its way to a novel answer. So when I ask how we should feel about LLMs being good and quickly getting better at philosophical reasoning, I’m talking about the situation as it stands today, not some hypothetical future.

[I added the ‘sic’ since LLMs don’t “reason,” they mimic reasoning, and apparently they’re getting very good at it!]

Professor Greco goes on to argue that AI doing philosophy is compatible with both the intrinsic and instrumental value of philosophy.

Please read the whole piece before commenting!

(Thanks to Philip Bold for the pointer.)