Leiter Reports: A Philosophy Blog

News and views about philosophy, the academic profession, academic freedom, intellectual culture, and other topics. The world’s most popular philosophy blog, since 2003.

  1. Brian W.'s avatar
  2. Steve Diamond's avatar
  3. Brian Leiter's avatar
  4. Phil Bold's avatar
  5. Gerald Postema's avatar
  6. current graduate student's avatar

    This is an interesting post. One important consideration that is not mentioned is that outsourcing RA-style work to LLMs, rather…

  7. Mark's avatar

An experience with feedback from an LLM

A colleague elsewhere, who subscribes to Astra, Open AI’s high end LLM model (at least $100/month), told me that he has found it useful for giving feedback on early drafts–he compares it to the feedback one gets from a good first-year graduate student. Not doing any historical work, he was curious how it would perform on that kind of paper, and kindly invited me to give him a Nietzsche paper I was working on, which I did. Astra generated several pages of feedback, that was impressive in a number of ways, but deficient in some others. On the positive side: it picked up incorrect citations to texts (e.g., GM II:23, not GM III:23 as I had it), filled in some missing citations (correctly), caught some logical ambiguities in formulations of points, and raise one or two interesting substantive objections. On the negative side: it has no sense of style, no sense of developing a dialectic (e.g., exploring a possible reading of a passage, then rejecting it in favor of another reading–it just wanted to go straight to the “preferred” reading), and it knows nothing substantive about Nietzsche. I recently discussed this same paper with a very good Nietzsche scholar, who gave me more meaningful feedback. But I think the comparison to a first-year graduate student is fair, with the caveat that many first-year graduate students know more about the primary text than Astra. What Astra could not do was challenge any aspect of my reading based on other texts of Nietzsche’s. What Astra could do is assess the internal logic of my argument (with the caveat that it gets confused about a developing dialectic). In only one case, did it misread the argument, and, as noted, n some cases, it read claims I was making carefully and revealed real ambiguities.

Curious to hear what other reader experiences have been with the better models. Please indicate what you work on before describing your experience.

,

Leave a Reply

Your email address will not be published. Required fields are marked *

20 responses to “An experience with feedback from an LLM”

  1. My suspicion is that you would get better feedback if you pointed the LLM at the additional relevant Nietzsche texts when asking it for criticism. It’s asking too much for the training weights to retain all possible interpretations and readings of all possible texts the LLM has been trained on and be able to comment on any given text from a cold start. (Even expert academics have to go back and consult classic texts because human memory is fallible.)

    In this case, I think it would be an interesting test to use an agentic AI which can access the Nietzsche texts on a computer alongside the paper. Why? Because the main limit AIs face is the context window they can maintain. The top Claude agents have a context window of 1 million tokens, but even that can be blown on a complicated task. The usual way around that is to break the task into subtasks and have independent agents read a single, separate chapter of Nietzsche, looking for the relevant parts for your paper. The independent agents write up detailed reports which are then sent to a single overseer who can integrate all the findings without having wasted time and context reading unnecessary parts of the text. This would then given a much more informed — and, I suspect — nuanced evaluation of your paper.

    Current Claude models can do this automatically: a single overseer will break the task down into multiple parts, allocate each task to a separate agent, collect the reports from all the individual agents (potentially validating their report using an adversarial agent which deliberately probes the report looking for mistakes and flaws), and then integrate all the separate findings.

    1. I also wonder if the paper was injected into the context window, in its entirety, at all. I don’t know Astra, but I know that many setups use selective retrieval on attachments (akin to interacting with the attachment through a search engine, with the LLM sending search queries to the engine, and the engine sending back relevant ‘chunks’ of the attachment). That can easily mean that Astra never ‘sees’ the whole paper at all. That would perhaps explain why it struggled to grasps the ‘developing dialectic’ of the paper, since that’s a more ‘meso-level’ feature of the text.

      1. The paper was uploaded in its entirety, it had access to the whole thing.

        1. No but that’s what I’m saying: if you upload it as an attachment, Astra may still be selectively retrieving chunks. It would still have access to the whole paper, in a sense, but not in the sense people usually expect. It depends on the particular setup whether this happens. If you manually copied and pasted the text into the chat window, that would ensure that it doesn’t selectively retrieve.

        2. But did you upload the primary Nietzsche texts themselves to Astra? I would be curious to hear the results of repeating the experiments if you did so. (The corpus is probably somewhere in its training data, yes, but it’s well known that the models can kind of ‘forget’ things like that, and making the sources upfront and centre in the context should really affect its ability to engage with them.)

        3. This is in reply to Kate’s question. Another philosopher very kindly uploaded N’s corpus, and had a different advanced AI (not Astra) write a referee report. The referee report was longer than my paper! Based on the first few pages of detailed comments, they are a mixed bag, although there were a couple of useful things (catching a citation mistake, for example), and it picked up the same logical ambiguity in one of my formulations that Astra had picked up (but neihter of the two Nietzsche scholars who read this had caught). But it’s reading of texts are pretty bad and flat-footed. It has no judgment, alas. I see why, given the way it works, it found the texts it found, and that is in a way impressive. It tends to be overly literal, however, and miss nuances of meaning that human scholars would pick up. So far it hasn’t pointed to any texts that actually conflict with my argument or are worth adding as putative “support”. Qua referee report, it’s sort of wild over-kill, in a way that is not productive (so far). It is impressive the way it is able to find bits of text, even if its judgment about their relevance is uneven.

          So far, the human feedback from real Nietzsche scholars has been more helpful. But I am still working through this long report, since there may be some hidden gems.

  2. I would readily give a passing grade on the process front, but I am not engaging these machines at the product front.

    We all have our process of assembling our thoughts and materials for a work. Over the course of a project I will have a large desk with piles of material, and I am essentially physically laying out for myself, the material I intend to build in the production of the work itself. We all have our ways at this level of “preparation” and “pre-production” so to speak.

    As a rough measure I would say my efficiency at this gathering-together stage is easily trebled by rustling AI’s ability to summon references and establish the basic “piles” on my work desk, so to speak. As a tool, the machine is responsive to this level of inputs at a delightful level. This use-case assumes the researcher is a good researcher and knows the subject well. As I say, an ideal undergraduate major that only exists in our dreams.

    On the production and post-production front, I am not sure I am interested in AI for that, one single bit.

    For that same reason, I am wary of being the one who says, “This is graduate level.” I’m with the Fields Medalists here. We risk destroying the discipline if we use it for our work. Graduate students, are not replicable.

  3. Brian, any chance of getting your colleague to specify how he used Astra to produce the feedback? Did he used standard chat, Canvas, Deep Research or Workspaces? What was the actual prompt? Did he use more than one? What was the total time spent by Astra to produce the final results?

    1. The prompt was someting like this: Summarize the paper and then make suggestions for improvement. The summary was a bit pedestrian, but as noted the suggestiosn for improvement had some useful and sometimes subtle suggestions about particular argumentative points, as well as corrections to references.

      1. It seems to me if one wants to compare the feedback of an LLM researcher with that of a graduate student, one should treat the LLM as one would treat a graduate student: Before giving the assignment, have a conversation about the feedback one is looking for, and the particular issues that are of most interest. Ask the researcher if he, she or it has any questions. After the researcher produces the feedback, one should ask followup questions and challenge the LLM (or human researcher) to produce a more sophisticated response.

        1. This is not how one treats a graduate student. If you share a paper with them, you say, “let me know what you think.“. And while you might discuss the substance of the paper with the student in light of the comments, you don’t give them feedback on how to give comments.

        2. Rob may be wrong about academic conventions, but he is right about model usage. First, performance will improve significantly with greater context. Here the comparison with a graduate student is unhelpful. Handing a graduate student a draft along with three pages of assumptions, expectations and analytical criteria would be counterproductive, but such extensive detail is what language models thrive on. Second, treat it as a dialogue. Set expectations upfront, challenge initial output and ask follow-up questions like, “Are there any passages from Nietzsche that potentially conflict with my thesis?”

          A pedestrian prompt yields a pedestrian summary.

        3. On reviewing this thread I realize there is a deeper point that is usually missed. Consider the behavior of a raw LLM, not what you use in any of the chat interfaces such as ChatGPT, Claude or Gemini, but the base transformer model trained solely on next-word prediction. It will often produce responses that are coherent but silly. For example, it might reply to “What is the capital of France?” with “What is the capital of Germany?” because the training data is full of geography exams. Responses can also repeat the same sentence over and over or slowly wander off topic, all the kind of behavior you would expect from a next-word prediction model. Early on it was discovered that if you preface the query with something like “I am a helpful, honest, knowledgeable chatbot,” much of this silliness disappears, not because the model has an understanding of “helpful, honest, knowledgeable chatbot,” but that this context makes the silly responses far less probable. Building on this insight, AI vendors provided additional training and developed a harness that shaped the model into a conversational agent.

          But all of this work only gets you a polite, knowledgeable general assistant. It doesn’t obviate the need to supply your own extensive context if you want optimum results. In that sense, an LLM is analogous to a massive standard database. The knowledge it stores is vast, but functionally inert. The intelligence required to extract something useful does not reside in the storage medium but in the post-training and scaffolding provided by the vendor and, crucially, in the query itself.

          There is a kind of “magic” here: you adopt the intentional stance, addressing the model in natural prose as if it were a colleague, though possibly an idiosyncratic one requiring detailed, extensive instructions. But you understand the workings of the model from a design stance: the model does not understand your intent; rather, the extended stream of tokens constrains and reworks its probability distribution, steering it away from a pedestrian response to something more insightful and coherent.

  4. My experience is with fairly technical philosophy of physics papers, and with Opus 5.
    – copyediting: amazing, much better than any human I’ve worked with. (I accept ~80% of its copyediting advice.)
    – structure and argument: variable, but sometimes very good. It has a bit of a CS attitude (perhaps unsurprisingly) but it has certainly spotted structural infelicities. (I accept ~30% of its structural/argument advice).
    – style: it tries to homogenize my style towards its own, and has a bit of a tin ear for anything imaginative. (I accept ~0% of its style suggestions.)
    – literature and sourcing: it’s excellent on what’s out there, less good on the content of what’s out there (i.e. it provides good links but doesn’t always summarize them well). It’s extremely good at matching technical statements I’ve made to places in the literature where similar statements have been made already; I would anticipate it’s much better at doing this in math or theoretical physics than in philosophy.

    My default prompt includes “I am fairly thick-skinned; if I get something wrong, tell me.” I think this has helped a bit with sycophancy.

    1. I am curious how you did this, like your workflow. It sounds interesting.

  5. Philosophy Undergrad

    I was always very worried about submitting my drafts to an LLM because what if it steals my work! Will my intellectual labor become part of its knowledge base? You can see why this would be especially anxiety-inducing for a budding scholar with no reputation. I submit my paper or writing sample to a committee, the committee then consults the LLM and says “Oh, well, this isn’t so novel; the AI was able to spit it right out.” Well, yeah, it spat it out because it first chewed up the work I gave it, but has no means of attributing it to me!

    I asked ChatGPT whether this was a legitimate worry, and it said not really. But if I was really worried, it said, I could simply adjust the settings so the model doesn’t train on our conversations. But still… I’d be curious to know what others think.

    That said, I did submit a short lyric poem I wrote and the feedback was excellent. It spotted things even I hadn’t realized were there, and it judged the poem to be “a serious and accomplished example of contemporary free verse.” So I can’t hate it too much. Also, it did offer some suggestions for edits, none of which I would consider taking.

  6. I work in political theory (political realism, democracy and AI), and I mostly use Claude Opus 5.5 nowadays. Your assessment matches my experience: it often feels like an intelligent colleague who is not a specialist in my specific niche. But results also vary a lot depending on how you prompt it.

    As others said, the more context it has, the better it performs. It doesn’t retain a complete memory of what it read in training, so it has to search the web. If the relevant academic sources are paywalled, or the sites block AI access, it may have to rely on lower-quality material (reviews of books, blog summaries, etc.), and the output suffers. I suspect fields that rely more heavily on open access get much better results.

    So what works for me is giving it the context myself. If I’m working on a paper that builds on my earlier work, I upload those older papers and tell it to use them as context. Beyond the substantive content of the papers, the model gets my writing style and the way I usually argue (lots of examples, occasional light humour), as well as a starting list of the relevant references.

    Also, when you don’t like the output, you can just give it feedback. Iterating on a task gets you a better result than a single prompt, though not yet what you’d get from a specialist on the subject.

    1. “I suspect fields that rely more heavily on open access get much better results.”

      That’s a really interesting observation. I noted previously that I thought Claude did better at math/physics than philosophy; I’d assumed that was because the content was more formal, but this might be as much or more of a reason. A lot of philosophy of physics is open-access, and virtually all of physics is; that might explain why my experiences have more positive than some others’.

      1. Thanks, that’s a nice data point. I suspect both are at work, and physics makes them hard to separate, since it is both formal and mostly open. I’m curious whether there’s a domain that is formal but mostly paywalled, which could serve as a rough test.

  7. I use ChatGPT Luna 5.6 because the Cal State system has a contract with OpenAI. I might prefer to use Claude Opus 5.5 all other things begin equal, but institutional access includes a reasonable expectation of privacy protections as well as research tokens and prioritization that even paid personal accounts do not tend to have.

    On a different thread here, someone recently suggested prompting a premiere engine to provide a referee report on one of one’s own papers. So I did. I gave Luna the submission instructions for Dante Studies and for Speculum and asked it to use those as a baseline for offering two different reviewer perspectives on two different articles (one unpublished, the other forthcoming in an anthology). The reviews were detailed and the “results” plausible — both revise and resubmits, but one correctly judged to be much stronger than the other. Some of the analysis of the arguments was excellent and showed soft spots in my treatment with a reasonable sense of how a specialist in Dante studies might evaluate differently than a “medievalist” (whatever that means) — for instance, the Speculum reviewer wanted more discussion of general intellectual history; the Dante specialist reviewer wanted more explicit survey of other prior work in the field. Those both seemed appropriately calibrated to the sort of reviewer that those journals might provide.

    In all four cases (both reviews for both papers) I would say that an excellent human reviewer might have offered some sorts of more valuable feedback, especially in terms of offering a meaningful assessment of the impact of the argument in a field of scholarship — the Dante Studies reviewer for one of my papers did not seem to have a good sense of the lay of the land on the issue presented in one of the papers I submitted. But we are not often lucky enough to have excellent reviewers, who, however critical their assessments might be, start from a place of hermeneutic charity and an effort to improve and fairly assess others’ work. I would not suggest that journals should replace human reviewers (Paul Bloom’s recent essay on this question seems to raise the right sorts of concerns about this — https://www.chronicle.com/article/could-ai-be-better-at-peer-review-than-humans), but I would say that getting feedback from an LLM for oneself might often be worthwhile. At the very least, one is not likely to get the sorts of reviews that completely miss the point or offer condescension rather than fair-minded assessment.

    As for other sorts of uses, my experience is similar to David Wallace’s. I do not find LLM’s helpful for generating ideas, but they are pretty useful at providing certain sorts of feedback. I also frequently use them to help consider various alternatives in language translation tasks or to locate passages I dimly remember reading but want help finding. If I am asked for an abstract of a paper I have written, I would have very few qualms about outsourcing that task to an LLM for an initial draft. But at this point, I do worry that if I send back an abstract generated by AI, I would open myself to stigmatization even if the paper itself were 100% human authored (whatever that means in a world where my word processor lets me know if I’ve used “it’s” when I should have typed “its”).

    There seem to be no settled norms around disclosure — the takeaway from the Oxford University Press survey is that attitudes about AI are all over the place, and it is therefore currently impossible to establish any reasonable disciplinary consensus on norms for AI disclosure.

Designed with WordPress