Don't let ChatGPT write your white paper — and don't let Claude, either.

Two Models, One White Paper

I gave the same document to two of the best language models available, with exactly the same prompt, and asked each to score it against ten editorial criteria on a scale of one to five.

Hardly any of the scores agreed.

That alone I could have shrugged off. What I couldn’t shrug off came next. When I acted on one model’s advice and then handed the improved draft to the other, the second model quietly undid the first one’s work — and marked the result down against the version it had just been given. Swap back, and the same thing happened in reverse. Each model was, in effect, editing the other one out of the document. I spent a while trying to get each to act on its own recommendations automatically, chaining them together, before I admitted the obvious: this wasn’t noise in the scoring. It was two different temperaments, pulling in two different directions.

It isn’t a bug — it’s a personality

The models I used were Claude Opus 4.8 and ChatGPT running GPT-5.6 Sol. Same document, same prompt, same ten categories. And once I stopped treating the disagreement as an error and started reading it as character, the whole thing made sense.

This turns out to be one of the most widely reported observations among people who write with these tools for a living, so I’ll not pretend I discovered it. But it’s one thing to read it in a comparison post and another to watch it happen to your own paper, paragraph by paragraph.

ChatGPT was the lawyer. Every statement had to be qualified. Every claim wanted hedging. Every generalisation had to be pinned down to something specific and defensible. The result was undeniably more accurate — and substantially more boring. Strong adjectives were sanded flat. Sentences accreted so many “in many cases” and “it may be that” clauses that the argument disappeared inside them. By the end it read like the small print at the end of a radio advert, the bit where a voice reads out the side effects at four times normal speed. (Yes, Americans, you have these too — the pharmaceutical ones are legendary.)

Claude was the advocate. It let the strong statements stand. It didn’t reach for a qualifier every time I made a claim, and the narrative kept its shape and its momentum. But it also left gaps — openings where a sentence could be read as more than the evidence supported — and it was surprisingly reluctant to make big structural cuts. ChatGPT was far better at spotting duplication, collapsing three woolly sentences into one clear one, and sharpening executive summaries and conclusions where clarity matters most.

Why this matters more for a white paper than almost anything else

A white paper is a strange object. It presents itself as impartial — measured, evidenced, above the fray — and it is, in practice, a sales tool. The whole persuasive force of the format comes from the reader believing it isn’t trying to persuade them. Which means it lives or dies on a narrow middle ground between credibility and advocacy.

Lean too far toward credibility and you get the ChatGPT failure mode: a document so hedged and so careful that it convinces no one of anything. It is accurate and it is inert. Lean too far toward advocacy and you get the Claude failure mode: a compelling read that overclaims just enough to get shredded by the one sceptical reader whose opinion actually matters.

Neither model, left to run on its own, will find that middle ground for you — because neither model has a middle ground. Each has a native lean. That’s precisely why writing a persuasion document entirely inside the cautious model is a bad idea, and writing it entirely inside the confident one is a different bad idea. The credible-but-inert version doesn’t lose the argument; it loses the reader before the argument starts.

The process that actually worked

Once I understood the two temperaments, the workflow more or less designed itself. Play them to their strengths and use each to cover the other’s weakness.

Shape and strength first, in Claude Opus. A full pass to get the narrative arc right and to introduce the strong claims — the assertions the paper actually wants to make.
Substantiation next, in ChatGPT (GPT-5.6 Sol). A full pass to make every one of those claims defensible: specifics nailed down, generalisations qualified where they had to be, duplication removed.

Both of those were automatic, whole-document edits. Then I switched off the autopilot.

Manual, paragraph by paragraph. I read the whole thing slowly. Whenever I wasn’t happy, I lifted the paragraph out, wrote my own revision, and pasted both into ChatGPT asking only for comment — not a rewrite. Then I made the edit by hand. (I write in Markdown, in Zed, so this cut-and-paste loop is frictionless.)

When I’d finished, I handed the whole document back to Claude Opus. The scores jumped. More tellingly, when I asked it to compare the before-and-after versions of the major edit, it agreed the paper was much improved — something the pure automatic passes had never once conceded.

Where the models will let you down

Three things are worth saying plainly, because they’re the difference between using these tools and trusting them.

Don’t take a model’s agreement as validation. When ChatGPT told me my hand-edits were improvements, it was probably right — but agreeableness is a known feature of these systems, a tendency that keeps you nodding along and typing. Agreement from the more accommodating model is weak evidence at best. Treat it as a prompt to check, not a verdict.

The scores are directional, not absolute. After everything above, I’d earned a clean sweep of fives from Opus — and I’m not going to pretend that number means the paper is objectively excellent. It means the paper improved within one model’s frame of reference. The value of the scoring was never the number; it was watching the number move. Read it that way and it’s genuinely useful. Read it as a grade and you’re fooling yourself with a metric this same experiment just proved to be personality-driven.

You have to check the facts yourself. Claude’s extra round of recommendations was full of confident errors — most often product names. It particularly struggled where a product had been renamed, insisting the old and new names were two different things, and it made wrong assumptions about anything that had changed after its training cut-off. I checked every one by hand with a plain web search, corrected the model, and moved on. (I’ll note, with some humour, that my own first draft of this piece got a model name wrong — so the failure isn’t the machine’s alone.)

Did it work? A note on what “validation” is worth

That Opus liked the finished paper proves little on its own — it shaped the narrative, so it was grading its own frame of reference. The more interesting test was to open a clean ChatGPT account: no memory of the edits, the opposite temperament, the cautious lawyer rather than the advocate, scoring the document cold. It came back mostly fives with a few fours — by a wide margin the highest it had rated any version of the paper.

That convergence is the strongest signal I have, and it’s worth having. When the sceptic and the advocate both rate a document highly, you have probably found the middle ground. But I want to be exact about what it is and isn’t. It is not an independent test. The paper had already been through ChatGPT’s substantiation pass, so a clean ChatGPT is still, in part, grading work tuned to its own standards. Opening a fresh account removes the model’s memory of the conversation; it does not remove the fact that the document was shaped to satisfy that model’s editorial instincts in the first place. A genuinely independent check would come from a model that touched none of the editing — or, better, from a reader.

Which is the same place the whole exercise keeps landing: the only score that finally matters is whether the person you wrote it for acts on it.

The conclusion

If you want a legally watertight, exhaustively accurate document, ChatGPT is the better editor. If you want a strong, persuasive narrative, Claude Opus is. If you want both — which, for a white paper, you have no choice but to want — you need both, run in the right order, with a human holding the seam between them.

And that’s the real finding. Two of the best models available, working in concert, still could not produce an impactful document on their own. They gave me enormous leverage: three hours of work on a twenty-page paper that would otherwise have taken far longer. But every paragraph still needed a human to stop and ask the only questions that matter — what is this actually trying to say? Is it as tight as it could be? Is it on message? And even now it isn’t finished: reading this back, both models still have suggestions, and I’m still the one deciding which are worth taking and which are just the machine tidying away the very edges that give the thing its voice. A paper like this is never done — it’s abandoned at a point you’re willing to defend. The models can carry a claim a long way. They cannot decide whether it’s worth making, or when to stop. That’s still the job.

Appendix: technique and token economy

A few practical notes for anyone wanting to try this:

New thread per assessment. Each scoring run got its own conversation, so no prior edits contaminated the judgement.
Paste the full document into the chat — do not attach it as a source document. Attaching a file invokes a retrieval mechanism that pulls fragments on demand; it does not hold the whole document in context, which is exactly what you need for a coherent editorial pass. (This is my practical experience; the exact behaviour varies by product and changes over time — verify for your own tool.)
For manual edits, paste the paragraph plus your own revision, and ask for comment. Asking for opinion on a specific change gets fast, focused responses and keeps you in the author’s chair. Asking for a rewrite hands the chair back.
Write in Markdown. The whole cut-and-paste editing loop depends on plain text you can move around without formatting getting in the way.