It can't summarise
This is the kind of thing I like to see. Somebody with multi-domain knowledge putting the work in and checking claims. In this case ‘Can ChatGPT summarise a document?’ And then looking at it from an unbiased point of view:
When ChatGPT summarises, it actually does nothing of the kind.
One of the use cases I thought was reasonable to expect from ChatGPT and Friends (LLMs) was summarising. It turns out I was wrong. What ChatGPT isn’t summarising at all, it only looks like it…
gctwnl
“ChatGPT doesn’t summarise. When you ask ChatGPT to summarise this text, it instead shortens the text. And there is a fundamental difference between the two”
There absolutely is! I never thought about that danger. Lesson learned. Be wary.
There’s an interesting point about the relationship between the context (the prompt, what you type in) and the parameters (what it’s trained on) and how these work together, or don’t.
If this is going to find a sustainable use, there needs to be more non-partisan ‘Sales’ going on. Telling you what it can and can’t do and pointing out the pitfalls rather than frothy ‘Change the World!’ narratives.