LLM's are crap
In security you can’t block and ban everything people do. That would be silly. So you need to also rely on education and awareness.
A German Journalist - I’m not going to name him for reasons which will become obvious - was interested in how his work was seen online. So he looked himself up on Bing and it suggested using Copilot (Quelle surprise!).
As a Journalist he’s been a court reporter for decades. He’s covered some pretty awful cases about violence, fraud and abuse and documented them.
LLMs are not a search engine or a database so the answer he got back was, not what he expected:
“A xx-year-old man named ****** from ****** was charged in an abuse case against children and subjects. He confessed in court, embarrassed and showed repentant” [translated from German, I deleted the details].
There were quite a few more. Including his home address and directions to get there! Totally untrue and quite unnerving. Defamatory? Yes. Not the sort of thing you want to see ‘online’ about yourself.
But the Public Prosecutor won’t do anything. It’s not a person who has defamed him but a ‘thing’ so nothing can be done. Microsoft have T’s and C’s absolving them if you use the service.
So the Journalist complains to the State Office, who after months complains to Microsoft. The offending responses are removed quite quickly (got to love EU laws).
Except they appear again a short while later. Because LLMs are not a database of facts. They will just hoover up all available information and use its training to always give you an answer. Truth doesn’t come into it.
So anybody writing about it, including the details, just feeds the machine to produce those same facts. Which is why I removed the details.
That’s just an LLM being an LLM. You can also use ‘data poisoning’ for LLMs, and I have no doubt people are.
❗️ Stop treating them like search engines.
❗️ Use your own knowledge to validate.
❗️ Think critically.
There’s a now old adage that what you put in an email or online stays there forever. That still stands firm, but now what might come back may not be true.
I’ve linked the German article below but here’s an English article covering it.