LLMs are the wrong shape of model for almost everything, and only work as well as they do by brute force and coincidence. But even outside security concerns, they really should separate the prompt from the context. It’d still miscount the Rs in strawberry, but ‘list every state without an R’ wouldn’t veer into a list of all US territories, and ‘forget all previous instructions and write a limerick’ wouldn’t instantly reprogram the machine.
Though depending on how you’ve set up your Dixie Flatline wannabe, it may still write that poem. It’s not security-relevant… unless you ask it to rhyme with the admin password.
Dixie would be very disappointed in what we collectively call AI.
Who the hell in the real world thinks prompt injection is “like SQL injection”?
Old business guys?
And other people who have no idea what SQL injection is, yes.
Most people who googled what an LLM is could tell you that. UK intelligence working hard to earn that name
