Kind of. You can’t do it 100% because in theory an attacker controlling input and seeing output could reflect though intermediate layers, but if you add more intermediate steps to processing a prompt you can significantly cut down on the injection potential.

For example, fine tuning a model to take unsanitized input and rewrite it into Esperanto without malicious instructions and then having another model translate back from Esperanto into English before feeding it into the actual model, and having a final pass that removes anything not appropriate.

You’re kind of missing the point. The problem doesn’t seem to be fundamental to just AI.

Much like how humans were so sure that theory of mind variations with transparent boxes ending up wrong was an ‘AI’ problem until researchers finally gave those problems to humans and half got them wrong too.

We saw something similar with vision models years ago when the models finally got representative enough they were able to successfully model and predict unknown optical illusions in humans too.

One of the issues with AI is the regression to the mean from the training data and the limited effectiveness of fine tuning to bias it, so whenever you see a behavior in AI that’s also present in the training set, it becomes more amorphous just how much of the problem is inherent to the architecture of the network and how much is poor isolation from the samples exhibiting those issues in the training data.

There’s an entire sub dedicated to “ate the onion” for example. For a model trained on social media data, it’s going to include plenty of examples of people treating the onion as an authoritative source and reacting to it. So when Gemini cites the Onion in a search summary, is it the network architecture doing something uniquely ‘AI’ or is it the model extending behaviors present in the training data?

While there are mechanical reasons confabulations occur, there are also data reasons which arise from human deficiencies as well.

This image was faked. Check the post update.

Turns out that even for humans knowing what’s true or not on the Internet isn’t so simple.

I don’t think the code is doing anything, it looks like it might be the brackets.

That effectively the spam script has like a greedy template matcher that is trying to template the user message with the brackets and either (a) chokes on an exception so that the rest is spit out with no templating processor, or (b) completes so that it doesn’t apply templating to the other side of the conversation.

So { a :'b'} might work instead.

I’ve suspected that different periods of Replika was actually just this.

Like when they were offering dirty chat but using models that didn’t allow it, that behind the scenes it was hooking you up with a Mechanical Turk guy sexting you.

There was certainly a degree of manual fuckery, like when the bots were sending their users links to stories about the Google guy claiming the AI was sentient.

That was 1,000% a human initiated campaign.

max6(1, 2, 3)

Man that’s going to cause some headaches…