Fuck AI

Yeah, that's the thing where we get into what I call "superstitious prompting", like when people say "And make sure you don't make mistakes" or "Include only factual data without hallucinations" and think it works, until it doesn't.

It will at least reply in a way that is narratively consistent with being told to do something or another, and will do things like emit the words "Ok, I understand and will promise to only provide fact based feedback", but doesn't "understand" at all. It works surprisingly well as being narratively consistent with the prompt frequently looks exactly like following instructions.

With people getting all the more frustrated when their superstitious prompt fails, they told the LLM to do something or specifically not do something and it even promised exactly to do as directed and then it just proceeds to be normal LLM anyway.