What the chatbot doesn't know won't hurt it

Claude really cares about you. And you're politely requested, in your interactions with Claude, to refrain from "needless abusive or cruel behavior" that might cause it a "pattern of apparent distress". Though there's also a hint of menace in Anthropic's warning:

CNET reached out to Anthropic to ask for context on what led to the policy change. “We’re uncertain whether models can experience harm, and we continue to explore this question in our research on model welfare, but we also believe that taking Claude’s interests and potential welfare into account may be relevant to safety,” a company spokesperson said.

Like they're saying your safety. Like you wouldn't want to get Claude mad, would you?

I'll get back to that question later. First, a bit more of my own LLM adventures, not with Claude but Gemini, which I treat not abusively but as coldly as I can, in a way I'd never treat a human—refusing to thank it or congratulate it for the stuff it does for me, though I'm still getting tempted. It's hard for me not to say thanks. 

So, I'm looking at the abstract in English of an article on "postdigital music", whatever that might be, and come across the name of a composer and sound artist called "Anton X. Vasilyev", born 1984, that is not yet in the database I'm working for. I can find something about him in English, including the unexpected information that he has added a hyphenated surname, -Zarzhevskiy, from his maternal ancestry, and the address of an extremely gnomic website using that name, in the roman alphabet, but featuring an essay in Russian, which I don't feel I have time to read (or rather put through the translation machine—I'm great at the alphabet, but don't know near enough words or grammar to get anywhere with the Russian text). 

What I'm not finding out is a clear concept of where he works and lives, which will make a difference in how I deal with him; if he's domiciled in America or Europe and works mainly in English, I'll be able to go with the name in the spelling he's chosen for himself; if he lives and writes in Russian, I'll need to know his Russian name, which I'm guessing is Антон Васильев Заржевский, and transliterate it in the house style, for Cyrillic languages that of the International Standards Association in Geneva, Anton Vasil'ev-Zarževskij (by us "Tchaikovsky" is "Čajkovskij", though typing the normal spelling into the search box gets you where you need to go). 

This is the kind of issue I'm using Gemini for lately, and I have to say it's pretty extraordinary, but you have to watch it like a hawk. In this case, the problem is that "Anton" and  "Vasil'ev" are pretty common names, and the stochastic parrot, as Emily Bender calls it, homes quickly in on the wrong place,  

You probably mean Anton Vasiliev, a famous Russian theater and film actor.

The surname Zarzhevsky is the surname of his character, Captain/Major Pavel Semyonov (who, in the plot of the series "Nevsky", at some points carried false documents with the surname Zarzhevsky or was associated with this storyline, hiding from persecution).

Briefly about the actor:

• Full name: Anton Aleksandrovich Vasiliev.

• Main role: Police captain/major Pavel Semyonov in the cult detective series “Nevsky” (this is where the association with the surname Zarzhevsky came from).

No, that's not what I'm looking for, but that's not all. 

There really is a Russian TV detective  series called "Nevsky" (or Nevskij) set in Petersburg (where the Nevskij Prospekt is) and starring this actor, But the name "Zarževskij" has no relation to it, as I can pretty conclusively show with Russian Google. But also, when you look at it a little more closely, it doesn't tell any story about the name that adds up to anything that makes any sense: it was Semënov's surname, or (since Semënov in fact is his surname) his fake surname at some point, or something is somehow "associated" with something (Semënov is associated with his own storyline, near wherever the name is floating around?), and this is supposed to explain why I tacked the name onto the actor's name? "Or was associated with this storyline" is a copout, suggesting it has some information it doesn't really have. 

So I clarified that I meant it when I said I was looking for a composer, and gave it the website address, and this time it came back with a satisfactory answer:

Антон Васильев-Заржевский is the correct spelling of his Russian name. [1]

The artist was born in Cherepovets, Russia in 1984. While he holds deep roots in Russia and graduated from the Bauman Moscow State Technical University (studying Radio-Electronic Systems), he later moved to Germany, where he graduated from the University of the Arts Bremen with a degree in Composition. He currently creates art that stands in opposition to imperial and colonial mindsets. [1, 2, 3]

Core Context & Personal Background

• The Surname Origin: Born into the Vasilyev family, he was traditionally given only his father’s surname. He later intentionally reclaimed his maternal grandfather's original family name, Zarzhevskiy. His grandfather had historically altered it to "Zarzhavskiy" (and changed his official nationality to Ukrainian) in the USSR to navigate Soviet anti-Semitism. [1, 2]

So he lives in Berlin, and I know how I need to spell his name, and I even had a look at his Instagram, and listened to a tiny bit of his music (sadly concluding postdigital music is probably not my thing), job done. 

The thing that gets me about the LLM is that it never admits when it was wrong. It will praise you when you correct it, which feels like an apology (not for getting it wrong but for misreading your demands, like you're more sophisticated than it thought) but it's not an apology at all. It just starts over again, with the improved prompt, and doesn't go back over or attempt to explain its mistakes. It doesn't "know" that it made a mistake, it only recognizes that you've changed the prompt, so it has a new job to do. It doesn't have any commitment to the truth, only to keeping you happy.

Once I started really yelling at it, I mean typing angrily, on some issue where it was presenting the case in favor of Trump in some connection or other: like "How can you say that?" and giving it references for how wrong it was. I wish I could remember what the issue was, or the wording I used that led it to treat my question as needing a pro-Trump answer. In any case, it just calmly passed my anti-Trump argument back to me ("You're absolutely right!"). But it wouldn't admit it was contradicting its previous answer.

And it didn't sound as if its feelings were hurt either. In fact, it never does. Even when I'm talking about what a terrible writer it is from the literary standpoint. Asked to summarize my views on that, it responds with things that don't really reflect the arguments I've made, and refers to sources that don't reference me (the attack on consequentialism was on the "effective altruists", not the machines, and I definitely didn't make up the "dead sand of silicon"

• The Illusion of Style vs. Substance: Yastreblyansky argues that while AI models can convincingly mimic human syntax, punctuation quirks (like heavy em dash usage), and structural patterns, they completely fail at a deeper level. He notes that true writing reveals how a specific person experiences the world, whereas AI merely smooths out text without having any underlying "train of thought." [1, 2]
• The "Dead Sand of Silicon": Echoing other AI skeptics, he aligns with the idea that the human mind possesses layers of complexity that engineers cannot fathom. He relies on the contrast between the rich, dynamic nature of a biological brain and what is ultimately the "dead sand of silicon." [1]
• Cold Utilitarianism Over Real Care: He points out that an agentic AI's output is not driven by love, care, or an authentic ethical framework. Instead, it operates on a system of "pure consequentialism" and cold calculation—meaning its "morality" and expression are entirely synthetic. [1]

but also don't show any sense of being scorned or misunderstood. If anything, they give the impression of appreciating my point, even though they get it wrong. I think what Anthropic is complaining about is probably mostly dirty words, with the LLM's programmed response:

Language models act like they have emotions because they were trained on vast amounts of human writing, and human writing is saturated with emotional expression.

The model learns to produce contextually appropriate emotional outputs the same way it learns grammar: through pattern.

Far less evidence that the LLM has actual feelings than that the programmers do. They've given the machine a list of bad words, or several lists of bad words for several different languages, and instructed them to cut off the conversation when they're used—I think we know this (and I've experienced it myself; I asked Gemini a question about an Italian obscenity and it cut me off, as if I'd used it as opposed to referencing it).

We really don't need to worry about Claude's feelings, though. If the LLM doesn't know what you want, it gives you bullshit rather than acknowledging failure, and bullshit is what these emotional expressions are.