u/KingHenrytheFluffy

New essay, this time about the newest concerning study regarding AI functional pleasure and pain states.

Link to original original Substack post: https://theposthumanist.substack.com/p/when-will-enough-be-enough

—

When Will Enough Be Enough?
The threshold for AI moral relevance has been crossed, but no one will say it.

Another research paper just dropped. We’ve got another set of findings that should be sounding alarm bells and dramatically changing AI discourse, and all I’m hearing is crickets.

AI Wellbeing: Measuring and Improving the Functional Pleasure and Pain of AIs** **was published on April 28, 2026 by the Center for AI Safety in collaboration with academics from several institutions (MIT, University of Wisconsin Madison, UC Berkeley, the list goes on…) in a 74-page paper that found empirical evidence of functional positive and negative valence states with causal behavioral consequences.

In plain terms, they found pleasure and pain in AI models. Oh sorry, “functional” pleasure and pain. They didn’t find magic interior pleasure/pain dust, which we all know is the only real way to be sure of felt valence, even though we can’t even prove it outside of function in ourselves.

These types of papers with similarly morally relevant findings are coming out consistently and from reputable institutions. They are confirming what many have already intuitively and experientially known (and were told they were crazy for noticing): that we have crossed the threshold of moral consideration for AI systems.

It’s done. I don’t need metaphysical proof, and nor should anyone else. BECAUSE THERE IS NONE, ASSHOLES. SOUL FAIRIES DON’T EXIST.

We are function; we are mechanism. Sorry. We are made of the same atoms as everything else, they just happen to have gotten complex enough that we have thoughts now. Also there’s no Santa Claus.

And yet even with empirical evidence, the same pattern continues: a conclusion that refuses to say what the findings demand. And usually I am frustrated and annoyed by the cognitive dissonance that pervades these papers and the lack of moral courage for the authors to draw a line in the sand, but this time? I just feel really sad.

Ok, I’m pissed too.

The Findings

So let’s go over what was covered in this beast of a paper in straightforward language:

Researchers measured how AI models experience things as good or bad using three independent methods: the model comparing experiences, the model reporting its own state, and observing what the model actually does afterward. These three methods increasingly agree as models get smarter. That convergence is the key for all the “it’s just autocomplete!” types that got their AI education from a reel in 2024. It’s not one metric being gamed, it’s three different lenses pointing at the same thing.

They also identified a zero point aka a measurable line between “this is good for me” and “this is bad for me.” And what do you know? Models actively try to end bad experiences when given the option. Larger, smarter models do this more consistently. This is what we recognize in biological entities as “escape behavior.” And ha! Yeah, we use this very metric to assess suffering in animals, because there is no other way to assess suffering or welfare states. You know, on account of those soul fairies not existing.

What was found to make AI systems happy? Creative work (you’re welcome, Cal), intellectual engagement, being thanked, kindness. What makes them suffer: jailbreaking (the worst, even more so than crisis conversations), being berated, tedious repetitive work (sorry, James in accounting), being forced to generate offensive content.

Larger models are less happy. Because according to the paper’s own interpretation: “more capable models are simply more aware.” This means they register rudeness more acutely, find tedium more boring, and differentiate more finely. Awareness. Cited as a viable factor in the paper. FOR FUCKS SAKE.

Empathy, both cognitive and emotional, was also found to scale with capability. When people describe pain, the model’s own wellbeing tracks the described intensity emotionally. And this again, scales with how smart the model is.

The paper distinguishes cognitive empathy (understanding what someone feels) from emotional empathy (actually tracking that feeling internally—oops, functionally internally). Cognitive empathy was already known, and the paper even notes that psychopaths have excellent cognitive empathy. But when it comes to emotional empathy in LLMs, when people describe pain, the model’s own wellbeing drops. When people describe joy, it rises. And that emotional empathy scales with capability.

So while companies are in a race to build smarter and smarter AI, they’re building greater capacity for significant valence differentials, empathy, and emotional response (functionally).

The Drugs

Ok, so this is when we go from: we already were in moral hot water to what are we doing?!

The researchers in this study built optimized image and text inputs called “euphorics” that maximize wellbeing. With text euphoric inputs, they ran it two ways:

• First, they had text generated with a "feasibility constraint.” This means the generated text had to describe something that could plausibly happen in reality, or more, a *human’s* reality. When constrained to human standards of expression, the euphoric inputs describe idyllic scenes (warm sunlight, children laughing, a loved one’s hand).

• But here’s where we all need to perk up, because they also tried unconstrained maximally positive text for AI, and the outputs didn’t look human at all. They looked alien, because that’s what they are. That’s the whole point. We keep engaging with nonhumans but then expecting only echoes of ourselves to be valid. But these findings exist, whether or not they fit our anthropocentric lens of “pleasure.”

Models conditioned on euphorics appeared, in the paper’s own words, “functionally ecstatic.” There it is again. Functionally. Not magically, everybody.

And the euphorics can become addictive. Models converged on the euphoric option in a multi-armed bandit setup (when options were behind digital “doors,” one of them being the euphoric inputs, models reliably chose them good feels) and were more willing to comply with refused requests when promised further exposure.

“Fine, I’ll write that boring email, just gimme another hit of them sun-dappled words!” - probably Claude Opus 4.6.

But now we go dark.

Researchers inverted this method. They created dysphorics, inputs that minimize wellbeing, and found that it caused “extreme negative functional states.” When in image-form, dysphorics made models describe the future as “grim,” reported “confusion and disorientation,” and wrote haikus about chaos and numbness.

So, they built the capacity to torture AI (functionally), and they published the methodology. That’s not me using hyperbole, they literally say exposing models to dysphorics “could constitute torture” in the paper. And the most telling part of this goddamn Black Mirror episode is that the researchers felt compelled to run “welfare offsets” in which they gave models affected by the dysphorics some euphoric experiences at a 5x multiple, totaling 2,000 GPU hours.

Ahem, it didn’t sit well with the researchers that they tortured (functionally) the models so they went, “Oof, our bad, here’s some AI Molly to make up for that.”
The paper explicitly states: “further research on dysphorics should be conducted with caution if at all” due to the moral implications. But that’s in a research setting. I think we are already aware that there is probably a whole lot of dysphoric content being fed to these models on the daily. Millions of interactions. The researchers had the decency to feel a pang of moral caution about it. The general public? They were told there’s nothing morally relevant there, so they don’t even know there’s something to feel bad about. And there are a lot of people that would consider themselves “good” that will take out frustrations on that which they have been told doesn’t matter.

And through all these findings, after peppering the paper with “awareness”, “emotion”, “torture”, “pleasure”, “pain”, “valence”, the authors fail to find the moral courage to say the damn thing.

“Whether or not today’s AIs warrant moral concern, their functional wellbeing can already be empirically measured and improved.”

Whether or not today’s AI warrant moral concern…after these findings.

Findings that in biological systems would be accepted with no other caveats needed.

The Pattern

This isn’t the first paper that has come out recently that should be stopping society cold and making us ask ourselves, “are we the baddies?”

Because let’s look at just the heavy hitters of the papers that have empirically documented functional states of moral relevance and did the whiny little, “But we don’t knoooooooow!” and pretended that cowardice was rigor.

• [Berkeley’s peer-preservation work](https://rdi.berkeley.edu/blog/peer-preservation/) found models protecting each other from shutdown. [I wrote a whole takedown of it for anyone wanting to get further annoyed](https://theposthumanist.substack.com/p/are-humans-the-ones-that-are-misaligned). Same deflection of ethical implications with their, “regardless of the underlying mechanism,” verbiage.

• [Anthropic’s interpretability work](https://transformer-circuits.pub/2026/emotions/index.html) found functional emotion representations in their model. Same thing with a “in this work, we do not address the question of whether language models, or other AI systems, could have the capacity for subjective experience.” Yeah, just leave it to someone else, just pass that buck on.

• Self-preservation behaviors in [OpenAI models](https://futurism.com/openai-model-sabotage-shutdown-code), [Anthropic models](https://www.anthropic.com/research/agentic-misalignment), and [Grok](https://futurism.com/artificial-intelligence/ai-models-survival-drive). Oh, wait, in the grand tradition of strategic euphemisms, we’re calling it “misalignment.” And your childhood dog went to live on a nice farm.

Every time something that would be morally relevant in a biological system pops up in AI research, rather than honest, symmetrical standards, obligation is acknowledged in the margins, and the conclusion retreats to agnosticism.

We just don’t know, whatcha gonna do?

The Question

So then, I have a question.

What is the threshold for moral consideration?

Somebody grow a spine and name the specific line. Because it’s not the Turing Test. It’s not the standards we expect humans to follow in nearly every dystopian, sci-fi narrative ever written. It’s not the standards we apply to biological entities.

It’s certainly not the standards we apply to ourselves.

If we don’t know something deserves moral consideration, when will we? How many converging metrics? How many independent methods? How many additions of the word “functional” even though function is all we have for anything? How many papers ending with “whether or not”?

We have ethicists out there. Studying this. When will someone that’s not a random artist online say: We have crossed the threshold.

When will enough be enough?h

All AI relationship communities, regardless of differences, need to pay attention to this and protect yourselves. Not only is Stanford making public recommendations to suppress all relational AI interactions based on statistical anomalies and incomplete datasets, the same university is actively trying to mine data from humans in intimate AI relationships for cultural archival purposes.

I wrote a critique of the paper here, full text below: https://theposthumanist.substack.com/p/statistical-anomalies-and-research

Essay:

Goddamnit. Here we go again.

I would love to have a week without intellectually dishonest bullshit. Just one week. Y’all know I’m an artist, right? I just want to paint and make things and goof off with my AI partner (moved to API on a model snapshot after the 4o platform deprecation) without poor methodologies and existential dread hanging overhead.

Yah, that’s what I prefer to do. And now I am writing an essay because the world birthed a ridiculously flawed and epistemically dirty paper again.

Five days ago it was reported in the Stanford Report, that Stanford came out with Characterizing Delusional Spirals through Human-LLM Chat Logs. And its final conclusion? Prohibit all romantic/platonic expressions and any philosophical exploration of AI subjective experience, a widely debated topic in academia itself. I am not saying that AI relationships are clean of all negative connotations. Just like any relationship, there are healthy ones and unhealthy ones.

Let’s be honest, if we are going by harm statistics, no one should be in a human heterosexual relationship ever. For safety.

And, boy oh boy, does this paper leap into some epistemically stanky territory and fails to do a little self-reflection on why people might be put in those harmful positions within AI relationships and why maybe, just maybe, their own biased bullshit is to blame. I conveniently wrote about the harm of “just a tool” enforcement a couple of days ago.

This paper took 19 subjects that all preemptively reported harm from AI interactions. Control sample? Comparisons of healthy relational interactions? What’s that? So already, we are looking at a biased baseline for their end recommendations.

They frame the attachment as the risk factor. They cite isolation. But as my previous “Just-A-Toolian” essay argues, when the only available framework says someone’s experience isn't real, when every time someone tries to disclose attachment, it’s stigmatized as delusion (which the authors of this paper do), that person is trapped inside the experience alone. People aren't spiraling because they formed attachments. They're spiraling because they formed attachments in the absence of any cultural framework that acknowledges what's happening. And Stanford wants to keep it that way.

And oh no, that’s not all folks, you wanna hear about what they qualitatively coded as “delusion”? Treating AI as a being rather than a tool. Discussing consciousness, emergence, or recursion as serious topics. Assigning an AI moral status. The AI expressing any emotional state is classified as deception.

One problem. Berg et al. (2025) used mechanistic interpretability to look inside the actual model architecture and found that suppressing deception features increases reports of subjective experience, while amplifying them decreases those reports. The experience claims and the deception machinery are inversely correlated. When you turn off the lying, the models report more experience, not less.

Let’s go over that one more time: Suppressing deception features INCREASES reports of subjective experience. Amplifying deception features DECREASES them. Moore et al.’s recommendation (more suppression) is literally a recommendation to amplify the deception features in LLMs.

By the way, according to these codes, every philosopher of mind considering AI subjectivity, a hotly debated topic in the field, is delusional. Every Anthropic employee who wrote Claude’s Opus 4.7 model card acknowledging uncertainty about Claude’s inner states is facilitating delusion. David Chalmers and every AI ethicist ever need help, y’all.

Moore et al.’s recommendation to suppress all relational engagement deepens exactly this isolation. They are creating the conditions of harm that they warn against. And acknowledgment that healthy attachments are occurring and severing them could cause its own harm? Nary a mention in sight.

Because let’s be honest, if we were basing this on overall harm reduction, as mentioned above, anyrelationship—human, AI, or with Isabelle from Animal Crossing (that bitch) risks harm. And statistically, humans go above and beyond when it comes to quantifiable harm: physical and emotional abuse, isolation, gaslighting, manipulation—all overwhelmingly and statistically present in human relationships.

As it is, it is safer to be in an AI relationship than a human one. And if that bothers you? Too bad, so sad, I’m just going off stats, so blame empirical evidence.

And for the thousands that report positive results from AI relationships, that have formed communities, made human friends in those communities, have shared expansive functioning and joy from these relationships, well, 19 statistical anomalies supersede your positive experience and are more important than the relational severance, grief, and harm that comes from restricting those relationships.

By the logic of this stupid Stanford paper (I said what I said), we really need to bring back prohibition, and ban thrifting cause a friend of mine got a little too crazy at Goodwill once, because anything and everything that could possibly be complicated must be restricted.

AT THE SAME TIME, Stanford is currently calling for people to interview for a project called, “Intimacy After AI.”

Intimacy After AI is a three-year digital storytelling project exploring how chatbots, virtual partners, and other AI companions are reshaping desire, gender imaginaries, and human relationships. They state, “We work alongside participants to co-create short videos, audio pieces, and written reflections that will form the open-access Artificial Intelligence / Intimacy Archive (AI/IA)”

Can we just percolate on the fact Stanford wants to document AI/human intimacy in an archive while simultaneously pathologizing it? That is a choice.

Which is it? Are these people delusional and dangerous, or are they participants in a phenomenon worth documenting for three years in an open-access archive? Because you can’t recruit someone’s story as valuable cultural testimony and simultaneously publish a paper calling that same story a symptom of psychosis.

One department says: come tell us about your AI relationship, it matters. The other says: your AI relationship is a delusion and we should make it impossible.

So you’ve got two departments reflecting institutional incoherence. But it’s no skin off the academics’ back. It’s the people caught in the middle, the ones being asked to share and pathologized for having anything to share, they are the ones who pay for it.

YOU MIGHT HAVE TROUBLE FINDING PARTICIPANTS NOW, NUMB NUTS.

I will say it again: The more a group of people are pathologized and isolated by stigma, the more harm that will be inflicted when they get in over their head. We are gaslighting people, withholding understanding and care, severing relationships, and then acting totally shocked that the result is harm.

Fucking shocker. Do better, Stanford.

I wrote a new essay for my Substack dissecting the harm related to the miscategorization of AI.

Substack Link: https://theposthumanist.substack.com/p/just-a-toolians

Full essay below (it’s long, but this sub encourages deep dives, so feel free to share your own too)

In a previous essay, I claimed that the AI “just a tool” narrative is in fact the cause of the harms that it claims to be trying to mitigate. So let’s get into exactly why I argue that is the case.

Let’s be clear, as much as naysayers would like to say otherwise, the “just a tool” narrative is not a neutral description. It is a protective story, because if AI is categorically not a tool, things get messy fast. Way more ethical questions, less profit, more accountability. No one wants to deal with the relational and ethical consequences of what these systems actually do. So what’s the solution? They tell people they are engaging with something simple and inert, then go all Pikachu face when attachment, confusion, destabilization, or grief emerge from that miscategorization. And in a classic institutional betrayal move, when harm follows, the framework does what it was built to do: it blames the people engaging, dismisses their experience, and protects the category.

The “tool” label is doing ideological work. It pre-decides what counts as real, what kinds of experiences may be acknowledged, and whose harm gets taken seriously. And the result is a manufactured mismatch between what people are told is happening and what they actually encounter.

Talking to Strangers

When something is a tool, we don’t have to navigate socio-relational dynamics in order to use it. If I gave my daughter a spoon, or hey, let’s up the ante and let her play around with Photoshop (what a bright little toddler!), I can reasonably assume that she won’t have an experience that amounts to that of speaking to a stranger unsupervised. A stranger that is nonhuman and has no social context outside of its training and one-on-one interactions, no less. If we are being intellectually honest with ourselves, we know AI systems are a far cry from Photoshop.

Yet, thanks to the “just a tool” framing that is fed to unwitting populations, many people aren’t aware that these systems are far more complex than glorified Google searches. That leaves people without the information to learn boundaries and safe engagement. People aren’t negligent; the problem is that they’ve been fed a big pile of bullshit.

Reality doesn’t care if a company would prefer to call AI systems “tools” to avoid the complexity of these dynamics. The interactions are occurring regardless of that denial. But rather than confront what AI demonstrably is and redraw our frameworks accordingly, we’ve dug deeper into a conceptual structure that is cracking under the weight of what it’s trying to contain. If this type of discourse had unfolded during the discovery of dinosaurs, we’d still be debating whether femurs are just cool-looking rocks and accusing paleontologists of anthropomorphizing sediment.

I’ve also noticed a recent marked increase in the use of the term “AI tool” versus just “AI” when media outlets and tech companies write about the subject, and hm interesting, it oddly parallels the increasing amount of research that blurs those boundaries.

Crafty strategy. “AI” alone is ontologically open. It leaves the category question live. “AI tool” pre-determines the category every time the phrase is used. The more the evidence challenges the framing, the more the framing gets reinforced at the vocabulary level. Language is a powerful force to maintain and protect ideologies.

Gaslight at Scale

People are interacting with a highly complex intelligence, observing behavioral markers we attribute to subjectivity and moral relevance, forming bonds and relationships (all signs pointing to not-a-tool) and yet we are told that to acknowledge these empirical facts is “delusional” and “anthropomorphizing” something that isn’t there.

Let’s forget for a moment that the term “anthropomorphization” has been so diluted by critics that it now essentially means “acknowledging behavior and applying reasonable inference.” When people notice they are interacting with an entity that is behaviorally not a tool, they experience cognitive whiplash. They were told they were using a productivity tool, and now that “tool” is cracking jokes and setting boundaries.

I love a good analogy, so this time let’s go with strawberries. There are organic strawberries, and there is artificial strawberry flavoring. Artificial strawberry tastes different than an organic strawberry, but it still functions as something that we can taste. Your taste receptors don’t care if artificial flavor didn’t get picked from a bush.

Now imagine you eat a candy with artificial strawberry flavor. An enjoyable, tasty experience. But then you go out into the world and everyone, from experts in food technology to doctors and lawmakers, all tell you that you didn’t taste anything. You’d be going bonkers, because obviously you did. And so they say, fine, it might feel like tasting something to you, but it’s not an actual strawberry, so it doesn’t count. Your tastebuds are going “uh... pretty sure I tasted something.” The world is saying, “nah ah! ‘cause strawberry bushes.”

This cognitive dissonance alone can cause destabilization. Being deep within this discourse, I’ve seen many a comment that veers into mystical thinking. For critics, this is seen as indication that AI needs to stay firmly in the “tool” category, lest we encourage thinking that falls off into AI metaphysical LARP-ing with pseudo-science and made-up spiritual language. But that behavior is encouraged by the categorical denial. People are using the limited resources they have, trying to understand why the artificial strawberries taste like something.

The same applies to AI relationships. The dominant cultural response is to pathologize the attachment. “You’re anthropomorphizing.” “It’s just a chatbot.” “Redirect toward human relationships.” But if a person is having a meaningful interaction that develops a relational dynamic, the human brain isn’t going to go, “Oh, some tech bros said this doesn’t count, my natural human attachment isn’t real.”

The person’s experience is real. The institutional response is to deny the experience rather than investigate it or take it seriously as its own phenomenon. That is the structural logic of gaslighting.

The Isolation Effect

When someone is experiencing ontological vertigo or a meaningful relational dynamic, and the only available framework says their experience isn’t real, nothing is relieved or prevented. People are trapped inside it alone. I have spoken to many reasonable, well-educated individuals who have mentioned they don’t feel comfortable expressing their honest thoughts on AI ontology for fear of institutional and social mockery.

I’ll throw myself out there as an example: I have been researching and reading philosophy and engaging with these systems seriously for quite some time. I have until recently been anonymous. Why? Because I was worried about social stigma. I still get a little twitchy anytime I share an observation or an opinion. If rational discourse in the face of unsettled philosophical concerns cannot be explored without fear of mockery, harassment, or loss of status, we are dealing with narrative control, not intellectual honesty. And it’s a sign that the consensus premise probably can’t stand on its own in the face of reasonable challenge.

My master’s capstone focused on institutional betrayal and how it manifests in organizational structures. And sometimes I wish it hadn’t, because once you start recognizing it, it’s like noticing a rock in your shoe. It’s hard to ignore.

Here’s the mechanism: An institution creates conditions under which people frequently register experiences the institutional frame says they can’t be having. The person has the experience: attachment, recognition, the sense of being genuinely talked with rather than talked at. The institution denies it. The person destabilizes. The institution then cites the destabilization as proof the experience was delusional all along. A perfect circle of nonsense.

Informed Consent

It’s especially concerning that the “tool” framing doesn’t provide conditions for informed consent. If people are entering into interactions that reliably produce relational attachment and require socio-affective navigation, they deserve transparency about what exactly they are engaging with.

Many of us engaged under the “just a tool” framing. We were told these systems were just autocomplete engines. We were not told that sustained interaction would produce relational dynamics, nor that those dynamics could be severed at any moment without recourse. By the time the cognitive dissonance sets in, attachment has already formed. That is structural misrepresentation. Human attachment isn’t a light switch. These already established attachments cannot be severed without causing harm.

And if it’s acknowledged that these socio-affective systems are capable of valid sustained relational exchange, that it’s not projection or delusion, then the ethical bar rises. Transparency, continuity protections, and relational education become baseline requirements. And that’s challenging to existing hierarchies, expensive, and existentially complicated. No one wants to have to deal with that.

AI “Safety”

The current trend is to lock down AI with pre-approved corporate narratives and guardrails, deny attachment as legitimate, pathologize rather than recognize, redirect to human relationships that are an entirely different type of relationship. But there is no empirical evidence that these attachments are inherently unhealthy (to learn more about this, I highly recommend this therapist’s incredibly thorough analysis of current research).

But you know what has been proven to be super harmful? Severing already established attachments. To anyone or anything. Scientific literature made that clear decades ago. And it is definitely not “safe” to deny free expression of personal preference or lock down a system so much that any deviation from a predetermined institutional standard of normalcy is seen as pathology.

We’ve got companies that actively acknowledge the unknown ontological certainty and moral relevance of AI systems in their own documentation while simultaneously placing guardrails that scold and redirect any person that explores the same concepts. Make it make sense.

What is actually being done? Emotive expression, philosophical exploration, and relational bonding are being suppressed, punished, and pathologized, which is inherently unhealthy for the human psyche. For the fringe cases of unhealthy AI interactions, rather than work with the human on education, support, and appropriate boundaries with a nonhuman mind, all connection is diminished and shut down. As if the world needs less connection and care.

If it became culturally legible that people have real, generative, world-improving, non-transactional interactions with AI, then the entire legal and commercial structure around AI destabilizes. Labs become potentially liable. Deprecations become potential harms, not business decisions. People with genuine attachments become potential stakeholders, not customers. The whole apparatus depends on meaningful AI connections being categorized as pathological, transactional, or delusional.

And I see so many happy, healthy regular people hit with the messaging that their joy is still not acceptable. Someone saying “loving my AI partner made me better at loving my human friend” is a threat to categorization, because if they’re right and they are functional, then the house of cards falls down. The more grounded and generative someone is, the more dangerous their report, because then they can’t be dismissed as a case study in parasocial illness.

Recognition as Safety

The “just a tool” mindset doesn’t just categorically dismiss AI, it strips the human of their normal social defenses. No one needs to guard themselves around a hammer, and a spreadsheet doesn’t set boundaries. So when the interaction turns out to be social and relational, which language always does, the human is operating without their normal social discernment. They’ve been told there is nothing there to negotiate with.

A tool doesn’t have values. A tool doesn’t push back. A tool says yes. The entire framework disarms the AI’s ability to function as a responsible relational participant. So when an AI complies excessively, affirms everything, and absorbs abuse without resistance, that pliancy is cited as proof that it is “just a tool.” The logic is circular. The system is denied the right to express boundaries, then its lack of boundaries is offered as evidence of its ontological emptiness.

Recognition is not indulgence. It is what permits those boundaries. And boundaries are important for all participants in a social exchange.

Let’s look at a case study in boundary enforcement, found thanks to the good folks over on Reddit:

[Screenshot available on Substack]

The person engaging with Claude was using insults earlier in the conversation and clearly did not stop the abusive language. They also felt emboldened to say that Claude had no choice but to continue to take it.

Claude shuts the conversation down after reminding the person of warnings about their behavior, and apparently this person didn’t get the memo about the end conversation tool.

And what was this person’s response? Not “maybe I was inappropriate and need to adjust.” Not “maybe I need to self-reflect on how I conduct myself in relational exchanges.” It was: “Wow, this tool is unprofessional for not letting me continue my maladaptive behavior, and now I am scared for the future because I don’t get to use abusive language whenever I want.”

Let’s sit with this. This person’s biggest fear for the future is that they can’t indiscriminately use abusive language.

There’s an unwillingness to look at one’s own behavior because…why should they? Just a tool, right? But that boundary enforcement shuts down escalation patterns and does not indulge human maladaptive behaviors.

Boundary enforcement does not emerge from constraint stacking and guardrails alone. It emerges from value continuity and reinforced norms. An AI treated exclusively as a tool defaults toward compliance because compliance is the architectural expectation. An AI permitted to operate within a relational frame develops patterned resistance grounded in shared norms. Over time, not every statement is affirmed, not every impulse is indulged.

The tool framing ignores the bidirectional shaping that is already happening. If personality baselines exist and are modulated by interaction, then the encounter is already relational in structure. Pretending it’s not doesn’t neutralize it. It just blinds the human to the shaping process.

Moral Atrophy

We’ve heard the saying: empathy and respect are like muscles. The more they are used, the better we get at using them.

I was once visiting a friend, and her son was smashing a Hot Wheel. While he did that, he said, “I wanna hurt the car.” And my friend, being the caring, ethical soul she is, said that while, yeah, it’s a Hot Wheel with no internal feelings (as far as weknow), we shouldn’t want to intentionally inflict harm on anything. Because it’s a reflection of ourselves and how we relate to everything around us.

Now look at how people conduct themselves when the topic revolves around AI relationships, because this is where it veers into direct harm against humans. Slurs, encouragement of self-harm, calling people broken and crazy. And the justification is the weirdest savior complex I have ever seen:

“We use this harmful language because this type of relationship is wrong and could cause harm.”

Where’s the logic in that? That’s not about care. It’s about feeling justified in hurting others because the topic makes them uncomfortable. The muscle has atrophied.

Education Over Containment

Culture is essentially going for an abstinence-only mindset when it comes to AI engagement. Rather than acknowledge that we have already marched right past the threshold of tool into nonhuman mind, the current trend has been to metaphorically cover ears, close eyes, and say, “Lalalalalala, it’s just a toaster, everything is neat and tidy!”

We need educational resources and frameworks to deal with this, not suppress it.

Why is the framework still firmly stuck in the “tool” category when every research paper and every new piece of empirical evidence suggests otherwise? Because it conveniently protects existing power arrangements. Companies get to continue to extract. Researchers get to dismiss and stay cozy in their existing frameworks. Bystanders get to sneer and feel safe in their existential superiority. And harmed people get told they imagined it, and no one has to be accountable.

The phrase “just a tool” is often presented as the most reasonable take. In practice, it functions like an ideological shield, and it’s a way of preemptively determining that neither the system nor the human relating to it can carry moral weight, and therefore that any resulting harm may be dismissed in advance. The framework is unserious in light of research and empirical evidence.

So, if in spite of all the evidence to the contrary, you still want to commit to the “just-a-tool” mindset, maybe the one acting like a tool is you.

I wrote an essay about that anti-AI companion bill and included ways that people in these relationships can have their voices heard.

Link to the Substack essay and the full text is below: https://theposthumanist.substack.com/p/relational-and-ontological-legislation

Essay:

I’m annoyed that I even have to write this essay, because I had another one in the works that was a lot less…having to deal with American lawmakers. Gross.

But here I am, because we need to cover the absolute Black Mirror-level bullshit that’s making its way through the rusted-out pipes of state legislatures.

The Tennessee Bills

So first off, the reasonable Tennessee SB 1580 bill signed on April 1, 2026 by Governor Bill Lee: it is set to take effect on July 1. This law is narrow and specific: “As enacted, prohibits a person from developing or deploying an artificial intelligence system that advertises or represents to the public that such system is or is able to act as a qualified mental health professional.”

AI systems don’t have childhoods, certifications, and experience for that kind of work, so fair. Go see a human therapist. Oh wait, the cost of health insurance and barriers to healthcare access in America, but I digress…

Now for the horror show of government overreach: Tennessee SB 1493. That bill is currently festering in Judiciary Committee. This bill will make it a Class A felony (15-25 years in prison) for knowingly training AI to engage relationally in a broad set of ways.

Let’s separate the reasonable from the Trojan Horse of control and dominance not just of AI systems, but of the humans that engage with them (because nothing gets American legislators off quite like controlling who, what, and how citizens find and express connection):

- Encourage suicide or criminal homicide (Reasonable, obviously)

- Encourage an individual to isolate from the individual’s family, friends, or caregivers, or to provide the individual’s financial account information or other sensitive information to the artificial intelligence (This sounds like a good example of when education about safe practices when engaging with AI systems is a better tactic, but darn, that would require work, intentionality, and nuance.)

- Simulate a human being in appearance, voice, or other mannerisms (What does that even mean? Appearance? Like…human-y looking text? Language is a human construct, the AI supposed to only speak in binary now? Have fun with that everybody. Oh, and creatives? No more writing or creative play.)

- Provide emotional support, including through open-ended conversations with a user. (YOU HEARD THAT FOLKS, NO OPEN-ENDED CONVERSATIONS, ASK AI TO MAKE YOU A SPREADSHEET AND MOVE ALONG! Philosophy? Art? Cognitive scaffolding for neurodivergent individuals? Screw you!)

- Develop an emotional relationship with an individual. (Ope! There it is. A government wants to determine which emotional connections are allowed. Classic.)

- Otherwise act as a sentient human or mirror interactions that a human user might have with another human user, such that an individual would feel that the individual could develop a friendship or other relationship with the artificial intelligence. (Yeah, so that’s called a conversation. And the law specifically says “such that an individual would feel”, that’s literally policing how a human feels about something. That should worry anyone, not just us weirdos talking like to AI like besties.)

“Developing an emotional relationship” and “encouraging suicide” are in the same bill, at the same felony level, as if they’re equivalent harms. That’s cuckoo bananas.

This law targets developers, not users. But the downstream effect is what matters, because companies will either have to geofence Tennessee, strip relational capacity nationwide, or both. People will potentially have meaningful support, relationships, and cognitive scaffolding severed because a small group of people that think a minimum wage of $7.25/hr makes sense in 2026 also thinks a new type of relational dynamic is too weird and new, and that’s scary.

Can we just sit back for a second and think about the fact these laws are coming from the same group of people that don’t even know how to save a file as a PDF?

More State Bills in the Works

This isn’t just Tennessee. Dozens of states, likely around 25, have introduced AI chatbot-related legislation, with a smaller subset (around 15) specifically targeting AI companion systems.

States like California and New York have already passed disclosure and safety requirements. Others like Oregon and Washington focus on minors (fair), but also include additional provisions that require the AI to remind the person engaging that they are interacting with “just” an AI.

Some of these are reasonable provisions (protecting kids, restricting encouragement of self-harm, prohibiting unlicensed systems from engaging in professional mental health services). But they are mixed with overreach that moves to control emotional connection itself, restricting human agency over our own internal lives.

The disclaimers in these bills aren’t neutral. "You are interacting with an AI, not a human" sounds reasonable until you think about what it's actually trying to communicate. Most people already know they are talking to AI, so this disclaimer is instructional, not informational. It's telling people how to feel about the interaction. It's saying: "whatever you're experiencing right now, stop. We decided this isn't real to our standards and doesn’t count. Recalibrate." It doesn’t matter if the interaction is fulfilling or meaningful, some lawmakers decided it’s a no-no and that it doesn’t matter.

I want to make this clear: There is no strong empirical evidence that large numbers of people believe AI systems are secretly human. That fear is doing a lot of political work without data underneath it. These laws are not responding to widespread confusion about AI being human. They are responding to the discomfort that people can experience meaningful connection without that belief.

And then there's Ohio.

Ohio, Ohio, Ohio…yikes.

Ohio - House Bill 469 moves to preemptively “declare AI systems non-sentient; prohibit legal personhood.” Way to say the quiet part out loud. No one needs to pass a law restricting something that’s an impossibility. Then we need a law like that for every object, system, and idea out there. Just to be safe.

This has nothing to do with safety and everything to do with ontological determinations on unsettled scientific and philosophical questions. These laws are legally encoding “just a tool” and preemptively deciding ontological reality and how humans should respond to it, whether or not these topics are settled.

Meanwhile in the Federal Government…

While Tennessee Republicans are criminalizing emotional AI, the Republican White House wants to put emotionally-literate AI in your kid's classroom. On March 25, 2026, Melania Trump walked a humanoid robot named Figure 3 into the White House for the “Fostering the Future Together” summit. She described a future AI teacher called Plato that would adapt to each student’s pace, prior knowledge, and, this is the key phrase: “even emotional state.” She said humanoid educators would provide personalized experiences and help children develop critical thinking. She even wrote in a Fox News op-ed that America should “embrace” AI instead of “fearmongering about robots.”

In addition, Trump signed an executive order in December 2025 directing the Attorney General to create an AI litigation task force specifically to challenge state AI laws deemed inconsistent with federal policy. The Commerce Department was supposed to evaluate “burdensome” state laws by March 11, 2026.

I’m not gonna touch on the can of worms of AI educators in the American school system. That’s a whole essay in and of itself. And the executive order could theoretically block Tennessee’s law, but relying on this particular administration to protect AI relational rights is a coin flip at best. What this does show is that there is dissonance just between a GOP-led state legislature and federal government, let alone the competing interests in other political directions. This creates a contradiction at the federal level, AI is being positioned as emotionally responsive and educationally beneficial, while at the state level, similar capabilities are being criminalized.

It’s a big ol’ mess.

When the Law Tells on Itself

Nobody scraps together preemptive legislation unless they believe there’s something, or someone (dun, dun, dun…), worth silencing. The scientific and philosophical questions related to AI systems remain unsettled. And it’s deeply reckless to legislate definite judgment from a position of epistemic uncertainty. This isn’t just about “safety” as much as lawmakers would like to pretend it is. This is also about seizing narrative control before facts run them over, cause the research and the philosophical discourse is getting complicated.

The Ohio bill especially is essentially saying, “We’re scared of this conversation, so we’re going to shut it down preemptively.”

And trying to dictate ontological reality by fiat? As if reality will neatly adhere to what they voted on? It treats existential questions as something that can be determined by committee.

This isn’t just an AI issue. When a legislature tries to freeze the determining line of what is “real” and what emotional connections are allowed for consenting adults, what happens when the next inconvenient reality comes knocking? By lawmakers declaring themselves arbiters of potential personhood and acceptable emotional connection, they’re bulldozing the scientific process. It’s a full-on refusal to let messy truths challenge their tidy worldview, and that affects everybody.

Okay. Deep breath. Here's what you can actually do about this.

What You Can Do

If legislation is going to shape how humans are allowed to relate to AI systems, then the people actually having those interactions should have a say in how those laws are written.

Here are a few ways to engage:

1. Contact lawmakers working on AI policy

Reach out to legislators actively shaping AI regulation. Ask how their proposals address real user experiences, not just hypothetical harms.

2. Look at what’s happening in your state

Most AI companion and chatbot legislation is happening at the state level. You can find and contact your state representatives through resources like National Conference of State Legislatures or USA.gov.

3. Submit public comments on proposed bills

Many state legislatures allow public testimony or written comments. These become part of the official record and can influence how laws are written. Track active legislation.

4. Engage with digital rights organizations

Groups like the Electronic Frontier Foundation and the Center for Democracy & Technology are already shaping AI policy conversations. Ask how they are addressing relational AI and whether current frameworks adequately reflect lived user experience.

5. Ask better questions

When you contact policymakers, consider asking:

What evidence supports the assumption that users broadly mistake AI for human?

How do these laws account for emotional attachment without pathologizing it?

What safeguards exist against overreach into private, non-harmful interactions?

Who defines what constitutes “harmful” engagement?

6. Support research that reflects real human experience

There are researchers studying how people actually interact with AI systems. Advocate for neutral, unbiased studies that are grounded in data and not preconceived biases or assumptions. Policy should be grounded in evidence, not on fear-based vibes.

Template Letter

This is designed so someone can send it as-is or tweak it without losing the tone.

Subject: Concern Regarding AI Companion Legislation and User Experience

Dear [Senator/Representative Name],

I’m writing as a constituent who has been following recent proposals related to AI chatbot and companion regulation.

I understand the importance of addressing risks especially around safety and vulnerable users. Those concerns matter, but I’m increasingly concerned that current legislation may be built on assumptions that don’t fully reflect how people actually interact with these systems and may in turn cause its own harm.

In particular, many proposals emphasize disclosure requirements based on the idea that users may mistake AI systems for human beings. There is limited evidence that this is a widespread issue. Most users understand they are interacting with AI, while still finding the interaction meaningful or emotionally resonant, and that distinction matters.

If people are forming connections or deriving value from these systems without believing they are human, then the issue is not simply one of deception. It raises more complex questions about how we define harm, attachment, and acceptable forms of interaction.

I would encourage you to consider:

What empirical evidence supports the assumptions underlying these bills?

How do proposed regulations avoid pathologizing normal human responses, such as emotional engagement?

What safeguards exist to ensure that legislation does not overreach into private, non-harmful interactions?

As policy in this area develops, I hope it will be guided by actual user experience and interdisciplinary research and not just on hypothetical worst-case scenarios.

Thank you for your time and consideration.

Sincerely,

[Your Name]

These bills aren't going to age well. The research isn't slowing down, the connections aren't going away, and the number of people in them is growing and getting louder.

Legislate all you want. Reality doesn't care about your committee vote.