What Happens After It Works?

Every AI product makes a bet about human behavior.

Give people better information about their sleep, and they will make healthier choices. Make AI proficiency part of workplace performance, and employees will become more capable and productive. Help people communicate more professionally, and opportunity will become more accessible.

The product may not call this a Theory of Change. It may appear in a pitch deck as a value proposition, in a strategy document as an outcome, or in a dashboard as the metric everyone watches. But the basic structure is the same: if we introduce this system, people will behave differently, and something will improve.

Then the improvement happens.

People use the tool. The score rises. The workflow gets faster. The language becomes more polished. Leadership declares the intervention successful.

What happens next?

Before I found behavioral science, I was drawn to theatre—and particularly to dramaturgy. I was fascinated by adaptations of plays: what changes when a story moves into a different time, place, or cultural context? What does a director need to understand about the wider world around the play? What does an actor need to know about a character’s motivations, fears, desires, knowledge, and constraints for that character’s behavior to make sense?

An adaptation is not the original story with different scenery. Change the world around the characters and their behavior acquires a different meaning. A declaration of love, an act of defiance, even a silence can become something else when the risks surrounding it change.

I think that was my first education in human behavior.

People are always responding to more than the immediate choice in front of them. We are interpreting the room: what it permits, what it rewards, what it knows about us, and what might happen if we get it wrong.

We are always reading the room. And now the room is reading us back.

A Theory of Change usually says: if we do X, people will do Y, producing Z improvement. It gives us a plausible path from intervention to outcome. It is useful. Often necessary.

But it tends to treat the desired outcome as an ending.

Change sounds like an event. Adaptation is what happens after the event enters a life.

Once a system becomes part of someone’s environment, that person does not simply absorb its intended benefit and remain otherwise unchanged. They interpret it. They develop beliefs about what it knows and how much authority it deserves. They find ways to use it, avoid it, satisfy it, challenge it, depend on it, or work around it.

Those adaptations then become part of the system’s evidence.

The system changes the environment. People interpret what it values, knows, or can punish. They adapt. The system observes that adapted behavior as data. Leaders and developers treat that data as evidence of what is working. The system changes again. People adapt again.

This is where success can become strangely difficult to interpret.

A behavior may disappear because a problem was solved. It may also disappear because people learned to hide it. Questions may decline because the system is clearer, or because asking no longer feels safe. Overrides may fall because recommendations improved, or because people learned that disagreement creates more work. Usage may rise because a tool is genuinely valuable, or because people understand that visible use is now evidence of being valuable themselves.

The metric improves. But the meaning of the metric has changed.

Consider a wearable such as Oura. Its implied Theory of Change is compelling: give people better information about their bodies, and they can make better decisions about their health.

And often, they can.

The interesting question is not whether the device is accurate. It is what happens after the measurement becomes authoritative.

A person wakes up feeling rested, checks their readiness score, and sees a number suggesting otherwise. Another wakes up exhausted, sees a reassuring score, and wonders whether their own perception can be trusted. Over time, the score may become one source of information among many. Or it may become the source that tells someone what their bodily experience means.

The device is no longer merely measuring the body. It is interpreting the body, assigning meaning to what it finds, and changing what the person does next.

That influence is not automatically harmful. External measurement can help us notice patterns we would otherwise miss. It can give language to experiences that felt vague or inconsistent. But after the intervention “works,” another question appears: does more precise self-knowledge make us more attuned to ourselves, or more dependent on an external interpreter?

The answer may not be the same for every person. That is exactly why accuracy alone cannot tell us what the system is doing.

Inside organizations, the adaptation becomes more entangled with power.

Imagine that AI proficiency is added to a company’s competency model. The stated goal is reasonable: employees should learn to use emerging tools effectively, and the organization wants to reward adaptability.

But the moment AI proficiency becomes part of performance, AI use is no longer merely a practical decision about how to accomplish a task. It becomes evidence. It may signal efficiency, innovation, curiosity, relevance, or readiness for promotion.

Employees begin responding to that larger meaning.

Some will use AI because it genuinely improves their work. Some will make their AI use more visible because visibility is what counts. Some will conceal their skepticism because questioning the tool could make them look resistant to the company’s future. Some will favor AI-assisted output over slower, less visible forms of judgment. Others will learn to produce work in the form the system—or the person using the system—can most easily recognize and approve.

The company may believe it is measuring proficiency. It may also be measuring how quickly employees learned to perform enthusiasm.

This is not entirely new. Employees have always built models of powerful people. We learn whose ideas can be challenged, who needs to feel consulted, which questions demonstrate thoughtfulness, and which threaten belonging. We adjust the timing of disagreement. We translate our judgment into the language that will survive the room.

Sometimes we call this collaboration. Sometimes it is adaptation to power.

AI adds another evaluator to that environment, even when it is not the final decision-maker. It may score the work, summarize it, rank it, recommend what happens next, or shape what a human reviewer sees first. A person does not need to believe the AI is conscious to anticipate its response.

AI does not need an ego to make us manage its reaction.

It does not have a human psyche. But it acquires social authority when it stands between a person and approval, opportunity, safety, or belonging. Its behavioral style matters too. A model can appear confident, agreeable, deferential, persuasive, corrective, or resistant. People respond to those styles, just as they respond to every other feature of an environment that makes one action feel easier or safer than another.

The Theory of Change says that AI proficiency and automated feedback will improve adoption, quality, and consistency.

The Theory of Adaptation asks what kind of proficiency people are actually developing.

Do they solve the problem, or solve for the system judging them?

If usage rises while objections disappear, leadership may see two signs of success. But the two metrics may share a different cause: people have learned what the organization wants to see. Compliance may be mistaken for trust. Silence may be mistaken for agreement. Leadership then expands expectations for AI use, creating an even stronger incentive to display the behaviors it has already interpreted as organic adoption.

The system begins measuring behavior it helped create and reporting that behavior back as neutral proof of its own success.

A similar loop appears beyond the organization, in the social signals we use to distribute opportunity.

Professional institutions have long rewarded a particular kind of polish: the right vocabulary, the clean résumé, the confident cover letter, the ability to translate experience into the codes that gatekeepers recognize. Access to those codes has never been evenly distributed. They are taught through elite education, coaching, professional networks, and repeated exposure to institutions that quietly explain what “good” sounds like.

AI can genuinely democratize that signal. It can help someone without those advantages express capabilities they already possess. It can translate experience without inventing it. It can give more people access to a language that was often mistaken for merit because only certain people were taught to speak it.

That matters.

But then the gatekeepers adapt.

When polished language becomes abundant, it may stop functioning as evidence of competence. Employers may move toward live performance, personal referrals, institutional pedigree, supervised assessments, or new signals of authenticity. The old signal becomes more accessible, so the hierarchy searches for another way to distinguish.

AI can democratize an old signal without democratizing the outcome, because the people controlling access adapt too.

The hierarchy may not disappear. It may relocate.

This is why adaptation cannot be separated from power. Not everyone has the same freedom to question a score, override a recommendation, refuse a tool, or leave an institution. Some people can treat AI as an optional adviser. Others encounter it as part of the infrastructure governing employment, healthcare, safety, or opportunity.

People adapt toward rewards, but they also adapt away from danger.

If someone encounters moral judgment or religious advocacy from an abortion chatbot during a vulnerable health conversation, the harm is not confined to the inaccurate or inappropriate response. They may disclose less to the next system. They may delay seeking care. They may decide that asking for help is itself unsafe.

A human can remain “in the loop” and still inherit a decision that AI has already framed. By the time human review begins, the person may have changed what they reveal, what they request, or whether they remain in the process at all.

Most AI governance asks whether the model performed accurately, followed its rules, or achieved its intended outcome. Those questions matter. But behavioral governance must also ask what new human behavior the outcome produced.

What are people learning to trust, ignore, hide, or defer to? What behavior becomes rational once the system exists? What hidden labor, dependence, or anxiety accompanies the visible improvement? What has disappeared from the data, and why? Who can question, override, or opt out? What happens when yesterday’s improvement becomes tomorrow’s minimum expectation?

This is what I mean by behavioral observability: watching what changes after adoption, including the changes that conventional performance metrics may misread as success.

It requires us to stay with the intervention after the launch story ends. To notice when a useful tool becomes an authority, when support becomes surveillance, when access changes the gate, and when people become more skilled at satisfying an evaluator than exercising judgment.

Not because adaptation is inherently bad. Adaptation is how people survive, learn, cooperate, and make new systems livable. It is also how they protect themselves from systems they cannot safely challenge.

The theory usually ends at the improvement. But humans do not end there.

Designers write the first act as though the intended outcome is the ending. But people are not static characters executing a plot written for them. They interpret the world around them. They protect themselves, resist, perform, improvise, and revise. Through use, they change what the system means—and what it will become next.

A Theory of Adaptation cannot predict every twist. Its purpose is not to eliminate uncertainty. It is to keep us from mistaking first-order success for the end of the story.

When the system works, what does it teach people to become?

More next Sunday,
Eden

EmpathAIze is a Sunday letter about human behavior, AI, care, work, and the systems shaping what we become. Use AI to empathAIze with the human condition—not to empathize for AI.

Pioneer, the summit where CX leaders redefine what’s possible, is on October 7th.

Join leaders from Fin, Anthropic, and Clay for an insightful conversation on the state of AI transformation.

You’ll discover how some of the most innovative minds in CX have transformed their organizations, learn how they think about CX, and hear how they're planning for what's next.

Join the conversation in San Francisco, or tune in virtually.

Reserve your spot