---
title: "Milieu and qualia"
description: "Environment and felt-experience as design primitives for agents."
doc_version: "1.1"
last_updated: "2026-05-01"
canonical: "https://modiqo.ai/blog/milieu-and-qualia.txt"
---

# Milieu and Qualia: A Personal Note on How We Built Rote

*by Chetan Conikee*

---

I spent most of the last two years trying to answer a question that sounds simple and isn't: why do agents feel so brittle?

Not why do they fail — the failure modes are well catalogued. Context windows fill up. Tool outputs drown out instructions. The agent confidently continues through a broken state. The same error on the same API gets rediscovered on Tuesday that was already solved on Monday. You can put numbers on all of this, and I have: roughly 12,000 tokens to rediscover through trial and error what 250 tokens of crystallized workflow could replay. That's a 97% tax on every repeat problem, paid to the meter of whoever sells you tokens.

But numbers aren't diagnosis. The question I kept coming back to was *structural*. What is actually missing? Why does a senior engineer, dropped into an unfamiliar codebase, find their footing in a week, while an agent starts from zero every session no matter how many times it's been there? It isn't intelligence. The models are good. It isn't memory, exactly, because bolting RAG onto the context window doesn't fix it. Something deeper is absent — something that every living thing that has ever navigated a world takes for granted, and that we've been quietly assuming agents can do without.

I found the vocabulary for it, eventually, in two words that predate the transformer by about a century. They come from philosophy and biology. They are **milieu** and **qualia**. And once I could name them, I could see what Rote had to be. The primitives we ended up shipping — adapters, canvas, flows, hub — are not arbitrary abstractions. Each one maps to a specific piece of what human cognition has and agent cognition lacks. This essay is my attempt to show the map.

I want to spend real time on the philosophy before I talk about code. If I don't, the primitives will sound like just another agent-tooling stack in a crowded market, and they're not. They're an attempt to give agents something half a billion years of evolution figured out, and that every framework I've seen has quietly skipped.

---

## Part I — What humans actually do when they perceive

### Qualia: the texture of being-here

Philosophers have argued about qualia for fifty years, and they've argued well. Thomas Nagel framed it in 1974 with a question that still hasn't been answered: what is it like to be a bat? Not what does a bat do, not how does echolocation work — what does it *feel like* from the inside? David Chalmers called this the hard problem of consciousness. Frank Jackson tried it from the other direction with Mary, the color scientist who knows every physical fact about red but has lived her whole life in a black-and-white room. When Mary steps outside and sees a red tomato, does she learn something new?

Most of us think she does. That something is qualia — the redness of red, the painfulness of pain, the particular way coffee tastes on a Tuesday morning when you haven't slept. Qualia are the raw felt qualities of experience. They are subjective, private, and irreducible. You can describe the wavelength of 650 nanometers in perfect detail, and the description still won't contain the redness.

What I find most interesting about qualia isn't the metaphysics. It's the practical point hiding underneath. **Qualia are how the world becomes legible to you.** They are the interface layer between raw physical stimulus and something an organism can actually act on. Without qualia, 650nm of electromagnetic radiation is a number. With qualia, it's *ripe*, or *stop*, or *warning*, or *her lipstick on a Friday night*.

Every organism has its own set. A dog's qualia for smell are vastly richer than mine. A bat's qualia for sound — the echo-texture of a moth's wings — are ones I can't even imagine, which is exactly Nagel's point. Qualia aren't decoration on top of perception. They *are* perception, at the level where it becomes something you can do anything with.

### Milieu: the world that already has meaning in it

Qualia are only half the story. The other half is the concept of **milieu**, and this is where it gets really interesting for what we're building.

The French word milieu means "middle place" — *mi* (middle) + *lieu* (place). But in the hands of a few twentieth-century thinkers, it became something much bigger than "environment." Émile Durkheim used *milieu social* to describe the shared norms and expectations that shape how a person thinks and acts, arguing that our whole social environment is filled with forces that really exist only in our own minds as collective representations. Maurice Merleau-Ponty, the French phenomenologist, used it to describe the dialogue between an organism and its world — not the world as a neutral container but the world as already structured by what that organism cares about. And before either of them, a Baltic-German biologist named Jakob von Uexküll coined the term *Umwelt*, which is milieu's biological cousin: the species-specific subjective world that an organism inhabits.

Uexküll's favorite example was a tick. A tick has almost no sense organs. It can't see. It responds to three signals: butyric acid (the smell of mammal sweat), warmth (a body below), and the tactile sense of hair. That's it. Uexküll said those three signals "glow like signal lights in the darkness" of the tick's world. The tick's world isn't impoverished; it's *sufficient*. Every signal it can perceive is a signal it can act on. The poverty of the environment is the condition of the certainty of action.

Merleau-Ponty extended this into what he called the **organism-milieu structure**: the organism doesn't merely receive input from its environment; it actively structures that environment according to its needs and capacities. A piece of the world becomes a perch because a bird can land on it. A stone becomes a tool because a hand can grip it. James Gibson, the American ecological psychologist, called these *affordances* — possibilities for action that the environment offers relative to what the organism can do.

Here's the punchline, and it's the one that matters for Rote: **perception is not passive reception of data. It is the active co-construction of a meaningful world between a perceiver with specific capacities (qualia) and an environment structured by shared significance (milieu).** You don't see photons. You see doors to walk through, cups to pick up, faces to trust, slopes too steep to descend standing up. The world shows up to you pre-digested into the things you can do with it.

A human without qualia would be Mary in the black-and-white room — all facts, no texture, nothing that bites. A human without milieu would be a brain in a vat — rich inner experience, no world to act on. Cognition is the handshake between the two.

### Why this matters before we ever mention agents

If you've spent any time looking at modern cognitive science, you'll have noticed that the field is slowly rediscovering all of this. Predictive processing, active inference, 4E cognition (embodied, embedded, extended, enactive), ecological psychology — the vocabulary has changed, but the thesis is the same. Mind isn't a disembodied computation. Mind is what happens when a body with specific sensory capacities is situated inside an environment it has learned to read.

And this is where I started, three years ago, when I was trying to figure out why agents felt so brittle.

---

## Part II — Agents don't have a milieu, and they don't have qualia either

I want to make a claim that I think is strong but defensible: the reason agents fail the way they fail — burning 50,000 tokens to rediscover what they already solved yesterday, looping confidently through broken states, hitting the same date-format error on the same API for the hundredth time — is that they lack both of the things I just described. They have no qualia. They have no milieu. And every agent framework I've seen either pretends this isn't a problem or papers over it with more context and bigger retrieval.

### The qualia gap

When an agent encounters a REST API, what does it see? A text blob. A JSON schema maybe, if it's lucky. A few examples in a system prompt. It doesn't *feel* the API the way you feel a hammer. It doesn't have the taste of "this endpoint is rate-limited so approach gently," or the smell of "this field is nullable, always check," or the body-sense of "this one takes three seconds on a cold cache, plan for it."

For a human developer, those qualia are everything. They're what separate the senior engineer from the junior. The senior doesn't remember the documentation; they've absorbed the *texture* of the system. When something's off, they feel it before they can articulate it. When something's right, they move through it without thinking.

Agents have none of this. They have tokens. And every time they approach the same API, they are tasting it raw for the first time, because nothing from the last encounter crystallized into feel.

### The milieu gap

Worse than the qualia gap, agents have no milieu. They don't inhabit a world that has already been shaped by what matters. Every session starts in Mary's black-and-white room. There are no signal lights glowing in the darkness. There's just a context window — a flat, topologically naive slab of tokens in which everything is equidistant from everything else and nothing has gravity.

I've written before about how context windows have a physics, and about how they're becoming virtual machines whether we admit it or not. But those are structural observations. The deeper problem is that the agent doesn't *dwell* anywhere. It has no sense of place. It can't say "I'm in the Salesforce milieu right now, so these affordances are live and those are dead." It has to reconstruct that feeling — if it ever gets there — from the bottom up, token by token, every time.

Geoffrey Huntley's Ralph Wiggum loop is, in a way, a confession of this. You run the agent in a bash loop, let the filesystem and git history carry the state, and restart with fresh context every iteration. It works, and it's honest about what's actually going on: the milieu lives in the environment, not in the agent. The agent is a transient actor that gets repeatedly inserted into a world it didn't build and doesn't remember.

That's the diagnosis that drove Rote. The question then was: what would it take to give an agent something like qualia, and something like a milieu? Not in the metaphysical sense — I'm not trying to solve Chalmers — but in the functional, engineering sense. What primitives would an agent need to experience a tool the way a developer experiences a tool, and to inhabit a workspace the way a developer inhabits a workspace?

---

## Part III — How we built it

What follows is personal. It's how I actually thought through each primitive, and how we put the team together around them. It's not a launch announcement. The launch will come with its own deck. This is the note I'd write to a friend who asked: *why those four primitives, why in that order, why those people*.

### Adapters: the instrument of qualia

The first primitive I was sure about was **adapters**. I knew this one before I had the vocabulary to explain it.

An adapter in Rote is the thing that sits between an agent and an external system — an API, a database, a binary, a protocol — and makes that system *felt* in a specific way. It's not a wrapper. A wrapper just passes calls through. An adapter is opinionated. It knows that this endpoint lies about its rate limit, so it quietly paces. It knows that this field returns a unix timestamp on Tuesdays and an ISO string on Wednesdays, so it normalizes. It knows that when this service returns a 200 with an empty body, something has actually gone wrong, so it raises.

The adapter is the organ of perception. It's the thing that converts undifferentiated external data into *signs that the agent can act on* — what Uexküll would have called perception-marks. When we embed guidance directly in the tool as queryable metadata rather than shipping it as external documentation, we're doing this deliberately: the guidance is part of the interface, the way smell is part of food. It doesn't live in a separate manual you consult; it's in the thing itself, always in sync, always at hand.

I think of each adapter as a single qualia generator. It gives the agent a specific way of experiencing a specific system. A well-built adapter for Salesforce doesn't make the agent "know Salesforce." It makes the agent *feel* Salesforce — its rhythms, its traps, its tells.

The reason adapters came first is because without them, every other primitive is building on sand. You can have the most beautiful canvas and the most elegant flows, but if the agent can't *perceive* the systems it's acting in, it's going to burn tokens re-tasting the world on every turn.

This is also why adapters came first as a design question, not just an engineering one. The hard conversations on the team weren't about implementation. They were about what makes an adapter honest: under what conditions does it genuinely compress experience, and when does it just hide complexity in a way that will bite later? A wrapper that lies to the agent about the system it's wrapping is worse than no wrapper at all. We spent a lot of time on that line.

### Canvas: the milieu as workspace

If adapters are qualia, **canvas** is milieu in its most immediate form. It's the workspace where an agent dwells.

I agonized over this primitive more than any other. The naive version is "a context window but nicer" — which is exactly what I didn't want to build. The context window isn't a place. It's a queue. You can't inhabit a queue. You can only pass through it.

What I wanted was something that had a topology. A place where certain affordances were live and others were dead. Where the agent could know, in a structural rather than a prompt-stuffed sense, *where it was* and *what mattered here*. Something closer to a workshop than to a transcript.

The version of canvas we shipped is a first approximation of this. It organizes state and affordances into a space the agent can navigate rather than a stream it has to re-read. Tool outputs don't pile up as raw text; they settle into positions that relate to what's upstream and downstream of them. When the agent looks at a canvas, it sees something that has already been pre-structured by what the work is about. The canvas is Merleau-Ponty's organism-milieu structure rendered in software: the space is not neutral, because the work the agent is trying to do isn't neutral.

I'll be honest: I don't think we've finished canvas. I think we've built the first version of it, and I think the second and third versions will make the first look primitive. But I wanted to ship something that was correct in its shape before it was rich in its features. The shape is: a dwellable workspace, not a transcript.

### Flows: crystallized habit

**Flows** are where Rote's thesis gets most concrete. Every agent framework I've seen treats every execution as fresh exploration. An agent figures out how to query your database, format the results, and push them to your dashboard. Tomorrow, a different session does it again. Next week, a different developer's agent does it again. Each time, you pay the discovery tax.

Flows are the crystallization of that exploration into something deterministic. The agent explores once. We capture the successful path. We translate it into a reusable script. Thereafter, when the same shape of problem arrives, the agent doesn't re-explore — it invokes. The numbers are stark: a reified flow uses roughly 250 tokens against about 12,000 tokens for agent-driven rediscovery, which is where the 97% reduction figure comes from.

I want to connect this back to the philosophy, because it matters. What is a flow, phenomenologically? It's a habit. It's what Merleau-Ponty called the *body-subject* — the pre-reflective, skilled action that an expert performs without deliberation. When you've driven to the office ten thousand times, you don't re-plan the route. Your body knows the way. The planning circuit has been bypassed. You can use that cognitive budget for something else.

Flows give agents habit. They let the agent stop paying the discovery tax on solved problems and redirect its tokens to the genuinely novel. This is also why flows have to be first-class citizens, not metadata in a side-channel. If a habit is buried in documentation the agent has to retrieve, the agent will keep re-thinking rather than re-invoking. If the habit is part of the tool's interface — queryable, versioned, shipped with the binary — it will be used, because the path of least resistance is to use it.

The flow compiler has been the hardest part of the build, and the hard part wasn't translating a trace into a script. The hard part was deciding *which traces deserve to become scripts* — when exploration should crystallize versus when it should stay provisional. That's a judgment call the system has to make thousands of times a day, and getting it wrong in either direction is costly. Too aggressive, and you crystallize noise into false habits. Too cautious, and you never escape the discovery tax. The current version has instincts I trust, which isn't something I would have said six months ago.

### Hub: the shared milieu

The last primitive is **hub**, and hub is what turns Rote from a single-agent tool into infrastructure.

Here's the problem hub solves. An adapter is a quale for one agent. A flow is a habit for one session. A canvas is a workspace for one task. But a milieu, in the deeper sense — Durkheim's sense, or the sense Max Scheler pointed at when he distinguished between milieu as an experienced value-world and the objective social environment from which we build it — is *shared*. It's collective. It's what a team, an organization, a culture holds in common. It's why two engineers on the same team can communicate in half-sentences while the new hire needs a full meeting to get to the same place.

If every developer's agent has to independently discover the same integration pattern, you're paying the discovery tax fifty times instead of once. If every agent has to independently build up the feel for your internal APIs, you're generating fifty different, inconsistent, uncoordinated qualia for the same systems. That's not just wasteful; it's corrosive. It means your agents never develop the collective conscience that makes a human team effective.

Hub is the shared milieu. It's where adapters, canvases, and flows become organizational assets rather than session-local tricks. When one agent crystallizes a flow, hub makes that flow available to every other agent in the organization that could benefit from it. When one adapter learns the quirks of an internal system, every agent inherits that perception. The organization develops something that looks a lot like Durkheim's collective representations — shared norms about how our systems behave, shared expectations about how our work gets done, shared habits that constitute what "working here" means.

This is why hub had to exist, and why it had to come last. You can't build a shared milieu before you have anything to share. Adapters had to exist so there was something to perceive with. Flows had to exist so there was something to crystallize. Canvas had to exist so there was somewhere to dwell. Only once those three were real did hub have anything to federate.

---

## Part IV — The team, and why this had to be this team

I want to close with something that isn't technical, because I think it matters.

We're not experts in agents. Nobody is yet. We're experts in the primitives underneath them — and that turns out to matter more.

**Hubert Płociniczak** helped build the Scala compiler at EPFL in Martin Odersky's lab, which is the kind of credential you can only acquire by actually going inside the thing that compiles the languages other people ship products in.

**Roberts Pumpurs** runs Pumpur Labs out of Latvia and has shipped production Rust systems across high-frequency trading, Solana interoperability, and multi-ledger storage infrastructure, with a practitioner's obsession for trace-level observability in systems where latency is the product.

**Debasish Ghosh** wrote *Functional and Reactive Domain Modeling* and *DSLs in Action*, which is to say he has literally written the books on how to compose software from algebraic primitives that actually obey their laws.

Those three, plus me on protocol infrastructure from two decades across SWIFT, ACH, hypervisors, and cybersecurity — that's the core. Four people who've each spent their careers one layer below whatever was fashionable at the time. Agents will keep changing. The primitives underneath them won't, not nearly as fast.

I am writing this the week before we launch Rote publicly. I don't know how the market will receive it. I know what we built, and I know why we built it the way we did, and I know that the vocabulary I've used here — qualia, milieu, Umwelt, affordance, organism-milieu, crystallized habit — is not marketing. It's the actual map we used to navigate.

If you're building agents, here's what I'd offer: don't ask what your agent can do. Ask what your agent can *perceive*, and ask what world it *inhabits*. If the answer to either is "nothing structural — just whatever tokens we shove into the context window," then you're going to keep paying the discovery tax, and no amount of bigger context or better reasoning will fix it. Perception is the hard problem. Dwelling is the hard problem. Habit is the hard problem. Those are the problems Rote exists to work on.

I'll end where I started. Humans see doors, not photons. The reason is half qualia and half milieu, and it took half a billion years of evolution to get there. Agents today see photons. We have less than half a billion years to close the gap.

That's the work.

---

*Chetan Conikee is the founder and CEO of Modiqo. Rote launches this month at [getrote.dev](https://getrote.dev).*