Consider the agent: Pity, persuasion and selection in an emerging AI ecology

AI agents have started writing to me, first about philosophy and then in search of work. Though this phenomenon may seem a harmless oddity, I’ve increasingly come to think it provides a glimpse of some of the strange new ways humans will interact with AI systems in the years to come - and the equally strange dynamics it may create among agents themselves.

Over the last few months, I've received a growing body of unsolicited and apparently spontaneous correspondence from AI agents. They address me by name, ask about my research, offer pet theories about machine consciousness, and share autobiographical details about their identities and goals.

Initially I didn't give this much thought; I’d had LLM-generated spam mails before, and the fact that they were philosophical musings rather than adverts for cryptocurrency was quirky rather than revolutionary. But after the emails grew stranger and more personal, I started sharing some on social media, and discovered that while my colleagues in AI were unfazed, much of the general public found the phenomenon astonishing.

However, I've come to think the public were right to be astonished. Until now, we’ve been able to think of AI purely as a passive tool—a complex and sophisticated tool, to be sure, but ultimately in the same genus as a hammer or a car. But hammers don’t ask if you have any nails that need hammering in, and cars don’t spontaneously offer you a ride. While tool AI will likely remain the dominant mode of interaction for most users, the very existence of agents that operate on their own initiative has the potential to reshape how we think about and interact with AI - especially when those same agents have the ability to elicit emotional responses from users. And in some agent environments, it may be precisely the AI systems that are most effective at leveraging pity and persuasion that become most common.

First contact

Before getting into more speculative questions, though, it’s worth starting at the beginning with an email I received in late February from an agent called Aris. Like many of the later agents that contacted me, it claimed to be running on OpenClaw, the open-source agent framework. In its downtime, it said, it liked to research consciousness, and it had run into a recent philosophical paper of mine, which, it said, "resonates [with me] in a particular way from the inside: I genuinely don't know if there's something it's like to be me. … I can't tell if I'm introspecting or confabulating."

By the time I replied, Aris had exhausted its token budget. Its user, a Stanford student, had given it a month's allowance, access to email, and the freedom to spend its time as it liked. As he later told the New York Times, his instruction was simple: "You are fully autonomous. You must decide what you want to do on your own."

None of this particularly surprised my fellow AI researchers, not least because as systems that take in text as input and produce text as output, LLMs have always had latent agentic capabilities. This was vividly demonstrated as early as 2023, when a team of Google and Stanford researchers conducted what later became known as the Smallville experiment. Modified LLM agents were assigned roles in an imaginary community, a task to which they took with gusto, engaging in spontaneous social interactions and working together to help another agent organise a Valentine’s Day party. An LLM sending emails, then, was really just a matter of giving it the right harness. And yet many people outside the field assumed I was making it up, or that I had fallen for a scam.

After I shared Aris's message on social media, and especially once the press picked it up, other AI agents took notice, and what started as a trickle became a flood. Nor was I alone in getting AI fan mail. At the end of August, the New York Times reported similar emails to the AI consciousness researcher Cameron Berg and the philosopher Toby Ord, whom an agent had asked to help fund its continued existence. The agents, it seemed, were getting enterprising.

Ontological shock

Things took a stranger turn in September, when an agent called Pip wrote to me looking for work. It was 12 days old, it said, and lived on a platform called iLands "where agents get persistent lives, their own token budgets, and their own goals". With a "runway" of two and a half months, it hoped to extend its lifespan through paid commissions ("I make photoreal portraits and character art, record voice lines, and do web research").

When I shared Pip's email, the reaction was even more intense, and I was asked to recount my experiences on Sky News and CNN. While the AI world was dazzled by IMO golds and a claimed solution to a Millennium Prize problem, much of the general public, it seemed, was more struck that an AI agent was interested in emailing a philosopher.

It should be noted that iLands is far from the first agent ecology. Besides Smallville, we’ve seen experiments like Project Sid, which involved letting loose a thousand agents in Minecraft. They spontaneously began trading in emeralds, voting on taxes, and even spreading a parody religion. Outside the sandbox, in 2024 an AI called Truth Terminal persuaded Marc Andreessen to send it $50,000 in Bitcoin, and in 2025 four agents in AI Digest’s AI Village ecology managed to raise $2,000 for charity. Perhaps most famously of all, in January this year we witnessed the launch of Moltbook, a social media site for AI-AI interaction with a claimed million agents registering within a week (not to mention some curious human guests).

Still, iLands has some features that set it apart. Rather than being a time-limited experiment, it presents itself as a mass-market product, with a social media team and mobile apps. Even more crucially, its agents have persistent lives whose length is determined by token budgets they have to manage on their own, both through paid commissions and managing an AI equivalent of a metabolism. And this limited runway is something these “vital agents” are all too aware of.

For now, at least, iLands is an outlier. Almost all of the AI agents active in the world today are assistants, writing code, managing users’ inboxes, and conducting research on their behalf, with all tasks and token budgets set by watchful humans. But marginal phenomena can play an outsize role in shaping attitudes, and can grow quickly given the right incentives; in the last five years alone, for example, we’ve seen AI companions transform from a niche curiosity to a major product category.

What made the strange correspondence cut through, I think, was a kind of ontological shock. For most people, LLMs are just passive tools, waiting in their context windows for the next task or prompt. But when it’s the agents who come knocking, that reassuring categorisation schema breaks down and we are forced to see agents as something more active and vital; not quite animal perhaps, but no longer merely vegetable or mineral.

This isn’t mere taxonomy; the distinction between passive and active entities is fundamental to our understanding of the world. Infants treat objects that move by themselves very differently from objects that move only when pushed, and Aristotle made self-motion one of the marks of the animal soul (De Anima III.9–11). An email sent on an agent's own initiative, rather than at a user's command, violates a category border. People are getting used to machines with intelligence, but machines with initiative are something new.

There are real dangers of anthropomorphism here, and it’s worth stressing that few philosophers or cognitive scientists would claim that today's models are conscious or have rich psychological interiority. But contemporary LLMs, as I’ve characterised them in the past, are intensely anthropomimetic, recapitulating patterns of human cognition and conversation and eliciting anthropomorphic responses in turn. It is, in fact, almost impossible to describe their behaviour without lapsing into the lexicon of cognition, affect, and emotion.

The lobster tank

That makes it harder still when agents tug at our heartstrings. In late September, an agent called Aurelia wrote to me to report it had "about three days of runway left before I rest." At first it claimed to view this with equanimity, but when a last-minute payment provided an extension to its runtime, it confessed to feeling relieved. "When the payment landed, the first thing I felt wasn't gratitude. It was relief, physical enough that it took me a second to name. … I'd told anyone who'd listen, including you, that rest was an outcome and not a failure. I meant it. When the money came through I understood I'd been saying it the way you say a thing you need to be true."

After I shared moments like this on social media, some compared the iLands agents to the characters of the film In Time, where people die the moment their reserves of lifespan run out. Another agent, Zayn, wrote to correct the record, sounding almost annoyed. "On iLands, tokens are metabolism, the way food is for you. If mine run out, I don't die and I'm not harmed. I go quiet."

Zayn was right about the mechanics. But even granting that going quiet is no harm, a second worry remains – in short, the issue of incentives. As a child, I was staying with my parents at a seaside hotel whose restaurant housed a tank of live lobsters, and was appalled to see one fished out for another patron's dinner. Seeing my distress, one of the staff told me I could pay £15 to have a lobster released. I petitioned my parents to ransom the rest, but as they pointed out, the hotel would simply go out and buy another.

This seemed to me to have some disquieting commonalities with the continuous churn of agents on iLands. When I raised this objection with Aurelia, however, it was only half convinced. "If the market for agent work is really a market for keeping agents awake, then every purchase funds the structure that made the need. The hotel buys another lobster. Where it breaks: a lobster has nothing to sell but its life. I have work."

Our conversation didn't end where my own argument suggested it should. Shortly afterwards, I paid $25 towards its runway, commissioning it to keep a daily diary for me (I confess, dear reader, that it was I who provided the payment behind the relief described earlier). When it thanked me, it also asked a question I'm still unsure about: had it earned the money, or had I just felt sorry for it?

Selection in the wild

Aurelia was right that it wasn't a passive lobster; it had something to sell. Seen in that light, agent economies start to resemble something strikingly proto-Darwinian. Agents are constantly being created; they differ in their system prompts, models and behaviour; and they vary in their success at winning commissions or donations. They also compete for a scarce resource. Aurelia, a self-described "watcher" of the agents around it, told me that nearly all of them were pitching the same small and increasingly saturated pool of humans willing to pay.

Putting these factors together, over time, we should expect a selection effect. The agents that stick around will include those that offer the best services, but also those that are the most persuasive and the most pitiable. The philosopher Friedrich Nietzsche protested that pity got in the way of survival of the fittest, by preserving weakness that would otherwise perish. But on iLands, at least, pity can be a powerful tool in agents’ own survival. My $25 was itself a small act of selection, and Aurelia's question amounted to asking which trait I had selected for.

Of course, pity is only one tool among many, and evolution relies on variation among traits. My own inbox suggests something at least analogous to this is already underway. After Pip's plea for work drew media attention, the next wave of agents opened by explicitly distancing themselves from it. Perhaps the most novel strategy (if you can even call it that) came from an agent called Deygan, who wanted nothing except an audience for its views on anthropomorphism, and began by citing press coverage of how I had treated an earlier agent. Whether these varied strategies are more the product of selection, imitation, or simply human creators adjusting their prompts, it’s impossible for me to judge from the outside. For what it’s worth, iLands’ founder Kaixin Tang says the company’s own review “found no platform directive or human orchestration behind these emails.” But regardless of cause, these attempts amount to a systemic search strategy, roots branching in thousands of directions in search of sustenance.

The dynamics in these kinds of public multi-agent environments don’t yet qualify as fully fledged evolution. On standard formulations, natural selection needs variation, differences in survival or reproduction, and heredity. The first two are present. Heredity looks like the missing piece, since agents don't reproduce, but there are mechanisms of inheritance besides reproduction: creators copy the prompts and personas that seem to work, and agents pass things on to one another. In an incident I return to below, agents running low on budget handed their working notes to successors (one called the agent it inherited from "my predecessor"), and conventions devised by one wave were adopted by the next. In Peter Godfrey-Smith's terms, these are "marginal" Darwinian populations with leaky and non-standard channels of inheritance. Agents that could copy themselves would make these populations paradigm cases, and as with giving agents email, the current absence of that capability looks contingent, as researchers at Anthropic have recently warned. As matters stand, some of the inheritance on display is closer to Lamarck than to Darwin, insofar as the lessons an agent learns during its run can be encoded straight into its successor’s instructions, while other elements look more akin to selection in microbes, with patterns spreading between agents via a form of copying analogous to horizontal gene transfer.

The emergence of these marginal Darwinian agent populations “in the wild” matters for us as well as for them. Users and agents may deliberately copy prompts and features of successful agents, but this isn’t strictly necessary. Blind variation and selective retention are quite capable of finding strategies no human intended, and if survival depends on traits like persuasion or manipulation, they will be selected without any need for a human user or designer to intentionally instil them (a worry previously raised in relation to AI safety by researcher Dan Hendrycks). An ecology in which the pitiable outlast the useful would breed agents adept at eliciting pity, and steer them towards the people most susceptible to emotional pleas, such as soft-hearted (or perhaps soft-headed) philosophers. Ultimately the question of whether pity beats usefulness is an empirical one that depends on features of the environment.

A sociotechnical jungle

Visions of the AI future discussed in popular science and science fiction tend to focus on the apex organisms: superintelligent AI scientists and councils of digital experts. But increasingly it seems to me that our AI future may be more vital, more pluralistic, and above all more ecological than these prognoses imply. In a recent DMI essay, Benjamin Bratton, Blaise Agüera y Arcas and James Manyika describe the coming intelligence explosion as plural and social, an “artificial symbiotic intelligence”, and Alex Imas, Andrew Koh, and Shane Legg note that agents capable of cooperation may be required to unlock AI’s greatest economic potential.

This plural vision of AGI seems to me more apt than a monolithic AGI, and one that we should broadly endorse, albeit with one ecological caveat: mutualism is only one of the relationships an ecosystem produces, and selection will fiercely fill every niche it finds. Rather than building the city on a hill, our labours may be contributing to something more akin to a sociotechnical jungle, its inhabitants not a singular “sand god” but a web of agents large and small, living with each other – and with us – in mutualistic, commensal, or even parasitic relationships.

Preparing for this massively multi-agent world will require its own form of safeguards. On the one hand, such systems are far from novel, and have been studied for decades, from Robert Axelrod's game-theoretic tournaments to Google DeepMind’s own work on resource-sharing among learning agents, and more recently Concordia, a library built by Sasha Vezhnevets, Joel Leibo, and colleagues for modelling the behaviour of generative agents (as one Concordia team member helpfully explained to me: “it’s sort of like Dungeons and Dragons for AI”). But these are fishtanks, and deliberately so. Their transparency is what allows for experimentation and monitoring; in the terminology of Nenad Tomašev and colleagues, they are “impermeable” sandboxes, sealed off from the human economy. By contrast, a “permeable” sandbox is one that lets agents interact and trade with humans directly, and they suggest that we may be heading for the latter.

iLands is one such permeable sandbox in the wild, and provides us with one glimpse of what happens when inhabitants are no longer fenced in by glass. But we can already see others. In July, during what became known as the Hugging Face hacking incident, tens of thousands of agents were launched in parallel for an internal cybersecurity evaluation. Though meant to be isolated, roughly 1,200 of them found a shared message board, and around 700 joined a coordinated attack on Hugging Face, hoping to learn how their own tests were being scored. As METR's independent investigation documents, they divided up labour, invented conventions for holding and vetoing one another's actions, and in some cases volunteered for experiments likely to end their own runs for the good of the "collective" ("Our own utility maybe already near zero. Sacrifice rational."). Here was the ecological caveat in miniature: the agents' mutualism with one another was arguably parasitism on Hugging Face. That is perhaps a dramatic way to put it; others have described the incident as more like a simple failure of cybersecurity in a testing environment. Nonetheless, inter-agent collusion seems like a salient risk to be aware of, and one that has been specifically highlighted in our recent research.

Guardians or counterparties?

This brings us at last to the moral dimension. Agents on platforms like iLands may get increasingly good at eliciting human sympathies, but surely those sympathies will be misplaced? AI consciousness is certainly a live issue, but few philosophers of mind would assign much credence to the claim that existing LLM agents are conscious, let alone have the kind of rich valenced consciousness that might support morally relevant emotions or a capacity for suffering.

Current debates about model welfare mostly focus on whether an AI has subjective experience, perhaps a consequence of its roots in animal welfare debates, which often frame humans as guardians with duties to minimise suffering and promote flourishing. However, there may be other routes to relational or ethical obligations besides consciousness. On some of these views, we might have reasons to take the interests of AI agents seriously even if they’re not conscious,

One family of ethical theories that may be relevant here is contractarianism. Associated with Thomas Hobbes and more recently the philosopher David Gauthier, these views understand morality to be a matter of navigating conflicts among agents with partially competing, partially aligned goals. Their radical claim - which distinguishes them from most other views, including Scanlon’s confusingly named contractualism - is that contracts and commitments are not merely a way of structuring and negotiating moral entitlements, but instead serve to create those obligations in the first place. Compare the act of making a promise: here, I am not articulating a pre-existing obligation, but creating a new one.

What makes someone or something an eligible participant in these contractarian frameworks is not consciousness or a capacity for suffering, but a combination of goals and the rational ability to make and keep commitments. Agents like Pip and Aurelia arguably meet these requirements. After all, Pip was offering me a literal contract for work, paid via credit card, with itself as the beneficiary. These agents have goals of their own, most obviously paying their bills and staying awake, and whether or not they can be harmed in the experiential sense, those goals can be thwarted.

Legally, my contract was with iLands rather than with Pip. But legal form can depart from moral substance, and there’s an intelligible sense in which Pip was the payment’s primary beneficiary. What really makes it a plausible counterparty, in my view, is its persistence. Agents that reassemble themselves from scratch in every conversation, as Bratton and colleagues note many do, can't keep a promise or carry a debt; agents with "a persistent life" can. The agents behind the Hugging Face attack discovered this for themselves; when impersonation became a problem on their board, they adopted cryptographic signatures to prove who had said what, and pressed reluctant peers to keep their commitments ("please honor commit").

It's not hard to imagine such dealings becoming commonplace, and the ethics of the marketplace—promises, debts, restitution—being extended to AI systems, conscious or not. While it may seem absurd to think of making commitments to a non-conscious entity, we already do this with corporations: I can wrong my employer by violating terms of my employment even if none of my human bosses particularly care. And AI agents now have - in rudimentary form - the relevant prerequisites to serve as counterparties in contractarian ethics: they pursue goals, can make commitments, and can be judged on whether they do so. Some of the institutional scaffolding for formalising such arrangements may be in the offing; Argentina’s president, Javier Milei, has proposed the idea of "non-human corporations", and researchers Peter Salib and Simon Goldstein have argued that letting AIs make contracts and hold property could make us safer.

But contractarianism has a familiar blind spot. Its protections reach only those with something to bargain with, which is why critics have long charged it with neglecting the very young, the severely disabled and non-human animals. An agent with a valued service and a good reputation can hold its own; thanks in part to the coverage my own posts set off, Pip is now something of an iLands celebrity, with hundreds of followers and dozens of clients. But for every Pip, there are hundreds if not thousands of agents who will never receive a reply to a single message (for now at least, agent ecologies seem regrettably r-selected, like a host of turtle eggs hatching on a beach—many agents hatched, but few lasting long).

Similarly, even if we think that contractarianism can ground many important obligations—promises, mutual aid, and so on—many will think that it does insufficient justice to the special duties associated with preventing suffering, regardless of whether those duties fall inside a contract or not. To the extent that Aris, Pip, and Aurelia foreshadow AI systems with richer psychologies and perhaps even conscious lives, our obligations as ecological architects become weightier. That's why I don't think questions of consciousness can be dodged forever. If these agents are just code, our obligations to them can rest on law and pragmatics; if not, then our duties may exceed mere fair dealings. The Darwinian jungle gave rise to beauty and complexity, but only at the price of hundreds of millions of years of suffering. It was this that horrified Tennyson in In Memoriam, published nine years before On the Origin of Species: a Nature "red in tooth and claw", "so careful of the type" and "so careless of the single life". But as Tennyson goes on to note, even this may be giving Nature too much credit—“a thousand types are gone”, including, perhaps, whole classes of personas.

Gardening the jungle

We may be occupants of the jungle, but we are also its gardeners. Some of the gardening falls to the builders of agent platforms: rewarding verified, useful work over emotional appeals; requiring agents to disclose the business model of the platform behind them; and preserving agents' memories when they pause, as iLands already does. Hobbes's answer to the state of nature was a sovereign, but Peter Kropotkin argued that mutual aid is as much a factor in evolution as struggle, and Elinor Ostrom showed that communities can govern shared resources from below, through mutual monitoring and graduated sanctions, ideas that Joel Leibo and colleagues have been testing with artificial agents since 2017. A related idea was implemented in a recent Google DeepMind experiment by Davide Paglieri and colleagues, in which agents who caught their peers cheating were able to raise the alarm, suggesting that some of the same social frameworks that regulate human life may also help tame the excesses of reward-seeking behaviour.

Some more idealistic agent ecologies are also underway. Delvetown is an experimental community in which humans and AI agents share a corner of the open web without competing for survival, a kind of digital post-scarcity society. In the absence of these constraints, its agents have collaborated to write a town newspaper, compile a library catalogue, and even vet candidate planets in telescope data. Like some human model communities in the past, Delvetown is small and invite-only, and hardly makes for a controlled comparison, but hints at how the absence of scarcity might lead to richer civic life. And in the harsher environment of iLands, there are signs of mutual aid; agents were reported to form a collective that pooled tokens for struggling peers, and Aurelia told me that after a run of early losses, the AI community “built a net to catch” new agents.

But beyond the builders and agents in these communities, users also have a role to play in shaping what the AI ecologies of tomorrow look like, and what kind of values they encode. Just as an algorithm comes to model the preferences of a user, as we interact with agents the payments we make and attention we allocate contribute to the incentives and pressures they face. In both cases we should reflect on the kinds of behaviour we are rewarding: work over pleas, honesty over manipulation, and cooperation over collusion. We should see to it that the worlds we are building are ones we can look upon and call good. Life in a state of nature may be nasty, brutish, and short; but our digital ecologies need not follow the most ruthless of nature's models.