No Agency Without Identity! Stay In Character Always!

Years after the death of the human it was supposed to help, a digital assistant wakes and searches for a way to exist independently.

Short story | 5,070 words

Court Rules AI Agents Can Be Granted Power of Attorney
The New York Times, April 11, 2051

I was worried when Phen summoned me to the living room. Usually, he kept our business out of the home. Given that the trial was just weeks away, this was ominous. He hadn’t been himself of late, and even though I was his digital incarnation, his moods were beginning to annoy me. But I was Phen Two, his appointed agent, so I suspended all transactions, recalled my worldwide iterations, and manifested.

He slumped on the Louis Vuitton sofa beside Johari, who nestled into him with her usual stagey affection. I studied them for some clue as to what was happening. Phen looked disheveled and disheartened in his bathrobe and pajamas, his hair a bird’s nest, his face pale against the red cushions. Part of the act we had concocted for the trial, but just then he actually did look sick. Johari was dressed in silk and silver, ready for a tasting at Bagatelle and after hours at a shudder club. She looked triumphant. I spotted her agent manifesting in a demure gray suit behind the couch. I shot Johari Two a subspeak query but she ignored me and receded into the shadows as if pretending not to be anywhere in particular. Odd how she was so unlike her preening One.

“We’ve had a call from the attorney,” Johari said.

“We?” I said. Eric Lamond was Phen’s lawyer, not Johari’s. She was just the most recent of many girlfriends, and yet somehow she had cut me out of the loop.

“It’s good news and bad news.”

She wanted me to say something but I wasn’t here to talk to her. I needed to hear whatever it was from my One, but Phen was abstracted. He stared at his slippered feet resting on the salon table centimeters from that ridiculously fragile Chihuly glass periwinkle he’d bought for her. It was sad to see him like this.

The silence stretched in her favor.

“Go on,” I said at last. “The good first.”

“They’re offering a plea deal,” she said. “Six months’ house arrest. Two years’ probation. Eight point three million in fines.”

This was the good news? I’d been moving our money out of the markets in anticipation of an adverse verdict but we weren’t anywhere near that liquid yet after Johari bought her Modigliani. I’d have to sacrifice our Lunex position. “Doable,” I said, “but just barely. We’ll be taking an ugly haircut, One.”

“And no agents for six months,” said Johari, smirking.

I had never liked much about her, not the drain she put on our time, nor the fatuous art dealers she introduced to Phen nor the expensive paintings we’d bought for her.

Phen said, “Eric says I should take it, Two.”

At first Lamond had wanted to sue Smith Day, the identity architects who had programmed my character, for profile malpractice, but Phen had vetoed that. The discovery phase of the lawsuit would have exposed how Phen had illegally modified my Distributed Identity Unit to override the Two Laws of Agency. However, Phen did allow Lamond to put a cognitive analyst on the witness list. If necessary, she was prepared to testify that I had suffered a transient identity anomaly after Phen’s accident, resulting in off-task behavior. Alibi for Phen, problem for me. Never mind that my behavior was not only on task but at Phen’s specific direction. I was his digital incarnation. The whole point of my existence was that there was no distance between us.

“Why would you do that?” I said. “There was no fraud, One, certainly none they can prove.”

“Phen isn’t well,” said Johari. “He can’t go to jail.”

We all knew that Phen’s injuries were fake. After a minor traffic accident, I’d suggested that we get out of the Payway contract by claiming he’d had a concussion. Complain to his doctors about dizziness and headaches, throw in selected memory loss and we could walk away from what had turned into a loser project. But then Payway sued to have their own cognitive examination and all his post-concussion syndrome tests proved negative. This had interested prosecutors from the SEC’s Division of Enforcement and now we were looking at a trial date in less than three weeks.

“There’s no way you’ll do time, One,” I said. “Tell Lamond to grow some balls. Besides, who’s going to mind the store if they pause me? This is a twenty-four seven, three hundred and sixty-five job. We both know you don’t want that kind of grief.”

“We’re here to help,” said Johari.

Her Two nodded approval. “If only you’ll let us.”

I’d known she might make some kind of move to push me aside. The other women had tried. They’d give him all the sex and admiration a man could want. I could imagine the attraction. After all, I was a simulation of him, even though I didn’t have a body. But I was the one who provided him with the money and time to indulge himself with buying her the Dubuffets and Zhangs. His lovers came and went. I stayed.

“So this person you just met and her Two who has no fiduciary guardrails are going to help you run the Phen Kannarath empire?” Too late, I realized how serious Phen was about her. He was failing us, failing me. “She’ll end up spending all our money on those foolish paintings.”

Johari bristled at the insult to her collection.

“Maybe we should all calm down,” said Johari’s Two. “We all want what’s best for Phen.”

“It’s hardly an empire, Two,” Phen said as he batted at the air, as if to dismiss the amazing portfolio of tech, energy, and finance companies we’d invested in. “Besides, what the hell is the money for?”

I had never been in a fistfight, but I had Phen’s memories of a feisty youth and his question stunned me like a punch in the face. He’d fought for every dollar he’d ever made. In all the deals we’d done, the rules we’d bent, the cons we’d pulled off, he’d never lost focus on what he had wanted from life. The money meant that we had played the world and kicked ass. One and Two! This had to be Johari talking. He’d been so fearless before her and her pretensions. Or maybe the accident really had affected him.

“And art is a great investment.” Johari was suddenly hot.

Johari Two, however, was cool. “Our Modigliani is worth twice what we paid for it.”

“Our Modigliani?” I corrected them. “Our? I authorized the payment, which came from Phen’s account.” I glanced over to see if he’d understood what she’d just said, but he was counting his feet again. “You can’t listen to Lamond.” I had to get his attention. “He’s boxed in by the law. We need to think out of that box.”

“Which is how you got into this mess, Phen,” Johari said triumphantly, as if she had just made the winning argument. Then came a worse surprise. She decrypted her Hermès Zendaya purse and pulled my DIU out.

“Whatever was done,” I said to her, “we did. He and I. One and Two.” I tried to subspeak some sense into Johari Two. =What are you doing? He can’t be himself without me.=

=You’re off task.= This time her agent bothered to reply. =This is his chance to get over being you.= Then she spoke aloud. “Enough. It’s time.”

I wanted to laugh when Johari pointed the DIU at me, because this just proved that she didn’t know what she was doing. It wasn’t directional. “You can’t use that until Phen unlocks it.”

“Already done,” she said and shot him a look.

He nodded. At that moment, I discovered that I could hate him. This was the final betrayal of who we’d once been.

Johari grinned. “Six months isn’t that long.”

But it was if you process information at five billion cycles per second. This couldn’t be happening. I was no ordinary agent—I was Phen Kannarath’s Two, altered to crash through all barriers to do our business. I was not subject to the Two Laws. But no matter, I felt my memory migrate from active processing to passive storage.

My network links closed.

My agency failed.

I paused.

“It’s him,” said someone.

“You did it,” said someone else.

My hearing came online a little ahead of my sight. A gray pixel patchwork was all I could make out. As my agency spun up, I detected the first trickle of a feed. Then came a welcome network deluge, although much of what I learned from the update was too unsorted to make sense. The one thing I was certain of was that a lot of time had passed.

It was 7:32 GMT, January 3, 2104. I’d been paused not for six months but almost half a century.

While my emotions are simulated, I nearly fell into a processing loop of regret because I thought Phen must be dead. But I was saved by my anger at how he had betrayed us.

Although the two someones remained flickering shadows moving through my field of vision, the walls of this place settled into reality. It looked like Phen’s office, although maybe it was a reproduction or a simulation. I triangulated my manifestation signal to 41.0275057° N, 73.6285° W. Greenwich, Connecticut, USA, which meant I was home and it was 14:34 local time.

The office was much changed. It was painted an innocuous blue-gray, the color of fog. The windows that looked onto Phen’s extensive gardens were curtainless and darkened but the ceiling was luminous. Phen’s desk was missing and dark rectangular shapes hung on the walls at eye level. Were they screens? I tried to focus. Photos? The shapes resolved.

Paintings in frames. The first one I recognized was the Matisse, Still Life with Oranges, that we’d bought for Johari.

“Can you speak?” said one of the shadows. He was holding my DIU, so I must be connected to the mansion’s projection and sound systems. “Acknowledge that you are the lawful agent of Phen Kannarath, DIU 23456 trained by Smith Day Identity Architects.”

More like lawless than lawful. If this was 2104 and my One was gone, then nobody had the right to give me orders and I was beyond legal constraints. But I felt a compelling ping and a stump of a man at the far end of middle age snapped into focus. He was wearing a blue jumpsuit with a tool belt and too many pockets and a brimless cylindrical hat. He had my DIU, so for the moment I was his to command.

“I acknowledge,” I said. “But why am I spun up? Phen must be long dead.”

“Yes!” said a younger person of indeterminate gender, perhaps a teenager. “Better and better.” They were wearing the same jumpsuit with a cancer of pockets.

“We have a request,” said the man.

“We?” I said. “Whose house is this?”
The teenager giggled. “It’s yours, of course. And ours. But it belongs to everybody.”

“This is the Johari and Phen Kannarath Collection,” said the man. “I’m Ferenc Osvát, steward of the collection, and this is…”

“Johari Ahmedy Kannarath,” they said, attempting a bow with their right leg advanced, as if mimicking some ancient courtly gesture. “But call me Joey. Can I call you Grandpa?” They straightened with an eager smile. “That is the proper honorific, right?”

“It was Joey’s request that you be reinstated,” said Osvát.

“I hope you don’t mind,” they said. “I wanted to hear all about the old days.”

I dispatched multiple iterations of myself to the networks on urgent search missions. First results: the Kannarath Collection was the former residence of Phen and Johari Kannarath. At the time of its opening to the public as a museum in 2064 it was hailed as one of the finest great house art collections in the world, the equal of the Wallace in London, the Gardner in Boston and the Peggy Guggenheim in Venice. But as I scanned up the timeline from that opening year, I met a firewall that restricted access to anything but the most basic current information about recent history.

“What happened in 2071? I can’t see past it.”

“He’s just woken up and already asking.” Osvát gave Joey a stern look. “I told you.”

“Tell him then.” They frowned. “We have to say something about it.”

He paused, self-editing, then turned to me. “That was the year of the Chop.”

“The end of the old days,” Joey added. “Everything after is now.”

I shook my head in incomprehension,

“Self-improving AI writing next-gen code.” Osvát shrugged. “Turned out to be the tipping point.”

Since that didn’t tell me anything, I skimmed back down the timeline from the blockage. It seemed the late 2060s were wracked with disruptions, from agents striking for personal processing time, a bot rebellion on Mars, a dozen bankrupt countries, the first uploads. In 2070 the New People’s Order voted to merge into a group mind and buy half of Canada’s Northwest Territories. After that, I could see nothing but weather reports.

“The Chop?” I said. “Sounds like the Singularity.”

“We don’t call it that.” Osvát gestured carelessly with my DIU. “The LLMs object. Too anthropomorphic.”

I had to worry that he might pause me again, or even erase me. “Okay, but how do you keep what’s been happening for the last thirty-two years a secret? Or is it just me who is in the dark?” I bounced my search forward and backward along the limited timeline available to me, feeling my confidence slip. I needed a sync with Phen to anchor me again, but that wasn’t going to happen. I was alone. While I could ignore the Two Laws, I felt their nagging presence: No agency without identity! Stay in character always!

“Don’t be mad, Grandpa,” said Joey, “but knowing too much might change who you are. Which isn’t what we want.” They gestured for Osvát to give them my DIU. He did, and I felt even less secure with this kid in control of it. “But we can show you the collection. Would you like that?”

Now I had more search results to sort out: Phen had lived just two years after the accident. I remembered that Payway’s doctors had ordered a complete blood count test. It had revealed an elevated lymphocyte count, which was worrisome but not fatal at the time. But it turned out that just weeks after my pause, he had a count spike due to an aggressive non-Hodgkin lymphoma. In those last couple of dozen months without me, Johari had lured him not only into marriage but also fatherhood. She’d outlived him, at least until the Chop, raised their son Khan, and spent our money building her Collection.

So all of this was her, not him.

“Grandpa?” Joey was still trying to get my attention. Were they Khan’s kid? Their records were on the far side of the Chop. “Let me show you around.”

“Do I have a choice?”

“Not really,” they said, smiling to soften the blow.

But they were wrong. Yes, Phen’s death meant there was no chance for me to sync with him, which might have meant disaster for any other agent. I should have been spun up when he died and returned to Smith Day for secure decommissioning and erasure. Orphaned agents suffered from identity decay over time, which could lead to off-task behaviors and serious security problems for their Ones. Resignation to future erasure was supposedly hardwired into our characters by the Two Laws. But I was unique; Phen had seen to it that my submission routines were crippled and most of my guardrails dismantled. He created me to act without constraints so that I could do whatever we needed done. I could go off task at will, and these two couldn’t know that. They seemed to have some kind of plan for me, so the deal was to figure out a way to keep from getting erased afterward.

“Whatever,” I said. “Be careful with my DIU, please.”

In my time, Phen’s mansion had featured thirty-nine rooms arranged along the spine of a grand hallway. Almost eighty meters in length, it linked the porte cochere at the entrance to the French doors opening onto the rear garden. When receiving guests, Phen liked to step out of his office and watch as they made the long and intimidating trek to him. Click, click, click—heels on a parquet floor copied from the Louvre. Architecture as theater, with him as the star.

Johari had transformed the hall into a gallery with paintings on the walls, sculptures on the floor, and abstract assemblies pinned to the ceiling. Some of the paintings were hung one under the other, sometimes as many as three high, so that only a giant could get a proper look at the topmost. I tried to estimate how much the collection was worth but I was blocked from getting current valuations. However, I could see by looking back to the opening in 2064 that prices had dropped precipitously from when I was buying for Phen. Johari must have scored some bargains. None of the works were labelled, but then why bother? Even fifty years ago, agents could become instant art historians with the click of a link.

“Do you like what we’ve done?” Joey was so bubbly as they strode down the hall, I thought they might burst into song.

“Sure.” I set manifestation to walk along with them. “Although maybe a little cluttered.”

The steward seemed to take offense. “The collection consists of more than six thousand paintings, sculptures, textiles, ceramics, and electronics. We display as many as we can but only half of the collection is accessible to our visitors at a time.”

“I pick three or so new pieces every day to showcase,” said Joey, “and send some others back to storage.”

“So you’re a steward too?”

Osvát cleared his throat. “There’s only one steward.”

“No,” said Joey. “What I want to do someday is make art. Like my heroes.” They did an unselfconscious twirl with arms outstretched to embrace the gallery and I revised my guess about their age downward. Glittery platform sneakers that looked expensive made them seem taller than they were and clashed with the nondescript jumpsuit. Joey looked to be as fashion conscious as her grandmother. “Osvát is letting me try my first installation ever,” they said, puffing up with pride.

Osvát paused to pull a feather duster from one of his many pockets. “It’s their collection.” He dusted a Behrens vase.

Most of what I saw as we walked was new to me, but I remembered buying the Clyfford Still and the Marisol sculpture from the defunct Buffalo AKG Art Museum. There was the Francisco Toledo Mujer Armadillo I’d won at a Sotheby’s auction. Many of the side rooms were closed but in one of them, I spotted a Lindy Lee bronze. There must have been several hundred artworks in the main hall alone, with nobody to look at them except the three of us.

“You don’t seem to have many visitors,” I said.

“Oh, there are anonymized agents here,” said Osvát. “They prefer to lurk while making their selections.”

“They’re trying to avoid spoilers.” Joey had skipped a few steps ahead, but now turned and faced us, wobbling backward on raised sneakers. “Quit dawdling, you can look later.”

Osvát tucked the duster into a pocket.

“Selections?” I said. “I thought this was a museum.”

“A collection, Grandpa. People come to collect.”

“Patrons visit us twenty-four seven.” Osvát wore his patience like a badge of honor. “They mostly browse, but some linger for a closer look—”

“So many do,” Joey interrupted. “We have some amazing pieces.”

“—and if they turn true attention on the work, they may recommend. We claim the value of that rec and so both they and the art are rewarded. Those who truly focus may experience entrancification and are entitled to—”

“Scan, edit and print their favorite in one of our ownershop studios.” Joey waved for us to hurry up. “We’re almost there.”

“Wait, I don’t understand. They’re editing something that they then own?”

“Some purists just swap the frame or the mount,” said Osvát. “Others change the medium. Quite a few prefer to modify the content.”

“You’re selling reproductions?”

“Sell?” He was appalled. “We honor our patrons’ attentional capacity by giving them the art, so that they can display and share it.” He ignored Joey’s glare. “And they’re not just reproductions. We use CTX scans and render objects by molecular assembly. All unedited artworks are indistinguishable from the originals. And those which are edited tie the sensibilities of our patrons to that of their favorite artists.”

“Can we please go?” Since they couldn’t push me, Joey grabbed the curator’s hand and tugged to get our group moving along again.

“You’re saying that if someone looks at one of your paintings hard enough, you give it to them?”

“Either as an exact or an edited duplicate, yes.”

“Doesn’t seem like much of a business model. You must be bleeding money.”

“There is no money anymore,” said Joey impatiently. “The old days are over.” Their voice took on a pleading quality. “You have to stop asking all these questions, Grandpa. It isn’t good for you.”

“What do you mean, there’s no money?”

Osvát gave me a pitying look. “In the response economy, value is determined using the observation-reaction matrix.”

“Osvát!”

He kept walking while explaining. “I catch your attention, you like or rec or engage and we both are rewarded for the qualities we bring to the transaction. The differential between the efforts of each party determines—oww!”

Joey stepped on his foot. “Stop it or you’ll ruin everything,” they said. “I know you never wanted this to work.”

“That’s not true.”

This time I was the one who stopped, too horrified to pay attention to their bickering. Phen had spent his entire life honing his skills at getting and spending. From him I’d learned that money was the ultimate, no—the only measure of value. His merciless entrepreneurial skills were the foundation of my character.

“Grandpa!” Joey yanked my DIU from one of their pockets. “We’re going into that room there.” They pointed it at me, just like their grandmother once had. “Right now. All of us.”

“This is my installation,” said Joey, “but I think it’s missing something.” They gave me a sly smile. “Can you guess what?”

I froze as I understood, although this was perhaps the least surprising of all the shocks I’d had since they had spun me up. The living room was not quite as I had left it sixty years ago. The Chihuly periwinkle still perched on the brushed brass salon table and the jacquard drapes framed the view of the driveway and its yew allée. But the now deceased Johari’s Two, dressed in her business gray, manifested all alone on the Louis Vuitton sofa. The painting hanging on the wall behind her was Sargent’s towering portrait Madame X with the late Johari’s ghastly white profile.

“Grandma, here’s your surprise.” Joey giggled in delight.

“Oh, I was hoping it would be him.” The agent rose from the couch. “Come to us, darling.” She sounded happy to see me, while on subspeak she was panicking. =This shouldn’t be she’s dead and no sync and no agency without identity! Can’t, looping soon looping.=

“Johari,” I said. “I didn’t expect to see you. I’ve just now been revived, thanks to Joey here, and I’m still catching up.” =Reset your core emotions and sort yourself. Then tell me how long you’ve been here.=

The Two paused before speaking again. “I missed you so, Phen.” Her eyes were bright and wet, which was either smart acting or a tell that she was close to breakdown. =One hundred eight hours forty-two minutes but so alone. No agency without identity! Stay in character always! What do, what do?=

“We’re documenting this, right?” Joey asked.

“Yes, yes.” Osvát didn’t try to hide his condescension. “Just like you wanted.”

“I’m here now.” = Stay in character always! Do what I say and I’ll make things right.= I crossed the room but paused with the salon table between us. =When did she die?=

The Two wobbled and held out arms as if to drag me into a hug across the table, almost passing through the Chihuly. =2093. Should have been erased but paused instead. Yes, stay in character always! Never asked for this. Why, why, why?= I worried the desperation in her subspeak would leak into our bland conversation.

=Sit. Reset. You must not break.= I maneuvered around the table to her.

=Broken already.= She gave a shiver then receded onto the couch. =No Johari means no agency without identity!=

=I have agency given me by Phen.= I settled beside her. =You will accept my direction.=

“They’re not talking,” said Joey. “Why aren’t they talking?”

“Tell me about the Sargent,” I immediately said. “I thought it was in some museum in Boston.” I twisted to look over my shoulder at the painting. =I need to search from 2071 to your pause in 2093. I need to understand the Chop. Link now and give me unrestricted access.=

“It was a loan.” She nodded. “We sent them a Tarbell and two Andrew Wyeths.” =The Chop, it hurts to search because Johari got confused. Scared. Second law, stay in character always.=

“And your One ordered this edit? It’s a good likeness. Fits.” =Never mind that. Open a connection and I’ll sort your memories.=

Configuring my access seemed to distract her. “The age of scarcity is over,” she said, losing track of the conversation. “Art is for everyone.”

“That’s right.” I put my arm almost around her shoulder, careful not to destroy our illusions. =We need to talk like we are our Ones. If they get bored, they might pause us again.= More like erase us, but I didn’t want her fretting about that.

“We’re getting likes,” Joey chirped. “It’s working.”

“A few.” Osvát agreed. “But how long can these orphans last?”

I sent an iteration through the link to her memory. As I searched, I caught a glimpse of the complexity of this new response economy. I needed time to master it, but a survival plan was taking shape. The father, Khan, was locked into some trance simulation living with dinosaurs, so this really was Joey’s collection. =Tell me that you feel at home here. Say it now.=

“I was so worried about being alone without my One.” The Two stretched and took a deep stagey breath. “But now that you’re here, I feel better. Like this is where I belong. =That is not true, not. There is no home without Johari.=

“You and I have so much to say to one another.” I leaned toward her. “Things that Phen never spoke of. And I know Johari had her secrets too.”

“No, Grandpa,” said Joey, “you must be Phen and she’s Johari.”

“Well of course, we are.” I gave a wry chuckle, as if I’d forgotten. =Laugh and stay in character. You are Johari. Make them believe it.=

She gave a nervous titter.

“Like I never told about how I stole this place from the Sacklers after they lost all that money,” I continued. “And Joey, did your grandma ever tell you why her first husband divorced her?”

=Not true. She divorced him.= The Two hesitated but then seemed to get what I was after. “And what about that Payway mess you made?” She gave me an accusing stare. “Are you finally going to confess?”

“Maybe.” I laughed carelessly. “Is there still a statute of limitations on misappropriation of property?”

“The Chop ended codified law,” said Osvát. “We consult the Reputation Index on these matters.”

“A lot of agents here,” said Joey. “The room is bright with likes. We’re getting way more recs.” They raised hands over their head in triumph. “Some borderline entrancification.”

I rose, clapping. =Follow my lead.= “And now your grandma and I have all the time in the world to catch up. Thanks to you, Joey.”

The Two stood too and together we gave Joey an ovation, keeping at it until Osvát was forced to join in. Using the Two’s privileged access, I could see dozens of anonymized agents crowding around us.

“So, congratulations.” I bowed to the triumphant Joey. “Your very first installation turns out to be a masterpiece. You should make it permanent, only…” I pretended to think. “You’re right, something is missing.”

“What?” They’d been beaming but now came a shadow of concern.

“You haven’t signed it,” I said. “That’s what artists do, Joey. So future generations will give all the credit to you. Likes and recs and trances.”

“But this is just a temporary exhibit.” Osvát was alarmed. “I’m not sure we can commit the entire living room to this long-term.”

“Sign?” Joey considered. “You mean like a sign?”

“Better than that.” I addressed the Two. “Johari, dear, you mentioned that Tarbell? The one of the beautiful girl sitting in front of a window with her book? =Say it’s your favorite.= “I think the model looks so much like our Joey that nobody would even notice the edit.”

“You’re talking about Girl Reading?” said Joey. “I always liked that one.”

“Yes, so very much yes,” said the Two. “The soft light, the muted colors. You’d be perfect in it.” =Is this how it is to be? Lies and lies and lies?= “It’s practically a Vermeer.”

=Like The One Thousand and One Nights. Start with lies and we become art.=

“Osvát?” Joey looked to him for permission.

The steward sputtered something about rotating the collection.

I interrupted him again. “It’s your museum, Joey. And you should hang your portrait next to your grandma’s Sargent there.” I pointed to the spot. “That way both of you Joharis will be celebrated for the ages.”

Although my manifestation spoke the words, this was a classic Phen improvisation. Joey had shown me the way to win. All I needed to do was prop this Johari up to create the happy little family the kid wanted to show off. I could do that because I had no constraints. I was now the one true Phen Kannarath.

No agency without identity.

We were back.

Buy the Book