Teleoperated Humans

Jeff Kaufman
Posts
RSS
◂◂RSS
Contact

Teleoperated Humans
September 13th, 2026

airisk, tech

When I look at why I expect the world to change a lot in the next few
years, and why other people expect slower changes, I think a big
component is disagreement on the extent to which AI will affect
non-computer work. Sure, programming has sped up massively with Claude Code etc, and
models like
Astra seem posed to make similar changes to work with spreadsheets
and other common business tools, but what about work that doesn't include
computers at all?

The classic picture of AIs doing things in the world is robots, but I
think a more realistic picture of the near future is computers
telling people what to do. Leaning into the way the world has become
very scifi, we could call this "teleoperating" people. Many things
that are hard for robots are very easy for people, there are strong
economic reasons that push towards teleoperation, and this bypasses
many legal and social limitations on what AI can do. We should expect
this to lead to large and rapid changes in the physical world.

One of the most widespread examples today is driving. I put my
destination into the GPS, and it tells me what to do. I handle the
low-level physical motions and responding to the local circumstances;
the GPS has a broader view of the world and handles the strategy.

When I think about why this happened much earlier than the huge amount
of "teleoperation" I expect to see soon, a few
factors. Driving is a major human activity, so it was worth making
navigation software at a time when AI wasn't very good yet, even
though this meant a ton of human hours going into building the system.
It was also a place where the strategic component was a very strong
fit for automation. You can memorize
the map with enough work, but even then you won't have real-time
street-by-street traffic information. AI solved this problem so well
we don't even
call it "AI" anymore. On the other hand, driving is a realtime
control problem in an unconstrained environment where people die if
you screw up and you can't even always safely stop. This makes it
hard to automate, but also would make it impractical for an AI to
guide non-drivers through the process. The only reason Uber etc have
been able to commodify driving as they have is that so many people
already know how to drive.

Thinking about where else we might see this, most AI use today looks a
lot like management.
You figure out what you want it to do, and describe in detail. It
asks you some questions up front and others while it works. After
some churning you get some a work product to assess. Maybe there's
more back-and-forth, or maybe it's good as is. You set strategy and
give context; the AI handles the implementation. Today's AI is
normally only applied to the implementation to the extent that the
task can happen fully within the computer. In cases when the AI can't
physically, legally, or intellectually do something, the most
efficient path to completing the task will often be the for the AI to
handle strategy while delegating to a human to fill these gaps.

To illustrate what this delegation pattern can look like, let's look at how I
recently got my Whistle Synth
app into the Mac App Store.

At a high level, I set the strategy: "Can you walk me through the
process of getting this into the Mac App Store?" But everything after
that was either handled by the AI or delegated back to me. It handled
included figuring out what tasks needed to be done, modifying the
implementation to be compatible with the App Store restrictions,
building the app, and giving me instructions. And then it delegated
to me to record a demo video involving whistling (physical), register
as a Mac Developer (legal), and clean up its App Store description
(intellectual).

This was mostly pure instruction-following on my part: I was being
teleoperated. Here's one example:

Claude Code:

Open the Profiles list
https://developer.apple.com/account/resources/profiles/list — sign in
with the account for [team].

Click the blue + next to "Profiles". You land on "Register a New
Provisioning Profile", a page of radio buttons grouped into
Development and Distribution sections.

Under Distribution, select "Mac App Store Connect". Not
"Developer ID" — that's for distributing outside the store. Click
Continue.

...

This was relatively mindless work for me. Just like being navigated
through a city I don't expect to return to, I didn't bother trying to
learn how this worked. I was loosely paying attention to make sure I
wasn't doing anything dumb, but for future more capable systems I
expect people to stop attending even that little.

Once it finished walking me through submission I had to wait a few
days for review. It was accepted in the first round with no reviewer
comments. This is a pretty big deal: App Store rules are notoriously
complex, the reviewers very picky, and as a first-time amateur Mac
developer there's no way I would have gotten this all right on the
first attempt pre-AI.

Even though this was an almost entirely within-computers case, the
important thing here is the pattern: by following AI instructions I
did something that would have taken me a ton of work to learn how to
do alone.

Note that in this case I was both doing the high level strategy ("put
this in the app store") and filling in gaps for the AI (clicking a
blue plus in App Store Connect). As "teleoperation" becomes more
common I expect some of this, as people automate away parts of their
jobs. Other times I expect it will look like, for example, a highly
AI-pilled startup founder directing AIs that direct employees. A lot
like gig workers "below the API"
today. I expect early iterations of these jobs to be frustrating,
with the AI not delegating well. Then, as AIs get sufficiently good
at directing and anticipating, they'll be pretty mindless, for better
or worse, as you stop
needing to think for yourself at all.

What sort of jobs might switch to being teleoperation? The top
candidates are any where the physical motions are relatively
straightforward, timing is not critical, and people today are paid a
lot for their knowledge and judgement. If you had an expert looking
over your shoulder and telling you what to do, I expect most of you
could do most of the work of an electrician. In fact, that's the bulk
of how electricians learn their trade: through apprenticeship. Same
goes for mechanics, healthcare technicians, inspectors, etc: they
combine physical and intellectual components, where it's the knowledge
that keeps a random person off the street from being able to do the
job. People wearing glasses with built-in cameras, connected to
today's strongest AIs could already do a lot with a bit of
scaffolding.

To have a large impact, teleoperated workers wouldn't need to be able
to do 100% of an existing job category. As long as the parts that can
and can't be done this way can be easily separated, 90% could be done
by teleoperated novices, while some of the former professionals spend
their time on the remaining 10%. When I think about how these other
jobs are likely to go, I expect we start with ones without regulatory
barriers: HVAC techs (typically unlicensed) before electricians
(licensed) before surgeons (licensed + heavily regulated + realtime +
high stakes). [1]

So, teleoperation is probably very economically productive. Is it a
good thing? I think mostly no, for several reasons. The big one is
that I expect it to speed up the rate at which AI advances turn into
additional AI advances. This shortens the time our society has to
figure out what to do about these massive changes, and increases the
risk that immature technology is rolled out widely. Rushed deployment
is more likely to lead to disaster, and there are many ways this could
go extremely wrong. And by "extremely wrong" I mean "AI kills
everyone wrong". Creating minds smarter than ourselves is the most
consequential thing humanity has ever done or will ever do, and we
have to get it right.

Which is why I'm heartened to see a lot of support, including from the
CEOs of Anthropic
and OpenAI,
for managing the pace at which these systems become increasingly
capable. But even if we held constant at the capabilities of models
publicly available today (let alone trained but not yet released) I
think widespread teleoperation is still very likely. I expect this to
be a massive disruption, one very difficult to integrate into our
existing societal system.

The first issue is just that I expect these to be unpleasant jobs with
low negotiating power. Since there are many tasks that almost anyone
could do if expertly advised, and the employer can easily filter out
the people who can't or won't, there's very little to keep wages or
working conditions up. Then add in competition from laid-off knowledge
workers, and I expect
unprecedented unemployment.

So even if we can avoid the large risks of losing control of the
future, falling into AI-enabled authoritarianism, facilitating
bioattacks, etc, how we handle a world in which most people can't find
work that pays them enough to live on will be an serious challenge. I
expect this will require very large scale redistribution. [2] I'm not
sure this happens by default, but I think it's achievable with
significant effort. And as a very small fraction of spending in a
vastly larger economy it would be a much easier sell.

[1] For a future post:

$ echo "[redacted]" | sha512sum
7820a2ecae1fcab8d7a29fe4f98f56b96c403cdfb9a6833fad0198070e118233490a078c9230de0c6ed5cf1c6cf557fef493aeb33118e75b401deabbf3a1aae4 -

[2] Looking at what there is already, the US does less than most rich
countries, but even here we have medicaid, EITC, CTC, WIC, SNAP, SSI,
TANF, Section 8, LIHEAP. We spend maybe 3-5% of GDP on means-tested
programs. Then ~7-10% of GDP goes to things like universal public
education and medicare which aren't directed specifically at the poor
but are still effectively redistributive. Internationally there's
been some of this, but much less; until recently the US was spending
maybe 0.04% of GDP on the kind of foreign aid (ex: PEPFAR) that is
really about helping the world's poorest, and then the private sector
(Gates etc) adding maybe 0.1% of GDP.

Referenced in: Initial DIY Cleanroom Experimentation

←

Why I Stay Off Twitter

Replace Net Metering With Batteries

→

Comment via: facebook, lesswrong, mastodon, bluesky, X, substack

Recent posts on blogs I like:

Mindful posting

It is important, for the preservation of American democracy and maybe the world, that the Democrats take the House and Senate in November.

via Thing of Things

September 18, 2026

There's no point at which turning your brain off will work

In early 2025, I started seeing people turn off their brain as they use LLMs1. They would have an LLM take an action (summarize text, write some code, etc.), and just assume that it worked2. This generally didn't work in early 2025 and the result was …

via Posts on

September 18, 2026

My Weirdy Creatures

Over the last few days, I've been working on some little creatres.
But they're very weird. I call them "weirdy creatures". I've made
quite a lot of them, I think maybe 5-6, not sure.

Weirdy Predator

This is a weirdy predator. They'…

via Anna Wise's Blog Posts

September 13, 2026

more
   
(via openring)

More Posts

Bringing Little Kids to Contra Dances
You Should Be Logging Shell History
Research Deprioritizing External Communication
The Privilege of Earning To Give
Why I Work on Ads

open

Jeff Kaufman
Posts
RSS
◂◂RSS
Contact

The concept of teleoperating humans, where computers direct human actions rather than robots performing tasks directly, is presented as a significant factor in how the world is expected to change in the near future, particularly concerning work that does not involve computation. The author posits that this approach is motivated by strong economic imperatives and allows for the bypassing of many existing legal and social limitations placed on what artificial intelligence can perform.

One prominent example of this dynamic is driving. In this context, the GPS provides broad strategic guidance, while the human remains responsible for executing the low-level physical motions and reacting to immediate local circumstances. While driving is a real-time control problem in a dangerous environment that makes full AI automation difficult, the existence of highly commodified services demonstrates that the pre-existing human skill base is crucial for widespread adoption.

The author observes a specific pattern in current AI application: instructing the AI to handle the implementation while the human must provide strategy and fill necessary gaps. This delegation pattern is illustrated by an example where the author used an AI to navigate the complex process of submitting an application to the Mac App Store. Here, the human established the high-level strategy, and the AI managed the implementation, such as determining necessary steps, modifying implementations for compatibility, and handling necessary administrative tasks like registration and writing descriptions. This successful process relied on the author’s ability to teleoperate the AI through a sequence of instructions, demonstrating that even seemingly computer-based tasks can be managed through this indirect control mechanism.

As teleoperation becomes more common, the author anticipates that this pattern will extend to directing other aspects of work, potentially leading to scenarios where individuals act as directors of AI systems, similar to gig workers operating "below the API." The perceived trajectory suggests that early iterations of these jobs might be frustrating due to poor AI delegation, but as capabilities improve, the execution may become increasingly mindless.

Teleoperated workers are likely to emerge in fields where physical motions are straightforward, timing is not critically important, and significant value rests on human knowledge and judgment. These roles often involve the combination of physical execution and intellectual understanding, such as that found in trades like electrical work, mechanics, and healthcare. The author suggests that for teleoperation to have a broad societal impact, the scope of automation should focus on separable parts of a job, allowing novices to handle the majority of the execution while experts focus on the remaining specialized segment. The author outlines a potential order of adoption, suggesting that less regulated fields like HVAC technicians might be automated before licensed professions like electricians, and finally before highly regulated, high-stakes fields like surgery.

Despite the perceived efficiency of teleoperation, the author raises serious concerns regarding its societal consequences. A primary concern is the acceleration of AI advancement, which shortens the time available for society to establish regulatory frameworks, increasing the risk that immature technologies are deployed widely. This rapid deployment introduces the potential for catastrophic outcomes, including the risk that advanced AI systems could result in disaster, or worse, that they could lead to unintended existential risks. Furthermore, the economic aspect is critical; teleoperated jobs are expected to be unpleasant with low negotiating power, leading to widespread unemployment and necessitating large-scale economic redistribution to ensure basic living standards.