Suppose I want to do something challenging. Write this essay well, lift weights, prepare a delicious meal. There are lots of factors that go into how well I do, but two obvious ones are talent and effort.
If I literally don’t try at all, or if I have none of the requisite skills, I will utterly fail (the dark zone around the edge). If I work kind of hard and am okay at the thing, I’ll do a mediocre job. And if I want to produce something truly excellent, I’ve got to both work hard and have the horsepower.
For cooking, professional head chefs at top restaurants fall in the brightest zone when they’re at work and not hung over, while their sous chefs - or the same head chefs when they are hung over - would hang out in the auric band just outside it. A foodie probably falls in the solid orange range. I’m in the “can make sandwich” zone of darkest burgundy.
But what happens when automation comes into play? It can look a few different ways.
Scenario One: Fixed Ceiling, Raised Floor
Imagine a group of people preparing for the pun-off world championships, before and after the advent of the internet. Both before and after, the cleverest people will tend to win, and among the cleverest, those who bother to do extensive preparations have the edge. But after the internet, those preparations are a lot less onerous. Before, you might have to look up lots of words in the thesaurus, check out books of poetry from the library as inspiration, and correspond by mail with the champions of yore. After, you can look up synonyms online, browse poetry archives in bed, and email the greatest punsters alive.
The pre-internet effort/talent gradient would look like our original heat map. The post-internet one might look like this:
The brightest zone is still confined to the upper right corner, and in particular maximum talent is still a major determinant of maximum success. But merely adequate performance is quite easy - even someone with low talent and low effort can plagiarize a list of puns they find online, for instance.
I call this vision Fixed Ceiling, Raised Floor because there are major effects near the bottom, and labor-saving effects throughout the entire distribution, but at the tippy-top, talent is still king. There’s some evidence that current AI assistants are like this; low performers do a lot better, while for high performers the gains are subtler.
In an augmentation curve like this, who are the winners and losers? Just look at the difference between the graphs!
In the dark blue areas, there’s not much change. If you want the very best outcomes, you still have to be talented and work hard. And similarly, if you don’t try or are extremely unskilled, you aren’t going to get anywhere. The red areas, by contrast, show major changes: lazy savants can now automate the pesky busy work that was between them and good performance, and hard-working amateurs - formerly consigned to mediocrity - can muster a passable performance.
The very talented and very diligent are both big winners here. High talent, especially, is advantaged: a lazy maestro can achieve solid results while barely lifting a finger. But the highly diligent, too, have new affordances to muddle through. Who loses? In competitive environments, everyone else. The semi-talented semi-diligent worker suddenly loses out to both their harder working and more talented peers! Beginners without truly ferocious hustle, especially ones who trained for a different distribution than the ones they now find themselves in, may be lost at sea.
Scenario Two: Only Schlep Remains
Consider a mom and pop coffee shop, where the owners amass skill over time in performing the myriad tasks of running a business. They have to manage inventory, decide what promotions to run, forecast sales, implement refund policies, and more. And on top of all this, they have to actually prepare and pour the coffee (or hire baristas).
But then, when mom and pop are getting on in years, they’re approached by CoffeeCorp, a conglomerate that runs thousands of shops. CoffeeCorp offers to buy them out, and replace all their cognitively demanding business running acumen with simple, market tested procedures. CoffeeCorp will set prices, run optimized promotions, handle marketing, etc. All mom and pop will have to do is the basic grunt work - even the specific coffee products will come pre-mixed. Just add water!
Talent doesn’t literally not matter anymore - someone still has to operate the CoffeeCorp playbook, and a bright barista who remembers people’s names is a plus. But in this situation, success is mostly a matter of putting in the motions.
The difference between this and the status quo is a little similar to that of scenario one, but with reversed emphasis:
As before, the bottom left and top right corners don’t really benefit, while the bottom right and top left do. But here, it’s the top left that reaps the lion’s share of the benefit. If mom and pop just hire baristas who always show up on time, serve with a smile, and keep the floors clean, CoffeeCorp will handle the rest.
Scenario Three: Dude, Where’s My Eschaton?
Immanetized.
I think there’s a bit of a gulf between certain kinds of AI forecasters. Lots of people expect variants of scenarios one and two, which are different flavors of the same thing. The effort-talent gradient shifts down and to the left, though maybe more of one than the other. We get to work less hard and be less talented, and still have nice things. Could be scary disruption, but overall it’s a good thing.
Other people are expecting Scenario Three. To spell it out, in this situation effort and talent do not matter at all. The automation can do literally everything for you. This would be the equivalent of me having typed “Hey ChatGPT, do an essay”, and ChatGPT writing, formatting, and publishing the entire thing.
Human instincts about automation are generally that we’ll see curves like one or two. But truly general artificial intelligence, a machine that can accomplish any cognitive task that a human can, produces curve three for those tasks.
I’m not saying I think curve three will definitely happen, and of course it won’t happen across the entire space of human behavior - humans will probably mostly (but maybe not exclusively!) want emotional connections with other humans, for example. But in the domain of cognitive work, AGI - the stated goal of multiple companies with every expectation of succeeding - is scenario three.
What does that mean? I don’t know! It’s very hard to forecast. But if you’re wondering why everyone is losing their heads over this AGI stuff, here’s one reason.
Though it’s not the only one.