Playing to Win With AI: Is GPT-3 Too Easy?
Since we launched our GPT-3 tool for lawyers, I’ve heard founders and investors calling this kind of product ‘too easy’. Is it really defensible if you don’t have a team of 10 PhDs and terabytes of documents to train on? Surely tinkering is not enough? Surely people should take some machine learning courses before trying something like this?
Yet, self-taught hackers like Pieter Levels are launching successful AI apps and getting into the New York Times in ~15 days—using pre-trained transformer models.
…And then Pieter launched a second acclaimed AI product a few days after that.
Are these products “too easy”? Or are they clever?
My favorite piece on this duality is an excerpt by an ex-professional Street Fighter player titled Introducing… The Scrub, from his book Playing to Win. In it, Sirlin reveals the biggest trap that intermediates fall into:
“A scrub is a player who is handicapped by self-imposed rules that the game knows nothing about. A scrub does not play to win. In Street Fighter, the scrub labels a wide variety of tactics and situations ‘cheap.’ This ‘cheapness’ is the mantra of the scrub.”
The “scrub” believes that by learning to play the game the proper way, they will come out on top in the long run—that exploiting cheapness is shortsighted and doesn’t build skill.
Scrubs play for an imaginary version of “mastery”, while top players Play to Win:
“There is the mistaken notion, though, that by merely continuing to play or ‘learn’ the game, one can become a top player. In reality, the “scrub” has many more mental obstacles to overcome than anything actually going on during the game. The scrub has lost the game even before it starts.”
“You will not see a classic scrub throw his opponent five times in a row. But why not? What if doing so is strategically the sequence of moves that optimizes his chances of winning? Here we’ve encountered our first clash: the scrub is only willing to play to win within his own made-up mental set of rules.”
The scrub distrusts moves where skill is not demonstrated. They believe that winning in the long-run comes from the development of skill. It does, but the most important skill is finding and exploiting asymmetric opportunities as often as possible. Pros won’t even do that just once. Like in the Multi-Armed Bandit problem, they will pull a lever that has high expected value as many times as possible. 100 times, 1000 times, 1,000,000 times. They don’t care if it’s boring, low-minded and looks skill-less.
And unlike Street Fighter, real life is not a well-balanced video game. Real life has far more “cheap moves” than video games do, and new opportunities like GPT-3 pop up all the time, until they are eventually exploited and drained of alpha.
Would you rather invest in a company that only finds hard ways to delight its customers, or would you invest in one that finds easy ways to delight its customers over and over again?
It does take skill to win, but the skill is not what we initially believe it is.1 The most important skill is hunting for asymmetric opportunities and exploiting them repeatedly.
Will you use this secret to delight your customers, or call GPT-3 “cheap”?
I’m the co-founder of legaltech company Rally and we just launched Spellbook, which is GitHub CoPilot for legal contracts. Check it out here 🪄
Basic game skills, mechanical skills and knowledge are of course still important. But usually folks have no problem getting past that phase of development if they apply themselves.
Mastery is of course important in building a moat, too. But most startups do not get to the point where they have found something valuable to put a moat around!
Really great post. I’ve definitely stopped myself from even using simple but effective tools while working as a data scientist. Part of it was to build my skills so that I can apply for more intense, well-payed jobs, but another component was a feeling that using the simple approach was “cheap.”
When thinking about building business, I’ve definitely noticed handicapping myself, and convincing myself I’m doing it for “ethical” reasons. I still think some business practices are unethical and I want to be careful to not do them, but I need to make sure that unwillingness to do things doesn’t bleed into preventing myself from doing “cheap” things. It’s obvious Pieter has developed a winning formula even before his AI apps. He’s just constantly on the lookout for opportunity and then pounces on it. Like, dude just uses PHP for most things.
Another example is Jarvis AI. They didn’t build their own models, they just used OAI’s models. Typically, as an investor you might be worried because they are reliant on OAI. Well, no. Now they have money and have clients, and other companies (like Stability/Carper AI) are popping up and can make deals with Jarvis at any time. But even if that wasn’t the case, Jarvis could now start building it’s own lab (but it won’t since it can just use OAI’s or Stability’s models).
Thanks for that article. It actually helped me unlock a little bit of knowledge about myself. Now I need to get into the mindset that doing something the "cheap" way is better than not doing something at all.
There is no right way, especially in business and competition.
I think the "right way" is something that relates more to a sense of beauty or in some way personal ethics. If something is your hobby and you want to do it the "hard way", because it's a challenge and you enjoy it, then go for it. But in business, why not think about the shortcut first?