ChatGPT and Cautious Optimism
Imminent Job Killer? Probably Not.
Unless you’ve been hiding from all internet things for a few weeks, which honestly isn’t a bad idea in general given the state of the internet, then you’ve undoubtedly heard of ChatGPT, which is the latest advance in AI processing from OpenAI.
OpenAI, of course, has brought us GPT-2, GPT-3, and DALL-E, which are generative language and image AI tools. And ChatGPT took the world by storm as an intelligent chatbot you can ask questions to, and get real (like more real than ever before) answers.
And if you’ve spent any time on TikTok, or Twitter, or even Slack threads, you’ve probably been inundated with hot takes like this:
Tiktok failed to load.Enable 3rd party cookies or use another browser
I know one UX design manager who, almost gleefully, posted about ChatGPT for all product managers to take heed, because it could write PRDs(!). And if an intelligent chatbot can write PRDs, then it’s only a matter of time before product managers won’t be needed at all.
Because that’s all product managers do I guess? And then the designers can take over? I’m not sure what the end game there is, but the undertones were certainly that product managers should be concerned (while designers need not be).
Hopefully, you’ve experimented with ChatGPT over the past week or two. While I don’t think anyone is imminently in danger, I do think it is a seminal moment in technology. I even took some time to have both my kids experiment with the chat tool so they could see what it could do. Unfortunately, they were far less impressed than most of the internet.
But we won’t let that stop us from dissecting this moment a little further, the implications for us in product and technology, and what could come next.
The Cool:
First, this is a huge step forward. In the space of a few years we’ve gone from GPT-2 to GPT-3 and even seen Dall-E, which can generate images based on text input. To now ChatGPT, which takes all of the underlying language training and makes it incredibly accessible in a way we haven’t seen before.
Second, this has the potential to level up humans by bringing knowledge to our fingers in a new way. If you’ve experimented with the tool at all, you know you can ask anything and generally get responses back. Whether that’s to write you an essay (high school students) or write you a block of code. So we have the ability to begin to automate away some of the more mundane tasks that we may be faced with.
In the example above, I asked for a simple chat application and got a response for Python. I’m still learning Python, so I can’t validate this code yet, but it probably is the beginning of a usable application. Certainly not the end or what I would want to go to market with, but a time-saver for me if I don’t want to do some of the simpler parts and just need them done more quickly.
Finally, ChatGPT has the potential to consolidate the search for specific answers that everyone was doing into a single, concise location. While Google acts like a librarian, serving up all the different places you could find an answer for “chocolate chip cookie recipes”, ChatGPT has the potential to simply serve you a recipe for chocolate chip cookies, along with baking instructions. And that may be exactly what you want or need.
The Misunderstood:
First, ChatGPT is still not right all the time. It serves up answers as definitive because it is searching all its known information and recognizing patterns within that data, and then putting those patterns together. But those patterns and that data could be wrong, leading to wrong answers that look correct.
For example, Stack Overflow has banned ChatGPT answers as many users, in their enthusiasm, were taking questions from the site to ChatGPT then posting the responses to Stack Overflow without validating them. That is a recipe for a lot of bad code as they stated in their reasoning:
Overall, because the average rate of getting correct answers from ChatGPT is too low, the posting of answers created by ChatGPT is substantially harmful to the site and to users who are asking or looking for correct answers.
Second, ChatGPT is just “pretty good” right now. I’ve lost count of the times I’ve heard that said on videos about the tool. “I used it to generate a contract, and it was pretty good.” “I used it to write an essay on a novel and it was pretty good.” “I used it to create a PRD and it was pretty good.”
Pretty good may be passable in a high school English class, at least enough to get a C- and move to the next essay. But pretty good doesn’t cut it when a contract is going to be binding in a court. Or when you are presenting to a board of directors. So for now, it seems like the answers, the writing, and the aggregation of information, is “pretty good.” That will likely improve with additional training, additional data, and more use, but we’re still in an adolescent phase.
The PRD I created below is pretty good, no doubt. It is a great starting point. It would certainly save me some time, but is in no way a finished product for the work we do.
Third, and along the same lines as above, we still need humans in the loop to ask the right questions and make the right prompts. My examples so far have been relatively simple. You don’t have to dig too deep to see where this breaks down, though.
For a more sophisticated feature, my prompts would have to be much more sophisticated to elicit the right response. There are plenty of videos and examples out there of users creating 200 word questions, filled with qualifiers and specific instructions, in order to get the right outputs.
Fourth, ChatGPT is not original in any way. No existing AI is. It is a recombination of existing data on which it has been trained. It is getting very good at that, and this is very useful (see my second point under “the cool”), but it is not giving us original thought or insight. For that, we still need people. And that’s probably a good thing for now.
Finally, it’s still a machine and bound by the amount of information we can feed it. ChatGPT and GPT-3 have been fed and trained on incredible amounts of data. We haven’t reached the limits on what we can train, but we conceivably could reach that limit at some point. And even before that, we can’t expect the machine to do more that what they have trained it to do. Which, for now, is incredibly impressive.
So…
We’re certainly seeing the beginning of an evolution of new technology. What is taking shape now will have far-reaching implications for all of us, and I suspect it can be a massive force for good (though there will be plenty of misuses along the way). While ChatGPT and the underlying AI aren’t necessarily the imminent threats that many have called them, they have the potential to reshape our industry and many industries in ways we won’t be able to predict. So hold on to your butts.
