Reflections on generative AI on the three year anniversary of ChatGPT 3.5
This is a crosspost from my “AI in Business Education” Substack.
The first week of December 2022, several bloggers I follow started posting about an astonishing new AI tool, ChatGPT 3.5, so I gave it a try. I was immediately impressed by its usefulness and flexibility. I spent the following weeks, really all of my winter break, experimenting and envisioning what this new generative AI could mean for education, our economy, and our society.
On December 22, 2022, I wrote an email to my colleagues in the Department of Marketing at the McCombs School of Business describing how I expected generative AI to be a civilization-altering technology that would ultimately transform student workflows, urging my colleagues to test the tools themselves and adapt their courses despite the software’s early limitations. The image at the top of this post was included in the email. Full text of my email is at the bottom of this post.
My perspective on evaluating generative AI was shaped by my professional experience with the emergence of the internet while a marketing executive at Dell, Inc. Less than two years after starting at Dell in 1992, I found myself leading a team charged with making the phones ring with people wanting to buy computers. Our only tools at that time were direct mail and print advertising. By the time I left Dell in 2002, thanks to Michael’s vision, Dell had become a global leader in e-commerce.
In December 2022, I recognized ChatGPT 3.5 as representing the leading edge of a new technology that would be at least as transformative as the advent of the internet, likely more.
What I got right
In rereading my email of initial impressions of ChatGPT, and a follow-up I sent January 3, 2023, I got some things right:
“Tremendous Impact on Our Civilization”.
Unlike other popular buzzwords that winter, such as Metaverse or NFTs, generative AI is proving to be not just a tech fad. Today, AI is integrated into coding, law, medicine, creative arts, and business strategy. It is altering the labor market and the trajectory of the tech industry.
“I expect my students will use ChatGPT as a starting point from which to further edit and enhance their answers.”
This is mainly how business students (and professionals’) workflow has evolved. AI is less “copy-paste-submit,” and more “generate outline/draft-edit-refine.” Students are using these tools regardless of intent or rules, forcing faculty to adapt.Faculty should “review assignments, quizzes, and exams and anticipate how students will use this tech.”
Our faculty have been learning how to make assignments “AI resistant” and have embraced integrating AI into assignments. AI can be used to create an engaging learning experience, and we don’t have to lose critical thinking with AI.“ChatGPT has an unrealistic perspective on the impact of its own AI technology.”
ChatGPT’s self-assessment that it wasn’t intended for education and couldn’t understand business concepts was unrealistic. Today, custom GPTs and LLMs are specifically fine-tuned for business education, data analysis, and tutoring. Education is likely to account for half or more of generative AI use today.
What changed faster than I expected
The next generation GPT-4 launches in mid-2023 and is anticipated to be 10x better.”
ChatGPT-4 launched in March 2023, earlier than I expected, and, in terms of reasoning capabilities and standardized-test performance, the jump from 3.5 to 4 was exponential, not incremental.
Today, frontier AI models continue to make impressive gains, although there is some evidence that recent gains, while still remarkable, may be slowing. ChatGPT is also struggling to keep up with the competition. As of today, my primary AI tools are Google Gemini Pro 3 for summarization, analysis, and content creation; Claude Opus 4.5 for creating presentations and spreadsheets; and Perplexity (with Gemini Pro 3) for searching the internet. I rarely use ChatGPT.
In the coming years, we have much change to absorb with this technology, but I am confident that overall, we have a lot to look forward to and will find ourselves even better off.
Peace through understanding.
Email to my Department of Marketing colleagues
From: Ben Bentzin
Subject: Anticipating the impact of AI on business education
Date: December 22, 2022 at 1:33:04 PM CST
To: Marketing facultyAs we settle in for our well-deserved holidays, the coming weeks of preparing for the spring 2023 semester are a good time to reflect on AI technology and anticipate the impact on our courses.
Over the past several weeks there has been considerable publicity about the OpenAI initiative with their free ChatGPT demonstration of a very impressive natural language AI, as well as their DALL-E image generating AI. If you haven’t already done so, I suggest taking both for a spin.
ChatGPT natural language AI: https://openai.com/blog/chatgpt/
DALL-E image generating AI: https://openai.com/dall-e-2/
Playing with these technologies over the past weeks it is clear the current GPT 3.5 (ChatGPT) technology will have a tremendous impact on our civilization. The next generation GPT-4 launches in mid-2023 and is anticipated to be 10x better. Our AI future is unfolding rapidly.
We should expect that our students will use ChatGPT and similar AI tech in completing their coursework. As faculty we can review assignments, quizzes, and exams and anticipate how students will use this tech.
ChatGPT still has a long way to go. The AI was instructed using a data set through 2021, so it isn’t knowledgeable about current events. The OpenAI Initiative cautions, “ChatGPT sometimes writes plausible-sounding but incorrect or nonsensical answers.” In my own testing, ChatGPT gave consistently incorrect answers to the prompt, “What are the natural lakes in Texas?”
I recently used ChatGPT to generate answers to a 10-question open book, open internet quiz I used in an Executive MBA course this semester. Using only ChatGPT to answer the questions would likely have resulted in a score of 3 or 4 points out of a possible of 10 due to specialized concepts and vocabulary, but all the ChatGPT answers were impressive. I expect my students will use ChatGPT as a starting point from which to further edit and enhance their answers.
We have opportunities to integrate emerging AI technologies into our courses. This blog is a good idea starter.
https://medium.com/@rwatkins_7167/updating-your-course-syllabus-for-chatgpt-965f4b57b003
ChatGPT has an unrealistic perspective on the impact of its own AI technology. The ChatGPT response to the prompt “How will ChatGPT impact business school education?”:
It is not appropriate to consider the impact of ChatGPT on business school education as ChatGPT is a proprietary language model developed by OpenAI and is not intended for use in educational settings. While ChatGPT may be able to generate text and responses to specific prompts, it is not capable of providing a comprehensive or nuanced understanding of business concepts or topics.
While ChatGPT may profess that it is not intended for use in education settings, our students will certainly use it. As faculty we should practice anticipating how AI will influence business education.
Interesting retrospective and prospective article on AI. While this article is centered on AI in upper education, it clearly has relevance for many industries.
The essential reality is that AI is a new tool. Those that learn to effectively use a better tool will have the advantage. I think this is the main point of your article: AI is here and an effective tool, academia must learn to embrace it and their students’ use of it.
To take that point further, those that wish to suppress the use of AI will only suffer as others who learn to use the tool effectively will outcompete them.
There are concerns that AI will replace the work currently done by people, and that the AI work product will be inferior to what competent professionals can create. Furthermore, AI may be embraced because by replacing people it will cost less for companies to use AI. I understand these concerns, and I think it will happen by unenlightened companies. But this is missing the bigger story.
Software engineering has the concept of levels of abstraction. Simply, as programming languages have progressed from machine code to compiled languages, libraries and drivers, the work happens at a higher level of abstraction. That is, instead of writing code to move the printhead and then tell it to strike the paper in the shape of a letter, now you just need to write the Print command, and everything is done for the programmer. The programmer never needs to know how a printer works. But by moving up to a higher level of abstraction where one line of code replaces hundreds of lines that were needed before, it frees up the programmer to think and work at a higher level.
AI is the same. Those that figure out how to use it to move up to a higher level of abstraction, a higher level of thinking, will have the advantage. Academia will serve their students well if they teach how to use AI to work at a higher level.
The winners are the ones who are unburdened by tasks in which they can add only minimal value and instead realize how to use this new tool to free up their minds to that higher level of abstraction, and create something heretofore impossible.
Remember, all to the physics to create the modern cellular phone network has existed for billions of years. Yet, only in our lifetime has the extensive cellular phone network come into existence. All because lower levels of abstraction were mastered and commoditized, so that a higher level, and then an even higher level of abilities were imagined and mastered.
AI is the next tool available for us to master. Those that master it the fullest and the fastest will take the lead.