Karl Ulrich and Christian Terweisch, two professors at the prestigious M7 business school, conducted the research, which pitted the creative power of Wharton MBA students from a 2021 class against ChatGPT-4, the updated version of the generative AI tool that launched earlier this year.
In 2021, the MBA students were tasked with finding 200 ideas for products costing less than $50 that would appeal to college students. Ulrich and Terweisch chose to use ideas from a 2021 class as they predated the launch of ChatGPT in November 2022.
The same task was then given to ChatGPT. First, it was required to come up with 100 fresh ideas without additional examples or contextual information, the Wharton professors then fed it a number of good product ideas it could use to generate a further 100.
The study found the generative AI tool produced 200 ideas in less than 15 minutes, far quicker than the average human being who typically produces five ideas in that time.
To test the quality of the ideas, Ulrich and Terweisch surveyed college students to identify which of the product ideas they were most likely to buy. The study revealed 47% of ChatGPT-generated ideas were likely to be purchased, compared with 40% of the ideas produced by Wharton MBA students.
The “seeded” ideas—those which were based on good product examples—elicited an even stronger result, with 49% purchase probability.
Terweisch, who wrote about the study in a recent Wall Street Journal Op-Ed and spoke on it during an episode of the Wharton Business Daily podcast, said the results proved using ChatGPT is a no brainer when it comes to generating ideas.
“It’s cheap. It’s fast. It’s good. What’s not to be liked? Worst case is you reject all of the ideas and run with your own. But our research speaks strongly to the fact that your idea pool will get better,” he said.
The speed of idea generation—a professional using ChatGPT could come up with as many as 800 ideas per hour, according to the study—also meant innovation could become a quicker process, he added.
“We should not boil it down to man versus machine. We need to find ways in which the AI becomes your creative co-pilot.”
Earlier this year Terweisch conducted a similar experiment at Wharton which saw ChatGPT outperform students in an MBA exam.
