Efficiency vs Learning

Are search engines more pedagogical than chatbots?

Welcome to Tech Futures Project. Each edition explores whether tech can support human flourishing—and what gets in the way.

---

Are we increasingly trading off learning for greater efficiency? A new study from researchers at the University of Pennsylvania sheds light on this question. They studied whether we learn more from using chatGPT versus a search engine - and found that efficiency comes at a cost.

In the study, participants across four experiments aimed to give advice to a friend on planting a vegetable garden. One group used a search engine to research and write their advice and another group used ChatGPT. The group who used a search engine took longer but they learned more, felt more ownership over their advice and produced advice which readers found both more original and more useful. Crucially, when blind recipients evaluated the advice - not knowing which tool had been used - they found the ChatGPT-informed advice less helpful, less trustworthy, and were less likely to adopt it. Though using ChatGPT was faster and more efficient, the participants learned less. This reveals a dichotomy at the heart of LLM use in their current chat-based format: while they are more efficient in the short run, they may reduce learning and human capability in the long-run.”

The ease of chatGPT prompts the brain to expend less cognitive effort, while we know that effort and ‘desirable difficulty’ is good for learning. ChatGPT users checked out of learning quicker (i.e. they thought they were finished sooner, perhaps indicating they were less aware of the limits of their understanding). Additionally, using a search engine appears to promote active synthesis of information which helps learning by forcing us to effortfully create schemas. The information form is crucial, according to the study. In one experiment the researchers held the information content constant - showing people the exact same facts, just formatted differently (as an LLM summary vs. as separate web links). Even with identical information, people learned less from the LLM format.

Does this study show AI will inevitably harm learning? I think it is not yet clear. We could design more pedagogical AI that is more difficult to use than a search engine (in some senses OpenAI, google and Anthropic are trying to do this with their socratic modes). However, adding friction resists the logic that has governed how digital technology has been built over the past half century at least (which has been to prioritise efficiency over learning.) As these authors argue, ‘disfluency’ and ‘friction’ are key ingredients to encouraging people to think hard and to learn. The smooth, frictionless and convenient logic of digital technology products tends to run counter to this.

As with tech products, our information environment in general has arguably not evolved to enhance human cognition since Daniel Boorstin wrote that film, mass publishing and photography in mid-20th century America would be ruinous for cognition: ‘We expect our reading, like our other experience, to be digested for us’. Indeed, wider economic incentives often favor efficiency over learning. Labour productivity is measured as output per hour, so if ChatGPT helps someone complete tasks faster, productivity increases even if their capability declines.

Technology is not deterministic and we don’t need to use AI to make everything easier.¹ As with any technology we can think about using them in interesting and useful ways. Perhaps the question is whether individuals and institutions have the desire to prioritise learning even in the face of improving efficiencies.

1. There is actually some evidence that friction is coming back into fashion.