Navigating the Jagged Technological Frontier: Field Experimental Evidence of the Effects of AI on Knowledge Worker Productivity and Quality

SSRN Electronic Journal, Journal Year: 2023, Volume and Issue: unknown

Published: Jan. 1, 2023

The public release of Large Language Models (LLMs) has sparked tremendous interest in how humans will use Artificial Intelligence (AI) to accomplish a variety tasks. In our study conducted with Boston Consulting Group, global management consulting firm, we examine the performance implications AI on realistic, complex, and knowledge-intensive pre-registered experiment involved 758 consultants comprising about 7% individual contributor-level at company. After establishing baseline similar task, subjects were randomly assigned one three conditions: no access, GPT-4 or access prompt engineering overview. We suggest that capabilities create "jagged technological frontier" where some tasks are easily done by AI, while others, though seemingly difficulty level, outside current capability AI. For each set 18 realistic within frontier capabilities, using significantly more productive (they completed 12.2% average, task 25.1% quickly), produced higher quality results (more than 40% compared control group). Consultants across skills distribution benefited from having augmentation, those below average threshold increasing 43% above 17% their own scores. selected be frontier, however, 19 percentage points less likely produce correct solutions without Further, analysis shows emergence two distinctive patterns successful along spectrum human-AI integration. One acted as "Centaurs," like mythical halfhorse/half-human creature, dividing delegating solution-creation activities themselves. Another "Cyborgs," completely integrating flow continually interacting technology.

Language: Английский

---

The Uneven Impact of Generative AI on Entrepreneurial Performance

Published: Dec. 21, 2023

Scalable and low-cost AI assistance has the potential to improve firm decision-making economic performance. However, running a business involves myriad of open-ended problems, making it difficult know whether recent advances can help owners make better decisions in real-world markets. In field experiment with Kenyan entrepreneurs, we assessed impact advice on small revenues profits by randomizing access GPT-4-powered assistant via WhatsApp. While are unable reject null hypothesis that there is no average treatment effect, find effect for entrepreneurs who were high performing at baseline be 0.27 standard deviations greater than low performers. Sub-sample analyses show performers benefited just over 15% from assistant, whereas did about 8% worse. This increase performance inequality does not stem differences questions posed or received AI, but how selected implemented they received. More broadly, our findings demonstrate generative already capable impacting—though uneven unexpected ways—real, open-ended, unstructured decisions.

Language: Английский

---

A Turing test of whether AI chatbots are behaviorally similar to humans

Proceedings of the National Academy of Sciences, Journal Year: 2024, Volume and Issue: 121(9)

Published: Feb. 22, 2024

We administer a Turing test to AI chatbots. examine how chatbots behave in suite of classic behavioral games that are designed elicit characteristics such as trust, fairness, risk-aversion, cooperation, etc., well they respond traditional Big-5 psychological survey measures personality traits. ChatGPT-4 exhibits and traits statistically indistinguishable from random human tens thousands subjects more than 50 countries. Chatbots also modify their behavior based on previous experience contexts “as if” were learning the interactions change response different framings same strategic situation. Their behaviors often distinct average modal behaviors, which case tend altruistic cooperative end distribution. estimate act if maximizing an own partner’s payoffs.

Language: Английский

---

Generative AI for Economic Research: Use Cases and Implications for Economists

Journal of Economic Literature, Journal Year: 2023, Volume and Issue: 61(4), P. 1281 - 1317

Published: Dec. 1, 2023

Generative artificial intelligence (AI) has the potential to revolutionize research. I analyze how large language models (LLMs) such as ChatGPT can assist economists by describing dozens of use cases in six areas: ideation and feedback, writing, background research, data analysis, coding, mathematical derivations. provide general instructions demonstrate specific examples take advantage each these, classifying LLM capabilities from experimental highly useful. argue that reap significant productivity gains taking generative AI automate micro-tasks. Moreover, these will grow performance systems continues improve. also speculate on longer-term implications AI-powered cognitive automation for economic The online resources associated with this paper explain get started regular updates latest economics. (JEL A11, C45, D83, I23, O33)

Language: Английский

---

Generative AI enhances individual creativity but reduces the collective diversity of novel content

Science Advances, Journal Year: 2024, Volume and Issue: 10(28)

Published: July 12, 2024

Creativity is core to being human. Generative artificial intelligence (AI)—including powerful large language models (LLMs)—holds promise for humans be more creative by offering new ideas, or less anchoring on generative AI ideas. We study the causal impact of ideas production short stories in an online experiment where some writers obtained story from LLM. find that access causes evaluated as creative, better written, and enjoyable, especially among writers. However, AI–enabled are similar each other than alone. These results point increase individual creativity at risk losing collective novelty. This dynamic resembles a social dilemma: With AI, individually off, but collectively narrower scope novel content produced. Our have implications researchers, policy-makers, practitioners interested bolstering creativity.

Language: Английский

---

Generative Artificial Intelligence Enhances Creativity

SSRN Electronic Journal, Journal Year: 2023, Volume and Issue: unknown

Published: Jan. 1, 2023

Creativity is core to the human experience. The advent of generative artificial intelligence (GenAI) holds promise for humans be more creative by offering new ideas and paths possibilities. However, provided GenAI may also anchor a creator, resulting in less output. Here, we study first time causal impact on production output, focusing creation short stories. In an online experimental study, some writers are offered opportunity obtain story from platform. We find that access causes increase writer's creativity 8% 9% over stories written with no assistance, as assessed third party evaluators. Stories considered better enjoyable, improvements up 22% 26% among writers. Despite positive subjective evaluations, analysis using embeddings texts demonstrates similar each other—and initial idea—than alone. Consequently, produced assistance viewed reflecting author's own ideas. Our experiment designed inference rather than personalized writing experience, suggesting further development able push boundaries further. results have direct implications researchers, policy-makers practitioners interested bolstering all sectors economy.

Language: Английский

---

Artificial intelligence and consumer behavior: From predictive to generative AI

Journal of Business Research, Journal Year: 2024, Volume and Issue: 180, P. 114720 - 114720

Published: May 23, 2024

Language: Английский

---

AI-Augmented Brainwriting: Investigating the use of LLMs in group ideation

Published: May 11, 2024

The growing availability of generative AI technologies such as large language models (LLMs) has significant implications for creative work. This paper explores twofold aspects integrating LLMs into the process – divergence stage idea generation, and convergence evaluation selection ideas. We devised a collaborative group-AI Brainwriting ideation framework, which incorporated an LLM enhancement group process, evaluated generation resulted solution space. To assess potential using in we design engine compared it to ratings assigned by three expert six novice evaluators. Our findings suggest that could enhance both its outcome. also provide evidence can support evaluation. conclude discussing HCI education practice.

Language: Английский

---

Generative artificial intelligence in graduate medical education

Frontiers in Medicine, Journal Year: 2025, Volume and Issue: 11

Published: Jan. 10, 2025

Generative artificial intelligence (GenAI) is rapidly transforming various sectors, including healthcare and education. This paper explores the potential opportunities risks of GenAI in graduate medical education (GME). We review existing literature provide commentary on how could impact GME, five key areas opportunity: electronic health record (EHR) workload reduction, clinical simulation, individualized education, research analytics support, decision support. then discuss significant risks, inaccuracy overreliance AI-generated content, challenges to authenticity academic integrity, biases AI outputs, privacy concerns. As technology matures, it will likely come have an important role future but its integration should be guided by a thorough understanding both benefits limitations.

Language: Английский

---

The Crowdless Future? How Generative AI Is Shaping the Future of Human Crowdsourcing

SSRN Electronic Journal, Journal Year: 2023, Volume and Issue: unknown

Published: Jan. 1, 2023

This study investigates the capability of generative artificial intelligence (AI) in creating innovative business solutions compared to human crowdsourcing methods. We initiated a challenge focused on sustainable, circular economy opportunities. The attracted diverse range solvers from myriad countries and industries. Simultaneously, we employed GPT-4 generate AI using three different prompt levels, each calibrated simulate distinct crowd expert personas. 145 evaluators assessed randomized selection 10 out 234 solutions, total 1,885 evaluator-solution pairs. Results showed comparable quality between AI-generated solutions. However, ideas were perceived as more novel, whereas delivered better environmental financial value. use natural language processing techniques rich solution text show that although cover similar industries application, exhibit greater semantic diversity. connection diversity novelty is stronger suggesting differences how created by humans or detected evaluators. illuminates potential limitations both solve complex organizational problems sets groundwork for possible integrative human-AI approach problem-solving.

Language: Английский

---