Interview with economist Tyler Cowen (George Mason University, Marginal Revolution) on how ChatGPT is changing work and the economy. Topics include his predictions on AI's short- and long-term effects, the "allocation economy" and why charisma and strong executive function may be the most rewarded skills, using ChatGPT for deep reading and research, treating it as a "universal translator" while traveling, pairing ChatGPT with Perplexity AI, why he worries less about hallucinations, his preference for the Playground, using ChatGPT in university classrooms, and a test of a "Tyler bot" personality clone. # Channel Identity Observations — Ep. 7 with Tyler Cowen ## Explicit Self-Descriptions by the Host - **Show format / recurring structure**: The host states a clear formula: *"what we tend to do on this show is talk very practically about how smart people use ChatGPT and AI in their lives to get work done."* - **Numbered episode identity**: The title includes "Ep. 7," indicating an episodic interview series. - **Audience questions segment**: The host solicits questions externally — *"before we recorded I asked Twitter for questions"* and *"a bunch of people asked questions. So I want to do a little bit of a rapid-fire question asking of Twitter questions"* — demonstrating an interactive, community-engaged format. - **Recurring interactive segments**: The host introduces an original bit, the **"Tyler test"** (*"a segment with you called the Tyler test, which [is a] play on the Turing test"*), showing the channel designs custom demonstration segments around AI. ## Intended Audience - **Knowledge workers evaluating AI's impact on their careers**: The host frames the framing question around people *"trying to figure out what AI is and how it's going to affect their work lives and the economy more generally."* - **Practitioners interested in tool workflows**: Detailed walkthroughs (playground settings, temperature, Perplexity vs. ChatGPT vs. Google) indicate an audience that wants hands-on usage guidance, not just theory: *"AI's for learning, Perplexity is for references, Google is for links."* - **AI-curious mainstream users**: The host personalizes tools via relatable anecdotes (his father as an early adopter "canary in the coal mine," reading *Moby Dick* with ChatGPT), suggesting accessibility for non-experts. ## Claimed Scope - **Interview-based show featuring notable thinkers**: Guest credentialing is explicit — *"you're an economist at George Mason University... You've written, I think, 17 books, and you've been writing your blog Marginal Revolution for... over 20 years."* - **Scope spans high-level analysis and practical demos**: The episode moves deliberately from macro questions (jobs, economy, AI risk) to live screen-share walkthroughs of the guest's actual chats. - **Cross-tool coverage**: Discussion includes ChatGPT, the OpenAI playground, Perplexity, Claude/Anthropic, DALL-E, and custom GPTs — the channel claims breadth across the AI tool landscape. ## Demonstrated Presentation Patterns (Observable, Not Claimed) - **Live product demonstration**: The host and guest manipulate settings in real time (temperature, deleting assistant answers, the "minus button"), with the host instructing: *"scroll over it, click into the assistant answer, and there should be a minus button there."* - **Host acts as interpreter/guide**: He explains technical concepts for viewers — e.g., temperature is *"sort of like creativity for the model"* and the system message *"is going to set the... personality of how GPT interacts with you."* - **Builder/host experimentation**: The host personally built a custom Tyler Cowen clone (*"I actually created a clone of you"*), signaling the channel creates its own AI artifacts rather than only discussing others'. - **Host shares own practices**: He describes feeding journal entries into ChatGPT/Claude (*"what are the psychological patterns that you're noticing here?"*) and using GPT for book discovery, blending personal experience with guest expertise. ## Guest-Side Evidence (About the Guest, Not the Channel) - Cowen promotes his own projects — *econgoat.ai* and an AI-read audiobook — which the channel amplifies by offering to link: *"I would love to link to that."* - These are guest claims about the guest's work; the channel's role is observed to be a promotional/amplification platform for guests' AI-adjacent projects. ## Summary Signal for Channel-Level Synthesis Observable evidence supports a channel identity of: **an episodic interview show (at least 7 episodes) about practical ChatGPT/AI usage, hosted by an experimenter ("Dan") who demos tools live, sources audience questions from Twitter, and targets professionals navigating AI's effect on work** — with all identity claims above traceable to quoted host statements in the transcript. # Delivery Observation: Tyler Cowen — Ep. 7 Interview on ChatGPT ## Overall Delivery Style Tyler Cowen speaks in **short, declarative sentences**, often answering complex questions with compact, aphoristic conclusions rather than extended explanation: > "Worse interview questions. I would just know less." > "Google is for links. AI's for learning. Perplexity is for references." He rarely fills silences and frequently accepts or extends the host's framing with minimal words ("That's right." "Yeah." "True."). ## Recurring Phrases and Verbal Habits - **"That's right"** as agreement marker — appears multiple times when the host summarizes his view. - **Letter-grade evaluation** as a recurring assessment device: > "On any exam, I'd give this answer an A. Maybe not A plus, but a solid A." Later: "For Tyler in this I'd give it a C minus." - **Self-deprecating disclaimers about technical competence:** > "I don't have any good answer. I mean, I'm stupid, basically." And: "You'll have to tell me, cuz like I said before, I'm stupid." - **"Mhm"** as a frequent backchannel while the host speaks. ## Teaching / Explanatory Habits - **Anchors abstract claims in concrete examples**: chess ("it's now almost godlike when it plays chess"), a Paraguayan menu, a bird photo in Honduras, inflation under Emperor Aurelian. - **Turns questions into heuristics or rules**: > "If you ask it a question, what is inflation answer as would Milton Friedman? You get a better answer usually." And: "Don't assume the error is in the GPT." - **Grades by comparison to alternatives, not absolutes**: > "I know I'm doing better than I would do any other way." ## Epigrammatic Compression A stable pattern: Cowen condenses ideas into quotable maxims: > "Context is that which is scarce as I like to say." > "I would feel much less smart." > "Maybe the stupider version of me is more productive." ## Handling Uncertainty He explicitly flags uncertainty when pressed, but distinguishes it from reluctance: > "No, I'm very willing to hazard a guess, but I want the uncertainty brackets to be understood." He also repeatedly deflates his own predictions: "We're waiting, and I expect many surprises." ## Interactive / Conversational Patterns - **Asks questions back to the host**, an interviewer-like habit: "Do you want to start with that?" "Do you use it?" "You think it's a mistake for me to use it?" - **Gentle corrections delivered flatly**: when the host says AI didn't exist in 2013: "Well, in chess we had real AI in 2013." - **Deadpan humor**, often understated: > "Maybe I should now be demanding my money back, right?" And after adjusting a model setting: "This is something they're going to ban on Twitter." ## Host-Side Patterns (Dan) — for contrast - Heavy filler and repetition ("and and and", "the the the"). - Frequent summarizing turns: "So to to sort of summarize what I what I heard from you is…" - Recurring connective framing: "That's really interesting." ## Uncertainty Notes - The letter-grading and "that's right" patterns are well-supported by repetition across the transcript. - The deadpan humor and self-deprecation appear several times but are thinner evidence; treated here as recurring *moments* rather than a firmly established pattern. - No curated metadata was supplied, so tone observations rest solely on transcript text — pace, pauses, and vocal delivery cannot be assessed. # Position Extraction: Tyler Cowen on ChatGPT, AI, and Work **Source:** Video transcript, "Economist Tyler Cowen on How ChatGPT Is Changing Your Job - Ep. 7 with Tyler Cowen" **Speakers:** Tyler Cowen (economist, George Mason University; identified in transcript) and the host (identified as "Dan" at the close of the transcript). --- ## 1. Background Predictions About AI and Labor ### Cowen's original reasoning (from *Average Is Over*, 2013) - Cowen states he believed **"for a long time, even well before 2013"** that AI was the most likely source of the next major technological breakthrough. - His stated reasoning: as a young chess player, he observed that skeptics claimed AI couldn't play chess because it was "too conceptual, too complex, too non-legible" — and AI overcame those hurdles, becoming "almost godlike." He reasoned: **"if AI can do that, chess is really hard. AI can do all these other things in time."** - He notes a qualification: "I'm not an AI expert in any technical sense" — the view was intuition-based. ### Revisions given current generative models - Cowen: **"What's happening now is a work in progress. I don't feel that we know yet which groups of people it will help the most and harm the most."** - Surprise expressed: he did not expect AI's **facility with words, emotions, and "bedside manner"** — he expected "a bit more of an autonomous reasoning tool... maybe more along the visions of symbolic AI people." - Revision: **"what are sometimes called the word cell classes may be under greater threat from AI than I would have predicted"** — though he qualifies: "this hasn't all quite come to fruition yet... I expect many surprises." ### Short-run vs. long-run effects (Cowen's "best guess" with explicit uncertainty brackets) - **Short run:** "quite egalitarian" — people who "can't do things well at all now can do them pretty capably" (e.g., writing a college essay; the smartest kids already could). - **Long run:** "less egalitarian" — AI benefits **people who can start projects**, giving them "better record keepers and translators and mathematicians and coaches." - Key claim: **"large language models... are not a source of actually doing the idea... they don't make the decision to go out and do the idea."** He predicts "some kind of hyped up executive function" will be rewarded over a 10-year horizon, which could be inegalitarian because "talented people... have a lot of project ideas anyway." ### Host's summarization (accepted by Cowen) - Host frames it as: bottom quartile gains new task capabilities; skilled workers gain marginal improvement; secondary effect enables "allocation economy" people (managerial/investing skills) to deploy resources without building human organizations. Cowen: "That's right." --- ## 2. AI for Talent Evaluation - Cowen: when AI can evaluate people's talents, it will be **"highly meritocratic in a narrow way"** but "I'm not sure we'll all be happy with it" — being told exactly how good you are may "on net diminish human happiness." He notes this "is not in place now." - Host shares personal practice: journal entries fed into ChatGPT/Claude to surface psychological patterns. Cowen agrees LLMs are "wonderful therapists and analysts" and "remarkably objective when that's what you ask for," though at age 62 it's "maybe of lower value" to him. --- ## 3. Management Skills and the Allocation Economy - Host's position: managerial skills (e.g., "How far into the details should I go?") may need to become more widely distributed as everyone "moves up a layer of abstraction." - Cowen agrees and notes top consulting firms' advice is "anodyne and boring and repetitive and clichéd" — a sign GPT models can do it well. Prediction: **"everyone will have free McKinsey-level advice or nearly free,"** but consulting firms may persist to "make it stick." - Host generalizes the "translation" idea: GPT facilitates English-to-English translation between fluency levels (his father producing a product-manager-quality spec). Cowen's aphorism in response: **"Context is that which is scarce."** ### Bio/terror risk (Cowen's contrarian concern) - Cowen: a big constraint on terror organizations is that **"they're very poorly run"** — a bigger problem than their inability to find pathogens. If AI gives them better management advice, "that's actually one of the biggest risks of AI technology." He speculates current guardrails may be bypassable and that "there'll be very good AIs that will tell you that pretty soon." - Host agrees it's an "underappreciated risk" — the human coordination problem is a significant deterrent. - Related Cowen claim: **"A lot of companies might become much smaller, but still rather potent"** (citing Midjourney's small team; Cowen hedges the exact headcount: "maybe more than that, but it's not going to be so many more"). --- ## 4. Infovore Skills and Autistic-Associated Strengths - Host raises Cowen's *The Age of the Infovore*: skills like ordering knowledge and deep small-detail memory (which Cowen associated with autistic traits) are also things ChatGPT is good at. - Cowen: **"in the short to medium term, AI accelerates the value of those skills"** — but "some point much further out" AI may do all that work, and "skills like charisma and executive function are what rise in importance." He qualifies that this flip requires AI "incorporated into major workflows and systems in a way that's fairly distant" — not "just a few more years." - Cowen confirms AI has already influenced his behavior: "I've decided to do more personal appearances." --- ## 5. Practical ChatGPT Workflow (Cowen) ### Two distinct usage modes - **iPhone (travel):** universal translator (Tokyo); reading menus (Paraguayan restaurant in Buenos Aires — "I'm an idiot when it comes to Paraguayan food"); identifying plants/birds from photos (Honduras). Value claim: the shared discovery with companions "adds flavor to the trip itself" and makes it more memorable. - **Laptop (playground, not consumer ChatGPT):** learning obscure history, background context for podcast interviews (ultramarathons/Lazarus Lake; insurance economics), reading aid (Byzantine Empire, Chrysostom, Emperor Aurelian's inflation), dog questions. ### Tool stack (Cowen's stated division) > "AI's for learning, Perplexity is for references, Google is for links." - **ChatGPT → Perplexity → Perplexity's sources (e.g., JSTOR) → people.** Qualification: for something particular blocking work, start with the person first to minimize delay. Perplexity use is new ("a few weeks") and patterns "will still evolve." - Google: used "less and less for information" but "still clearly best by far" for getting clickable links for Bloomberg columns/blog posts. ### On hallucinations (Cowen's atypical stance) - **"I worry about hallucinations a lot less than most LLM users."** Low-stakes errors (bird ID) don't matter; asking "are you sure?" or "please correct any hallucination" corrects errors in his estimate **"80 to 90%"** of the time. - Domain matters: obscure history has fewer errors because questions point at "an intelligent part of the information space." Example: hour-long Jonathan Swift podcast simulation with **no errors and no redos**. Counter-example he offers: "What are the three best books to read on Jonathan Swift?" — his guess is one won't exist. - On Aurelian's inflation: GPT's "over a thousand percent over half a century" figure "has a decent chance of being a hallucination," but "the hallucination may not be worse than the hallucinations of... my fellow economic historians." - Pendleton Act query (from Jordan Schneider on Twitter): mostly hallucinated books, though it usefully jogged memory of Robert Wiebe's *The Search for Order*. Cowen: "I would know to begin with not to ask it that." ### Prompting techniques (Cowen) - Persona instructions ("answer like a sophisticated investment advisor," "answer as would Milton Friedman") point the model "toward smarter bits... in the matrices somehow." - "It loves compare and contrast" — often yields the best data. - Insurance whole-life critique answer: **"on any exam, I'd give this answer an A. Maybe not A plus, but a solid A."** ### Playground vs. ChatGPT - Cowen on why he uses the playground: "I'm stupid, basically. I like the fact that it doesn't look like a consumer product." - Host's assessment: playground sacrifices code interpreter, DALL-E, unified experience, but offers more control (system message, model, temperature, max length) and likely looser content restrictions. Temperature experiment: max temperature output "off the rails"; mid temperature produced the same points "in a better prioritized order" before derailing. - Cowen: "If I used it for poetry, I would use the higher temperatures more... I'm actually pretty fine with it at one." ### Writing - Cowen: **"I don't use it to write anything I do. I'm really very very careful not to do that. I just think it's a bad habit and it's in many cases unethical or even illegal."** If he did, he'd use Anthropic — "It sounds more natural and more human." - Host agrees Claude is "a bit better at capturing a specific tone." --- ## 6. Classroom Use - Current class (history of economic thought): **GPT-4 on the reading list; all students required to pay the $20/month subscription.** - Prior law class: students required to write one of three papers with GPT — "very successful," students learned "limits and scope of GPT." - Policy on misuse (Cowen to students): plagiarizing won't produce a great paper anyway, he'll probably know, but "I'm not going to bust you for being on one side of some vague line... Just don't do something stupid." - Observation on foreign students (especially from China) using it to improve English: **"I've seen people go from like bottom 10% of writing skill to top 10% overnight."** Qualification: "This is a real risk... It becomes a crutch and you never learn a lot about English." He personally thinks it's fine. --- ## 7. Rapid-Fire Q&A (Twitter questions) - **If ChatGPT stopped existing:** "Worse interview questions. I would just know less... I would feel much less smart." Qualified with genuine uncertainty: "maybe the stupider version of me is more productive... the returns to being smart in labor markets, they're not as high as you might think... So it may just be private consumption being smart." - **Most over/underrated:** "the problems of hallucination are somewhat overrated"; "almost everyone under uses it for most things"; he has no sense of overall ratings. - **Reading flow:** "It slows you down quite a bit. But you're still reading with GPT... you should do it in areas where you don't know a lot." - **AI speaking for him:** unsure the choice will be up to him; the audiobook of *GOAT* will be AI-read and he's "happy about it, not just fine with it" because it spares him a taxing exercise. (Host link: econgoat.ai RAG chatbot, co-created with Jeff Holmes — Cowen: "pretty good. It's not perfect. It'll get better.") --- ## 8. The "Tyler Test" Segment Custom GPT clone made by the host (no web browsing, no uploaded books), compared against Cowen's live answers: | Question | Cowen's answer | Clone's performance | Cowen's grade | |---|---|---|---| | Core lessons of economics | "Incentives matter"; "there's always an opportunity cost" | Added more principles; "no such thing as a free lunch" = opportunity cost | **A** | | (Compression follow-up) | "In economics... often less is more" — would "just pick the first few" | Initially good, then "going off the rails" summarizing in bullets | good marks, A retained | | Five words spouse would use | Curious, works hard, loves to travel, good father, should walk the dog more | Intellectual, curious, analytical, eccentric | "not worse than my answers"; missed dog/travel — "Maybe GPT-5" | | AI's impact on economics/economy in next decade | Won't fundamentally alter economics in a decade; distant possibility of simulating small economies, with **data collection as the constraint** | Answer about information costs/decision-making — "a pretty good generic answer" but "doesn't sound like me... no Tyler in it" | Accuracy fine; **C minus** as a Tyler bot | - Host's methodological note after a transcription gap in the clone's answer: "Don't assume the error is in the GPT." Cowen: "That's true. We can't assume that." --- ## 9. Publishing Experiment (*GOAT*) - Cowen made a "deliberate decision to publish a whole book 'quote' 'unquote' in GPT-4," with Claude and Gemini versions in progress. Position: **"this is the future for at least some book publishing. Like why not interact with the thing? It's an option. You don't have to do it."** - Learning: reader questions reveal "it's not what you want them to care about" — e.g., someone asked the bot what Tyler would think Milton Friedman would say about Ayn Rand. Conclusion: "people want something really quite subjective out of even economics books... that will shape how we write and think over the next few decades." --- ## Preserved Tensions and Qualifications - Cowen repeatedly brackets predictions as "best guess" with "uncertainty brackets" — the host initially read him as unwilling to guess; Cowen corrected this. - Simultaneous claims: short-run egalitarian effect vs. long-run inegalitarian effect — both held, not resolved. - Hallucination risk both acknowledged (specific examples: Aurelian figure, Pendleton Act books) and downplayed ("overrated," domain-dependent). - English-improvement via GPT: praised as transformative and flagged as a crutch risk in the same passage. # Speaker Attribution Record ## Video **Title:** Economist Tyler Cowen on How ChatGPT Is Changing Your Job - Ep. 7 with Tyler Cowen **Video ID:** `01a0f15b-34e3-7733-a381-5201ed650db7` ## Identified Speakers & Roles | Speaker | Role | Basis | |---|---|---| | **Tyler Cowen** | **Guest** | Economist at George Mason University; author; blogger (Marginal Revolution); host of *Conversations with Tyler* podcast — per host introduction in transcript | | **Host (Dan)** | **Creator / Interviewer** | Self-identifies via "Thank you, Dan" from Cowen at close; runs the show, screenshares, reads audience questions | **Note:** No curated metadata was supplied; all role identification derives from in-transcript statements. ## Referenced Sources & Third-Party Claims (as attributed in transcript) - **Books by Cowen** (authored, first-party): *Average Is Over* (2013), *The Age of the Infovore*, *Talent*, *GOAT: Who Is the Greatest Economist of All Time* (published online at **econgoat.ai**) - **Publications/Platforms referenced:** Bloomberg columns (Cowen writes for), Marginal Revolution blog, JSTOR, scholar.google, Reddit, Amazon, Twitter/X, Zoom, Descript - **AI tools discussed:** ChatGPT/GPT-4, Claude/Anthropic, Gemini, Perplexity AI, DALL-E, Midjourney (cited as small team ~7–8 people per "I read" — **secondhand claim, unverified**) - **Named third parties:** Alex Tabarrok (co-host of a planned insurance podcast), Patrick McKenzie, Jordan Schneider, Henrik, James, Daniel, @thisisgray, Andrew Mason (former Groupon CEO), Lazarus Lake (ultramarathon figure), Robert Wiebe (*The Search for Order*), Richard Jones (historian), Milton Friedman, Jonathan Swift, Emperor Aurelian, John Chrysostom, Jeff Holmes (co-creator of the GOAT chatbot) - **Quote from *Average Is Over*** read aloud by host (source attribution to Cowen's 2013 book) - **Cowen claims:** AI-read audiobook of *GOAT* "sounds remarkably like me"; prior law-student class required GPT-assisted papers; current class requires $20/month GPT-4 subscription ## Audience-Submitted Questions - Questions sourced from Twitter (host states he asked Twitter pre-recording); askers named above. **Comment availability not stated** in transcript or metadata. ## Promotional Segments — Flagged - **econgoat.ai** — Cowen promotes his online book and its RAG-based chatbot multiple times; host says "I would love to link to that." - **Marginal Revolution** and **Conversations with Tyler** podcast referenced repeatedly in intro and body. - Host mentions writing an article "about this for Friday" and the current show's format — self-referential, not third-party promotion. - **Perplexity AI** — extended discussion of its capabilities and funding round; informational, not flagged as sponsored (no sponsorship disclosed in source). ## Uncertainty & Gaps - Host's identity confirmed only via closing "Thank you, Dan"; no surname given. - No timestamps, episode date, or channel metadata available. - Midjourney headcount is an unverified secondhand claim ("I read"). If ChatGPT stopped existing today, how would that affect your productivity? I would feel much less smart. AI's for learning, Perplexity is for references, Google is for links. I was in Tokyo, I use it as my universal translator. Paraguayan food. I was in Honduras, mostly when I travel, but AI adds flavor to the trip itself. I actually created a clone of you. I want to do a segment with you called the Tyler test. I ask you a question, I'll have you answer it, and then I will also ask the clone that I made the same question. We'll see if it the question in the way that you would. What are the core lessons of economics? The first would be incentives matter. The second would be there's always an opportunity cost. Tyler, welcome to the show. Happy to be here. Thank you for having me on. Of course. I'm really excited to do this. For people that don't know you, you're an economist at George Mason University. You're a prolific writer. You've written, I think, 17 books, and you've been writing your blog Marginal Revolution for almost or over 20 years, and I find you to be a just incredibly deep thinker. You think a lot about the impact of technology on life, work, and the economy, and I think it's all incredibly relevant to to the show, and I'm just, yeah, super excited to have you. Happy to be here. Great. Cool. So, what we tend to do on this show is talk very practically about how smart people use ChatGPT and AI in their lives to get work done. But, I think before we get to that, there are some sort of high-level questions I want to ask you about your work and your view on AI and its impact on technology and and jobs and the economy. Um, so, I think let's let's start there. Um, and in particular, I think there are a lot of people right now who are trying to figure out what AI is and how it's going to affect their work lives and the economy more generally. And I think you've been thinking about that question about maybe AI in particular for for a little while, but like generally about how technology affects our work lives for a really long time. Um and in your book Average Is Over, which you wrote in 2013, you talked about this like stratification in the economy that's driven by what you call what you called intelligent machines. And this is before like real AI had had even sort of come out. Um Well, in chess we had real AI in 2013. true. I I guess uh uh you know, before let's say just before this generation of consumer AI had come out or was even like on the horizon. Um and one of the things that you wrote is that if you and your skills are a complement to the computer, your wage and labor market prospects are likely to be cheery. Um and if your skills do not complement the computer, you may want to address that mismatch. So, you talked about this sort of stratification between um intelligent technical knowledge workers and the rest of the economy. Um And and and you saw that gap and you saw that widening in 20 in 2013. And I'm just kind of curious for you to talk about that prediction, like why you wrote that, what you saw then, and what you think now given the current generation of AI models. Well, for a long time, even well before 2013, I thought artificial intelligence was the most likely place for the next major technological breakthrough to come. And I'm not an AI expert in any technical sense, but my intuition there was pretty simple. I was quite an avid chess player when I was very young. And at the time people thought, well, AI can't really play chess. Chess is too conceptual, too complex, too non-legible. And pretty quickly, as I think you all know, AI managed to overcome all those hurdles, and it's now almost godlike when it plays chess. Right. So, I thought if AI can do that, chess is really hard. AI can do all these other things in time, and I was convinced this would happen. And it's happening, essentially. So, that's the background of the book, and I wanted to be ahead of the trend and write about where I thought it was all going. And um and I guess in in your So, I think you're I think it's clear that you were you were ahead of the trend. Um and and I guess you you made some predictions about uh what how that would impact the economy. Like, do you feel like that was that was spot-on? Do you Are there any revisions that you want to make given what you you know, I think it's probably at least somewhat surprising the the these the exact specifics of this current generation of generative models. Um and I'm sort of curious how you reflect on that. What's happening now is a work in progress. I don't feel that we know yet which groups of people it will help the most and harm the most. Like many people, I didn't expect that AI the AI breakthrough would have so much facility with words and with emotions and with bedside manner. I don't mean that I ruled out the possibility, uh but back then I would have been surprised if I was told, "Well, those are some of the things that we'll be best at." So, I thought it would be a bit more of an autonomous reasoning tool Mhm. in a way different than what it is, maybe more along the visions of symbolic AI people. Mhm. Uh but it is incredibly facile at taking some idea and writing a rap song around it or a poem or a shanty song or whatever you might want. Uh and that is different. I think uh what are sometimes called the word cell classes may be under greater threat from AI than I would have predicted. Mhm. Uh but again, this hasn't all quite come to fruition yet. We're waiting, and I expect many surprises. Got it. So, so it's it sounds like you're not you're not really sure, and you don't and you don't necessarily want to like hazard a guess cuz it's too early to say. No, I'm very willing to hazard a guess, but I want the the uncertainty brackets to be understood. Okay. So my best guess is this, in the short run it will be quite egalitarian. Mhm. As people who can't do things well at all now can do them pretty capably. Mhm. Say like writing an online essay to get into college. The smartest kids already could do that well. GPT doesn't help them that much. And they were going to work hard on it anyway. Uh but I suspect the longer run effects will be less egalitarian. That as people learn how to use these things and they get better. That more more people will use AI to sort of help them build out projects. So the people who can start projects will be the major beneficiaries. They'll have better record keepers and translators and mathematicians and coaches and colleagues and advice givers. But large language models, they're not a source of actually doing the idea. You can use them to get ideas, but they don't make the decision to go out and do the idea. So some kind of hyped up executive function maybe is what will be rewarded say over a 10-year horizon. Yeah. And that could be inegalitarian because it's talented people who have a lot of project ideas anyway who might benefit the most. That's my best guess. I think that makes sense and and so to to sort of summarize what I what I heard from you is right now what it can do is it can give people skills to do tasks that they wouldn't ordinarily not have been able to do at all. So like maybe you're not really a programmer and it can help you build a build a very very simple version of an app or maybe you couldn't really write a college essay, but it can it can it can do that for you. So it can bring the sort of the bottom quartile of of skilled of skills up. Um and then of right? Most people cannot write a college essay. Most people can't finish college. Okay. Right. So So it can it can it can augment those people's skills to let them do something that they couldn't do before. And then for for other people who are who are more skilled in a particular domain, let's say a highly skilled programmer, it makes them slightly better but not it's not going to necessarily make them do things that they could never possibly do before. And that's the that's the immediate effect is what you're saying. Yes. Right. And then the And then the sort of secondary effect um is something like uh it enables people uh to who who want to make things in the world to like go to go and make those things with the with the intelligence that it that it uh that it unlocks. So like one of the thing one of the ways I've been talking about it is is like the allocation economy. It's like people who know how to allocate resources. Um so those are people who have like managerial skills or investing skills um are going to be able to deploy those much more quickly and cheaply by using AI instead of using uh like building a human organization. That's right. Yeah. I think there's another longer-run effect that is not at all in place now, but at some point will be. And that is when we can use AI to evaluate people's talents. Mhm. Uh and that will be highly meritocratic in a narrow way. I'm not sure we'll all be happy with it. Mhm. To be told just how good you are at something is perhaps unsettling. Yeah. And on net diminishes human happiness. Uh but we will find potentially successful people much more easily at some point that is not in place now. I definitely find that it's it's really quite good for helping me understand who I am. Like giving me a picture of who I am. So like I'll I'll often do things like I'll uh take long journal entries and I'll throw them into ChatGPT or into Claude, and I'll say like, "Hey, what are the psychological patterns that you're noticing here?" And it's really good at like taking them out and putting them into words for me. And I think having that reflection, if you ask for it, is really powerful. You know, I'm about to turn 62. So, that's maybe of lower value to me, but I agree with your point Yeah. about large language models. They're wonderful therapists and analysts, and they can be remarkably objective when that's what you ask for. Yeah. Yeah. I want to just This is This is a sort of This is a sort of slight tangent, but I want to just dig in on the sort of allocation economy idea that we've been talking about cuz I've I'm writing an article about this for Friday, actually. And one of the one of the I think interesting implications of that is right now, the skills of of being a manager um are not that widely distributed or talked about because there's only a small group of people who are managers. Um but I think those skills may need to be um more widely distributed in the near future as everybody is moving up a layer of abstraction. So, they're even junior employees, for example, are thinking about things like For example, a really common management problem is like, "How far into the details should I go?" Um like I asked someone to do something. Do I micromanage them, or do I let them go do it, but then comes back wrong. Like, that's a that's a specific skill that managers have to learn that junior employees don't have to learn that I think might be valuable in in an economy like this. Have Have you thought um more about like, "What are the specific if you're uh if we're thinking about allocation or allocating resources as like a as a core skill of the next generation, have you thought about the like specific ways or skills that are that are bundled under there that are going to be important?" That's a very good point. Also, if you look at even the top management consulting firms, the kind of advice they give to many people, it seems rather anodyne and boring and repetitive and clichéd. And that's a sign GPT models can do it really well. Yeah. And I think they can. So, that might just end up automated and you might still need the consulting firm to make it stick or to be the focal voice putting the message forward. Yeah. It's the actual amount of labor you need doing that. So, it's like everyone will have free McKinsey-level advice or nearly free. But, another thing I worry about, as you know, there's a lot of talk about AI and bio risk, what about developing dangerous pathogens? I think it's a big constraint on terror organizations that they're very poorly run. Mhm. That's a much bigger problem than, oh, they can't find the optimal pathogen to do us all in. Right. They just don't succeed or even try very often. So, if they have access to AI is giving them better management advice, I think that's actually one of the biggest risks of AI technology. That is fascinating. pathogen, just the here's how you run a successful terror group. And maybe current models are sufficiently protective that they won't tell you, though I even there you can probably break through the dam, but there'll be very good AIs that will tell you that pretty soon, right? Yeah, I think that that's such an interesting underappreciated risk because it yeah, I think anyone that's thinking about AI risk is really interested in the hard problem of like here's how you assemble chemicals in a specific way to like make a bomb. Um but the the human coordination problem is actually like a significant deterrent for people doing bad things. Um and human coordination yeah. Yeah. And human coordination is like one of the things that I think AI is going to be really good at. Um and and we see that people talking about that already in in the in terms of business. Like you even yourself wrote um you wrote an article recently about ChatGPT and and and your career trajectory. And what you what you said is that um small integrated teams will produce the next influential big thing. Um which I think you meant um you can do a lot more with AI with a small team. And I think that's a really positive thing, but it but the the the negative is that you can do a lot more harm. That's right. A lot of companies might become much smaller, but still rather potent. Yeah. I think Midjourney, when it first had its breakthrough, had what, seven or eight people working there, I read? Uh maybe more than that, but it's not going to be so many more. Yeah. It's very small. Um I'm kind of curious like on on that on that point, um one of the things that you've written a lot about um and you you wrote about this in in in a previous book um called uh The Age of the Infovore, is um there are certain skills that are super valuable in the knowledge economy, um specifically like ordering knowledge um and also like being very attuned to, being able to remember and use small bits of information within a specific subject area, that you associate with people who are autistic. Um and and and that book in particular talks a lot about um why those traits are actually quite valuable uh in this current era. And one of the things that strikes me is like that's also a lot of the things that like ChatGPT or or other AI tools are are fairly good at. And I'm curious how that changes or what that does to your perspective to like to look at it that way. I think in the short to medium term, AI accelerates the value of those skills. Mhm. Ordering information, grasping how things fit together, knowing a lot. Because you now have this extra way of learning. But there could be some point much further out where the AI simply does all the work and the value of that skill becomes quite low. And skills like charisma and executive function are what rise in importance. Yeah. But I think to get to that point, we would need AIs really incorporated into major workflows and systems in a way that's fairly distant. So I don't think it's oh, just a few more years and then this is going to flip. I think for the foreseeable future, it will help the info wars. GPT models. And so I I I assume you you you include yourself in that. Sure, of course. Yeah. Are you thinking for yourself like about upping your your charisma level or you feel confident enough that it's it's far enough out that you don't need to really worry about that or maybe you feel charismatic enough you don't need it. Well, that's for the world to judge. I will say I've decided to do more personal appearances both as a way of projecting but also as a way of learning things that I can't learn from you know, say a large language model. So it's definitely influenced my behavior already. Okay, cool. Uh I'd love to start to get into some of the practical aspects of this um which is yeah, more specifically how how you use ChatGPT. And before we like you you're you're you're already in in a screen share. Um but before we we dive into it, I I'm kind of curious like how would you summarize at a high level the way that it fits into your life and your work? There are two quite distinct ways I use ChatGPT. One is on iPhone and the other's on my laptop and they're totally different. So I don't know if you want to cover both or take them in sequence. But the the iPhone is pretty simple. Do you want to start with that? Let's do it. So if I'm in a foreign country, I was in Tokyo, I don't speak Japanese, not many people there speak English, I use it as my universal translator. Mhm. It will get better than that, but it's fine already. Yeah. It's not Star Trek great, but it's amazing. And the other thing I do is I use it to read menus. Mhm. Or even just tell me what should I order. Yeah. So I was in Buenos Aires. I was at a Paraguayan restaurant. Paraguay is one of the countries I've never been to. So I'm an idiot when it comes to Paraguayan food. I took a photo of the menu asking, well, GPT, what should I order here? What What are the classic dishes? And it tells me. Now that's amazing. I was in Honduras a few days ago. You see a plant, you see a bird you don't know. If you can take a photo of it, you take the photo, you ask, "What's this?" And it tells you. Right. So I use it for that mostly when I travel. Again, that's distinct from my main uses, but I'm quite sure it will be enduring. What do you think that that opens up for you? Like what is that What is that like to be able to take a photo of a bird or take a photo of a menu and just like know what the menu says or know what the bird is? Well, you just learn something, right? But a big part of the value is the interactive experience with the other person you're with. That you in some way are discovering this together makes it more memorable. And I think you're more likely to do it with another person or people than if you're alone. So it adds flavor to the trip itself. Like you have these now multiple modes of how you discover things. That's really interesting. It reminds me um I don't know if you know Andrew Mason. He's the former He's the former CEO of Groupon and he runs Descript, the the podcasting software company. I might have met him once, but I don't know him. Yeah. He used to run this company that was it did like audio tours of cities. And so what you would do is and they were all I think created by the company and then eventually created by like third-party tour guides where where wherever you were like I did this in Rome, you could walk around and it turned the whole city into an audio tour sort of similar to like the one that you'd go on in a museum, but it was the entire city. Um and you could pick one that's like history or food or architecture or whatever and you would walk to a specific spot and then it would say like look to your left and this is a building and here's everything about the building. And it that company didn't work, but it was a it was a pretty magical product experience and it sort of strikes me that in some ways ChatGPT is that without needing it to be a a separate company. Yes, and if you're sort of on the road out there and there's some fact you want to know, like oh what's the population of the city? Uh it's easier and quicker than Google or Wikipedia. Maybe it's a small edge. But for for something as important as that, I'll take a small edge. Yeah. I just ask it. Right. What if you like this this actually reminds me of before we recorded I I I asked Twitter for uh questions for people that people wanted to ask you and um Patrick McKenzie who you interviewed on your podcast recently uh asked me to ask you how do you check the output for correctness workflow-wise and where do you feel comfortable not doing this? And I think that's let's kind of related to this example cuz like particularly in a in a country like Paraguay or Japan where you don't speak the language, um it's you're it's like hard to actually check the output. So like do you have any specific rules or heuristics for like when you when you trust the output and um when you don't? I worry about hallucinations a lot less than most LLM users, I think. So a lot of my questions, oh what bird is this? Let's say it's wrong and it's just pulling my leg. Okay, like I didn't know what bird it was and now I think it's the wrong bird. Like, who cares? I'll just ask it again and move on. Uh but pretty often I'll just ask it, are you sure that's the right answer? Or please correct any hallucination. And most of the time, if there's a problem, it does correct it. Not all of the time. I'd guestimate 80 to 90%. If you just ask it again, it'll correct an error. Yeah. So, that's one layer of check. Uh I think also the areas where I tend to use it. This is now laptop GPT. It just makes way fewer errors than the areas where a lot of other people tend to use it. So, uh that also limits the degree of hallucination. How do you characterize the areas where it makes fewer errors? I couldn't give you a general answer, but I can tell you I'm typically using it to learn obscure history. And obscure history, maybe by the nature of the questions, you're sort of pointing it in the direction of an intelligent part of the information space that's not too crazy, not too controversial. If you ask it, you know, about the old Byzantine Empire, it does pretty well. I did a whole podcast where I made believe I was talking with Jonathan Swift, you know, the the Irish writer 17th and 18th century. I asked it a series of detailed probing questions for an hour. This has been put online. Uh it didn't make any mistakes. Not one. And I There were no redos or no trial runs. I just did it. And what I did on the first try was the final output. So, presumably it's read a lot of Swift, read a lot about Swift. It's obscure and detailed enough. Maybe it just ends up boxed into the corners of truth a lot. So, for me, uh you know, I know how you can get it to hallucinate. If you just ask it, "What are the three best books to read on Jonathan Swift?" My best guess is one of the three answers won't exist. I understand that. But I don't run into that so often cuz I'm not asking it that question. Right. That's really interesting. Um I definitely have found that, too. I haven't used it for obscure history, but I use it a lot actually to read old books. So, um one example, I was reading Moby Dick recently, and it's really helpful to just take a picture of a passage, ask it to explain what's going on, or even to like visualize the passage where you're just like, "Hey, can you can you show me what this looks like with Dolly?" Um and I think the reason why it's really good for that is uh it has access to all the text. It's read all the text, and there's a lot of supporting there's many decades of supporting commentary, so it knows what to say. Um whereas for more recent stuff, it's more likely to be copyrighted and and less likely for it to know have read it. Another thing I use it for a lot where hallucinations are not a risk. So, I have my own podcast, Conversations with Tyler. I have guests on. I want to learn the background context to what they do. Yeah. So, I had one interview with Lazarus Lake. He runs ultramarathons. He hasn't really written anything. There's not that much written on ultramarathons that's readily available. But if you keep on asking the same questions about ultramarathons, you acquire all this background context. Maybe some of it's wrong. It's not really going to matter. I'm the one asking questions, not answering them. And I just feel I know my way around the topic pretty well. Yeah. Uh I'm learning now about the insurance industry. We can talk about that more. Which actually the published literature on economics of insurance and its history, it's very bad and sparse for whatever reason. If I keep on asking GPT, again, I'm not convinced the answers are perfect, but I know I'm doing better than I would do any other way. Right. And I'm not the one required to know the exact fact. Mhm. What I want is context. That makes sense. Yeah. I guess uh when you're when you're using it, for example, in an in the insurance area, and you say you're not the one that is required to know the exact fact, like don't you get sort of nervous that you're going to have something in your head that's like kind of slightly wrong? Well, I already do, right? That's the problem. And I could show you one of the things I asked it if you want to look at something concrete. Yeah. I want to sort of set the stage here really quick. So, we're going into your use of laptop GPT. Yes. And one thing that I think is really important to note is you're not in ChatGPT, you're in the playground. Um and so, before we we head into a specific chat, why do you use Why do you use the playground? Tell us. I don't have any good answer. I mean, I'm I'm stupid, basically. I like the fact that it doesn't look like a consumer product. Uh-huh. And it works for me, and the maximum length I can slide across, make it longer, shorter. Maybe it doesn't even matter. Yeah. But I just enjoy that, and I'm used to the look. Yeah. So, I use it. You think it's a mistake for me to use it? I actually don't think it's a mistake. I mean, you're you're sacrificing certain things, right? So, like uh you don't have access to code interpreter, you don't have access to DALL-E. Uh there's no unified experience between mobile and your desktop. Um so, there's a bunch of things that are not as good about the playground, but one of the things that is going to be better about the playground is you have a lot more control over the output. So, like you can see, right right on your screen, you can see that you can set the system message. So, this is message, for people who don't know, is going to set the um kind of like the personality of how GPT interacts with you. And and and setting that can be really really helpful. On the right, you can also see you can set the the model, you can set the temperature. So, temperature is is sort of like creativity for the model. Um how likely it is to be factual, how likely it is to sort of like make stuff up. Um you can set the max length, all that kind of stuff. So, you have a lot more control. And then another thing that I think is probably true of the playground that's not true of ChatGPT, is the content restrictions are going to be looser in the playground because it's built for developers and not for consumers. And so, they're going to be less careful about what the model says cuz it's only being used by a certain group of people who are using to make make products instead of like hundreds of millions of people who are using it every day for random stuff. So, I'm not stupid. That's nice to hear. I mean, I have DALL-E and all the rest. I just don't use them much. Uh the images are not my main interest. I think it's amazing. Uh but I'm not sitting there playing around with DALL-E. I don't think it's a waste of my time. Yeah. Um okay, so so tell us so let's let's go into into the first chat. You you have a chat right here. I think you're I think it's about insurance. Tell us about like um tell us about how you did this chat. What What was the motivation behind it and then and how you started it? Well, Alex Tabarrok and I agreed that we would be recording a podcast about the history of insurance and insurance economics. Mhm. And I'm really not well informed about that area in part because the literature is not that good. I do read a lot. So, one question I had was about this difference between whole term and regular life insurance. Uh the whole product is like a savings investment bundled with the insurance itself. And the other is just the plain old life insurance. So, that distinction between the two kinds of insurance, when did that arise? I don't know. Mhm. Probably I still don't know, but I thought I would ask GPT. And you can see my question on the screen. I can scroll down and show you the answer. Yeah. So, to read the question, you said, "When did the distinction between the whole term between whole term and plain life insurance arise historically?" Right. And it looks like GPT said the distinction between whole whole life and term life insurance evolved over time with the development of the life insurance industry. Term life is designed to provide coverage for specific period with no savings. So, we're getting some definitions. Whole life insurance uh offering a death benefit along with the savings component allowing the policy to accumulate a cash value became more common in the 19th century as the industry matured. Um so, yeah, what did you think of that answer? Like, did it did it give you what you wanted? Well, I don't know if it's true. Uh so much of the insurance industry matured in the United States after the Civil War. Mhm. So, if that distinction came at that time, it's at least plausibly the case. Uh and then it says, "Well, the distinction became clearer in the 20th century as life insurance markets became more diverse." Yeah. Uh I thought it was a good answer. I hope it's a good answer. I'm not going to flat out assert that in a dialogue, but if it had said, "Oh, that was there from the beginning in the 17th century." Or it came only in 1964, Yeah. then I would definitely look more into it and try to learn something quite specific. Mhm. But right now it's just telling me it's following some general patterns I might have expected anyway. Mhm. That's useful information. Mhm. I probably won't use it directly. Mhm. Uh but then I go on to my next question. Can you see that now on the screen? it. Yeah. So, you said, "Why would anyone want life insurance bundled with the savings return? Doesn't it mean inferior returns?" Tell me tell me about that. Why'd you ask that question? Well, it's a common piece of advice for Americans investing that you should not invest through your life insurance policy, that you can do better with a direct purchase of equities through a low-cost diversified mutual fund. Right. It's pretty standard advice. So, I'm asking GPT, why would anyone want to save through their life insurance? Mhm. Which is a known question. So, it gives eight reasons. Uh a few might have I hadn't thought of at all. One or two may not be good reasons, but they're probably actual reasons for particular human beings. Yeah. And it's a good answer. What can I say? I learned something. I don't think it screwed up. That was my answer. Right. Yeah, how do you think um how would you summarize the kinds of answers? It seems like there's there's some things you would have thought of, some things you wouldn't have thought of, and some things you think are not that great. Like, what is the what is the value of that for you? It spurs my thoughts. I get additional bits of information. Mhm. Uh I one thing I didn't do, but what I would then continue to do is ask it a question like, well, if someone were writing a criticism of whole term life insurance, what points against it would they make? Yeah. And my guess is that answer will be quite strong. Mhm. Do you have any feel I need to ask it. Partly I know, but uh that would be the sort of follow-up. You want to try it? You think we could try it? Sure, let's try it. We can talk about what's in between also. Yeah, we can go back up in in a sec, but I'm sort of curious about that. If you were to write a critique of why individ uals should not buy whole term life insurance, which points would you stress? Please answer like a sophisticated investment advisor. Those add-ons you don't need as much as you used to I find, but they're not going to hurt. Interesting. Tell me about Tell me about that. Tell me about asking it to sort of simulate a sophisticated investment advisor. Again, you're pointing it to a more intelligent part of the information space. So, if I ask it a question, say from economics, what is inflation? The answer is not wrong, but it's not really better than Wikipedia. Cuz the question's too general. If I ask it a question, what is inflation answer as would Milton Friedman? You get a better answer usually. It's not nothing to do with agreeing with Milton Friedman or not, though. I mean, you might. I guess I do. Uh you're just again pointing it towards smarter bits, you know, in the matrices somehow. Right. And this what this this stuff it knows connected with Milton Friedman is smarter than the stuff not connected to Friedman. Right. So, as you get more specific, you're going to get more specific detailed responses, which is what you want. That's right. And it loves compare and contrast. Even if you don't want to compare and contrast, you often get the best data by asking it to compare and contrast because it's like crossing two swords, and maybe something about the order of regressions. They just get better. Interesting. Uh so, here's the critique. There's some general blah blah blah at the top. Uh cost efficiency. Whole term life insurance has higher premium. Uh investment flexibility. It's not as flexible. Less transparency, less control. That's true. You're less liquid, it mentions. That's number four. Uh five, it's comparing the tax considerations to a Roth IRA. Excellent point. I hadn't explicitly thought of that. Uh the whole term can have more financial complexity. Higher opportunity cost, which is the key main point, could be invested elsewhere. I'd like to see it mention equities there, but okay, it didn't. Risk diversification, the insurance company may not be optimally diversified. That's also true. You know, on any exam, I'd give this answer an A. Maybe not A plus, but a solid A. Right. It's all correct, nothing wrong with it. It did great. One thing that we might want to try, which could be could be interesting, is if you delete this specific answer, um Yeah. which I think you can you can do pretty easily. Um You'll have to tell me, cuz like I said before, I'm stupid. Yeah, yeah, scroll over it, click into the assistant answer, and there should be a minus button there. Learn something new every day. Yeah, I see the minus button. There you go. Okay, so I got rid of it. Yeah. So, what I want you to do now, so I think it'd be fun to play with the temperature. So, right now the temperature is one. Um but if you move it up, it'll get more creative. Yeah. Um it might be interesting to see if you go all the way all the way out, what does it do? Ooh, yeah. This this is something they're going to ban on Twitter. We'll see. Let's try. It'll become grok, maybe. See, there you go. is it's too creative, right? Too creative. Um but I think that's that's an interesting reflection. You can you can try it again with a slightly less Yeah, slightly less temperature, like maybe midway. Um and if you just uh click the minus button again. Yeah, there you go. Minus? Yep. And then Ask again. Uh press uh press cancel. See where it says cancel? Yeah. Yeah, and then submit. Submit. Opportunity cost comes first now, which is I would say better. Mm, interesting. So, you think it's maybe slightly better so far? Slightly better. It's the same points. Yeah. Uh but in a better prioritized order. And then it went off the rails. nonsense. That was off the rails. But that's fine. Yeah, I I think that that's I think that's a really interesting thing about using the playground is you get to do this and you get to see these are some of the rough edges that like they don't really want you to see in a in a consumer app. Um but it does allow you to sort of explore the space of possibilities. Yeah. And the the upside The downside of exploring the put the space of possibilities is that you sometimes get junk. But the the upside is you'll might you might find stuff that no one has found before. Um and in this case it like before it went off the rails it actually gave you a slightly better answer and I think that's an interesting trade-off for people to consider when they're thinking about when to use ChatGPT or when to use something like the playground. If I used it for poetry, I would use the higher temperatures more. Right. I've experimented only a bit, but I'm actually pretty fine with it at one. Right. Yeah, that seems that seems like it makes sense for you. Um I'm curious like Yeah. obliterate this. We can talk about some of the other queries in here. it. Yeah. Okay, so uh here's one where I couldn't quite pull the teeth out. So I was reading a book about the the Byzantine Empire. Mhm. Area I don't know much about. Mhm. And it mentioned inflation under Emperor Aurelian, which is early part is still Roman Empire times before it splits. And I said, "Okay, well, how high was that inflation?" I wanted to know. I really wanted to know. I didn't just do it for the show. And I asked it and it understood the the query very well and it told me all about the limited data. Mhm. It may just be the best answer. Maybe we don't know. We just know of texts where people complain inflation was high. Mhm. It mentions historian Richard Jones. Mhm. I have an imperfect recollection that he says the right name to say, so that's helpful. I recognize it but wouldn't have thought of it. So, I could put that into scholar.google. Mhm. But then at the end it says quantifying it with any exact figure da da da da da you know, it's like Right. falling asleep on the job. Right. Not willing to commit. So, what do I ask if you had to guess a specific number? Maybe I should now be demanding my money back, right? Sometimes that works. Now, it just goes into full evasive mode. As an AI developed right, don't even need to talk through that. Everyone listening knows the deal. Right. Right. Then more blah blah blah. Basically, it should have just said I'm not going to tell you. Mhm. Uh but here finally some scholarly estimates say it might have been over a thousand percent over the course of half a century. Mhm. Now, I don't believe that. I think that has a decent chance of being a hallucination. Mhm. Uh but it told me something. And it's like some estimates we're not sure. Like I know we're not sure. So, the hallucination may not be worse than the hallucinations of you know, my fellow economic historians. That's okay. I'll live with that. If it just said, oh we think it was 30%. Uh maybe I wouldn't trust that, but it said something. Okay. This is this actually reminds me of of of a really kind of important point going back to you know, talking about the skills of managers being more important in this in a in this new world. Um so like one of the skills of a manager that I think you're you are picking up on and you already have cuz you do this I think probably with research assistants is you have a sense for when you get a fact like that, 1,000% it's probably not right. There are some things in that answer that are probably right and some things that are not right. And I think there are a lot of people who are evaluating GPT ChatGPT that see that and they're immediately like, well, I can't trust that. I'm not going to use it. It's not useful, right? But what managers of people know how to do generally is to take an answer from someone and know which details are like maybe they need to follow up on and maybe the ones that they they don't. Um and that is a skill that you are deploying here to hold lightly to the things that that uh that don't that sort of trigger your sense of I should probably check that and then maybe I don't know if you did this in this chat, maybe like go deeper on it or maybe switch to another tool to like follow up and that's an example of I think a skill that managers have to know that junior employees don't generally have to as much. That's right. Now, I might try Perplexity AI at this point. Which is turning out to be great. Do you use it? Yeah. Under Aurelian, what was the rate of price inflation? And I'm clicking. And I'll tell you the answer. We go here. We go into Perplexity. There you go. Some sources say 3% a year, others say 5% a year. Debasement of the currency. Now, one of the great things about Perplexity is that it gives you citations. Mhm. And I would track this down. Mhm. And uh one is Reddit and the other's Jstor. I I would look at Jstor. Mhm. Most of all, I know who it is I should ask and if I needed to know, I would do that. Right. That's my process would proceed. I would next try Perplexity rather than torturing GPT anymore cuz I know sometimes when you torture it uh it's like no, the dog just doesn't want to go for a walk, right? Right. Right. So, let's go back to the playground. So, like let let me let me just let's unpack that. Let's stick with that for like a sec. So, it sounds like the stack for information gathering is something like Chat GPT then Perplexity then maybe the sources from Perplexity then people. Does that sound about right? Often. Now, if I need to know something quite particular to say finish a piece of work, Mhm. I might start with the person, of course. Mhm. Mhm. And there's an advantage to starting with the person. The person may be delayed in responding, but you know the other things won't be. So, that which will delay you ask first to cut down on the length of the delay. Right. There's something really logically make sense. Right. There's something really important missing from that list, which is Google. When do you use Google? I use Google when I need links, which is quite often. So, I still use Google a lot, Okay. but I use Google less and less for information. Mhm. But when I write a column for Bloomberg or do a blog post, I need a link. Mhm. For getting the clickable link, Google is still clearly best by far. Got it. But not for learning things. Not for learning things. Now, the link may lead you to what you want to learn. I get that. Uh, but Google is for links. Google is for links. AI's for learning. AI's for learning. Perplexity is for references and sometimes has context that GPT doesn't because it's looking to tie it to references. Mhm. And why do you think you start with Chat GPT versus starting with Perplexity? It depends on the query. If it's something recent and I want to know the citation above all else, I might start with Perplexity. Mhm. But I've only been using Perplexity a few weeks. Uh and just my patterns with it will still evolve. Right. So, it's a little less familiar. It's pretty new. It's really having a moment right now. Like, it's They raised a They raised a big round and and a lot of the guests on the show have been using it. I've started using it more. My dad's using it, which is actually like usually a a big sign for me. Um he was using Zoom like years before anyone else. Um and so, he he just He's sort of like the canary in the coal mine for me of new new interesting software. Um so, uh going going mainstream. So, uh yeah, I'm I've I've I'm really interested in following their trajectory. Yeah, someone should try to buy them if they can. Yeah. Um Now, here's a question where I got hallucinations. Do you want to do one of these? let's do that. So, on Twitter, Jordan Schneider asked me and here you can see on the screen, "What is the best book on the civil service reforms of the progressive era as related to the Pendleton Act?" Mhm. Some detail up further on the screen, but don't worry about that. Yeah. And I I asked GPT, but I knew that wasn't the way to go. I just thought, "Well, I'll do that as an experiment." Mhm. in part for this program. Mhm. And the first book it mentions Pendleton Act of 1883 by a Jonathan Grotsinger. It's not an Amazon. Uh I'm not sure it exists. Probably it doesn't exist. Uh-huh. It's sort of the book I would want. Mhm. But, it doesn't exist. Uh This other book, Reforming the Civil Service, I couldn't find. So, it's hallucinating more than usual. Mhm. Uh Then it somewhere, maybe it was the the query one up, it recommended Robert Wiebe. He had The Search for Order, which is quite a good book. Mhm. And it's a good book for me to recommend, not mainly on civil service. But, that was something it where it jogged my memory. Also, mostly it failed on that question. But again, I would know to begin with not to ask it that. It's asking for trouble, right? I've personally found it to be stunningly good for book recommendations. And in particular, what I what I like to do with it is have it help have it help me to identify my taste. So, I'll throw in a bunch of books and then be like, "What are the commonalities between these books?" And it'll be like, "Here's Here's the kind of stuff you like." And then I'll push it further and be like, "Okay, tell me more stuff that I haven't heard of." And I've found a number of books that I that I absolutely love from it. I use people for that, basically. And that works very well for me. That's also good. Yeah. Yeah, if you like a if you like a person liking their book recommendations, you're you're pretty you're pretty likely to like their book recommendations. Um well, let's move on. Do you have other chats you want to show me? Well, let me see what I have in here. So, I need to bring you back Yeah. to screen the playground. Oh. We did Aurelian. Yes, this was Again, I was reading the book on the Byzantine Empire. Mhm. And Chrysostom is mentioned. Mhm. And I know I've known about him, but it's rusty knowledge to me. Mhm. So, I just thought I would ask about him and get more context. Right. Who was he and why was he important? So, you're kind of Yeah, go for it. The answer seemed quite good to me. Mhm. Again, it's obscure history. Yeah. And it's working. So, you're kind of using it as almost like a second screen for reading. Exactly. Where you're reading a primary source and you want more details or insight into it and and you're just you're turning ChatGPT for that. That's right. Yeah. Now, another thing we'll use it for a bit. This is typically with my wife. We'll use it for the dog. Mhm. So, we have a dog who was given to us by our daughter who now has kids. So, we we haven't had the dog for long. We're not dog experts. Mhm. So, there's different questions about the dog. Like, will the dog get fleas? Or if the dog is limping, should we worry? Or Mhm. how much, you know, should the dog be eating? Or or everything about the dog. Yeah. And that's useful. I I think it it must know a lot about dogs. Yeah, I'm sure I'm sure it does. Um I I do that, too. I do that um not not necessarily I don't have a dog, but for like basic health questions and stuff like that. Sometimes it won't answer, but a lot of times it gives me like a general sense, which is nice. Um So, so far what we've done is we've covered I think we've covered a decent amount about how you read with it. So, reading with it as a second screen. I think we've covered a little bit about how you write with it. So, doing a little bit of extra research on topics that you might not be as familiar with or sort of getting the lay of the land on topics. Um I'm curious if there's anything else writing-wise that you've started to use it for um or if that that's really the extent of it. That's most of it. I think just as a pure writer, Anthropic is better. Mhm. But, I don't use it to write anything I do. I'm really very very careful not to do that. Mhm. Mhm. Uh I just think it's a bad habit and it's in many cases unethical or even illegal. Mhm. Mhm. Uh but, I don't do it. Don't want to do it. But, if I were doing it for whatever reason, I would use Anthropic. Got it. It sounds more natural and more human. That makes sense. Yeah, I I I find the same thing. I think I think um Claude is a bit better at capturing a specific tone um that you're going for if you if you need to use it for that. Um And then, I'm curious if there's any way in which you use it or don't use it in the classroom. Well, I'm teaching a class right now. We started last night and uh GPT-4 is on the reading list. Everyone in the class is required to pay the $20 a month to subscribe to it. Mhm. The class is history of economic thought. So, it's mostly a class in obscure history. Mhm. And I think it will uh work very well for the students, but the class just started. Now, I taught a class the year before to law students where I made them all write one of their three papers using GPT. Mhm. Not solely, but you and GPT together figure out how to write the paper. Uh and that was very successful. The papers generally were good. Mhm. People liked the experience. They learned like limits and scope of GPT. Yeah. And uh that's how I've used it in the classroom also. That's interesting. So, it seems like you're you're leaning into it versus banning it. Um Oh, of course. And I told the students last night, I was like, "Look, if you want to \{quote\} plagiarize with this thing, your paper still won't be great. Mhm. So, like don't do it and probably I'll know you did it. I know enough about this stuff, but just figure out some way to use it wisely and in a smart manner and you're going to learn things about it that I don't know and Yeah. just figure it out. That's what I told them. I said, "I'm not going to bust you for being on one side of some vague line or another. Just don't do something stupid you could ever get in trouble for and it should be fine. And I think that's going to work out. I'm not worried." That's really interesting. Have you found any any specific or interesting ways that students are using it that you wouldn't have predicted that are helpful for them? Well, foreign students, especially from China, use it to improve their English and it has a big big effect not just a marginal effect. And I don't think they even always know that maybe they're not supposed to or maybe it's allowed or maybe it's a gray area. Personally, I think it's fine. Universities deal with this in different ways. But I've seen people go from like bottom 10% of writing skill to top 10% overnight. This is a real risk, right? It becomes a crutch and you never learn a lot about English. Yeah. I I've I've seen the same thing. I think uh there are a lot a lot of business obviously is is conducted in written form and there are people that I work with or work for me who uh speak great English but writing it is a little bit challenging and you can sort of tell. But the minute ChatGPT came out, their writing turned perfect and it was so helpful, you know? And um it made me think about cuz I think that that's actually true in a lot of other areas where a lot of a lot of people on the people I'm talking about like they could write in English but it just you could tell it wasn't totally right. But I think that and and they just needed this like little subtle translation from their level of English to like natural fluent English. But I think that there's actually a lot of other of those kinds of translations that have to go on that GPT facilitates. So like a an example is uh my dad who who I brought up earlier, he's not he's not a particularly like technical guy um but he's has an idea for an app. And usually when he does that when he has an idea he'll go to me and say, "Hey, I need you to write this up for me so I can hire someone to like do it." Um but with GPT he can actually just like say his idea and it will write a product spec that looks like it's written by a product manager. And that kind of like translation, it's all in English. It's an English to English translation, but it's it's a translation from you know, his level of technical fluency to a product manager level technical fluency is like super helpful and important cuz previously he wouldn't have necessarily been able to do that. Um and I think people sort of miss that when they think about GPT's translation capabilities. Context is that which is scarce as I like to say. Mhm. Yeah. Um So Okay, so we we've gone through I I think that I think the classroom stuff is is really fascinating. Before this episode, we um we I tweeted about the fact that you're coming on the show and a bunch of people asked questions. So I want to do a little bit of a rapid-fire question asking of uh Twitter questions if that's okay. Sure. Cool. So Henrik asks, in the least possible number of dimensions, if ChatGPT stopped existing today and other LLMs, how would that affect your productivity? Would it be less posts on Marginal Revolution, worse interview questions, something else? Worse interview questions. I would just know less. I don't even know how to measure my productivity, right? Mhm. Maybe it wouldn't go down. I would feel much less smart. However, Mhm. maybe the stupider version of me is more productive. But much much less smart I think is a really important interesting thing to say. But the returns to being smart in labor markets, they're not as high as you might think. That's why there's a lot of uncertainty in my tone here. So it may just be private consumption being smart. Mhm. Uh I don't think say it would get me more people listening to the podcast. It may get me fewer. Right. Interesting. What do you think of as the most underrated or overrated use cases for ChatGPT? And James asked that question. I don't have a good sense of that. I'm not even sure like what is rated how. Mhm. I think the problems of hallucination are somewhat overrated. Mhm. Uh that I I do have a sense of. Things like programming I don't do. Uh it just seems to me almost everyone under uses it for most things. Mhm. Mhm. Yeah. That makes sense. Um Uh Daniel asked, which I think is relevant, how do you manage to switch between reading books and using ChatGPT to interact with the content while still maintaining your reading flow? Well, it slows you down quite a bit. Yeah. But you're still reading with GPT. You're just reading on a different screen. Mhm. Uh but you should do it in areas where you don't know a lot and really do need to learn something. Don't just do it all the time. Yeah. Yeah. That makes sense. Um @thisisgray asked, "Would you use AIs to speak for you online?" Uh I'm not sure I understand the question. Um let's say like would you use an AI at some point there will be text-to-speech and vision models that are good enough that you could have a little clone of yourself that might go on a podcast. I think is is the is the thrust of the question. Would you Would you allow a clone of yourself to go on a podcast? I'm not sure the choice will be up to me. Uh I'd like to see what it's like. I will say this, you know, I published a book online this last year called Goat, Who Is the Greatest Economist of All Time, Mhm. and the audio book version will be read uh by an AI, and it sounds remarkably like me, and I'm happy about that, not just fine with it, happy about it. The other stuff I'm not sure. Yeah. Why are you happy about it? Reading your entire book out loud isn't a very costly and taxing exercise, and this means I don't have to do it. That makes sense. That makes sense. Um okay. So, we are rounding toward the final part of the interview, uh which is uh it's apropos of this of the previous question, which is about cloning yourself. Because I actually created a little bit of a clone of you. Um and uh in this I want to do a segment with you um called the Tyler test, which sort of play on the Turing test, where I ask you a question um I'll have you answer it, and then I will also ask the clone that I made um the same question. We'll see if it answers the question in the way that you would, and we'll see how good it is. Um and just for a little background, the way that I made this is you can make a custom version of ChatGPT. You can give it a personality. I think one of the things that makes it likely to work for you is you've written so consistently for so many years that GPT has probably ingested a lot of your writing. And we'll see how it does on on some of some of the questions that I have for you. And uh and one important thing is like I there's no web browsing and I didn't upload any books or anything like that. So, it's only what's actually just in GPT, what it knows about you itself. One thing I would note, you know, my online book, Jeff Holmes and I created you know, a a rag-based GPT bot to answer questions about the book. It's not claiming to be me. Overall, I think it's pretty good. It's not perfect. It'll get better. We want to upgrade it, but I'm happy with it and glad I did it. That's cool. I would love to link to that. Um so, definitely send that to me afterwards. Um That's econgoat.ai is the website. econgoat.ai. Perfect. So, we are going to get started with this segment. It's called Tyler Bot. Welcome. My first question for you, what are the core lessons of economics? The first would be incentives matter. The second would be there's always an opportunity cost. Those would be the two most important principles of economic science and trying to apply them consistently would be the first mark of a good economist. Mhm. So, we we're asking Tyler Bot and we said, "What are the core lessons of economics?" Incentives matter. There's no such thing as a free lunch, which I I have heard you say before. Yeah, that's the same as opportunity cost. I you know, I would give it that answer as correct. Okay. Interesting. So, it's going beyond two. Yeah, the answer is very good. Very good. How would you rate it out of 10? I'm not sure what the scale is, uh but I'd give it an A. Okay. Um so, that's so that's number one. That's our first question. Um second question. And I got this from uh from your most recent book. much now. Like I get that it it feels it ought to give more. Mhm. Uh but there's something about economics where often less is more. I'm not going to take back the A. Uh but a good answer to that question is actually pretty short. Let's see. If you had to make it shorter, how would you do it? Make it short. Well, I would just pick the first few. Yeah. Well, let's see if it knows to do that because that's that's what you would do. Okay. So, the first two are now the same. Now we're and the third one is correct. So it's doing this well. And now it's just now it's just summarizing it in bullet points instead of so it's going off the rails. It's not quite it's not quite you but it's it did six. Yeah. I I'd give it good marks here. Okay. Okay, here's here's my next question which I I took from your book talent. What are 10 words your spouse would use to describe you? Curious. Works hard. Loves to travel. I'm not sure I should mention them all. Let's see. Maybe we can do five. So you're you're you're you're got four. Good father. Nice. And should walk the dog more. It's not going to get that last one. Okay, what are 10 words your spouse Oh, I say what are five words your spouse would use to describe you. Intellectual. Curious. We got curious. Analytical eccentric. Those are good answers. They're not worse than my answers. You know, you can't expect it to know about the dog. Right? That's why we have to ask chat GPT about the dog. A year from now we'll know everything about the dog. Well, you know, I like that it got it got curious. That's that's a that's that's the main overlap. But it it missed the travel and good father and it missed the dog. But maybe GPT-5 maybe GPT-5. Um. Maybe Tyler five, right? What what do you say? Is it Sarah or my error? Don't assume the error is in the GPT. That's true. We can't assume that. Okay, last question. Um in your view, what is the most underappreciated way in which AI, particularly tools like ChatGPT, will fundamentally alter our understanding of economics and human behavior in the next decade? I don't think in the next decade it will fundamentally alter our understanding of economics. Mhm. I think at some more distant point it will be possible to simulate small economies Mhm. using GPT-like methods. Mhm. And we might learn a lot from those simulations. Mhm. Uh those seem distant with with data collection as the constraint, not not a constraint on the AI side per se. Mhm. So, I don't think it will matter for quite a while. Mhm. Okay. Well, let's see let's see if GPT Tyler, GP Tyler, shares your shares your view. Huh. Do you agree with this? It's saying, "I think it will change radically transform our conception of information costs and decision-making processes." Those might be true claims, but I took you to be asking, "How will it change economic science and research?" And it's answering, "How will it change economies?" Mhm. Yeah. I think I I I I'm more meant it as how it will change the economy. Um but I think the I think the answer of how it will change economics is a good one. It doesn't sound like me, this answer. I don't think they're bad answers, Mhm. but it doesn't seem like a Tyler bot. It seems like it couldn't find what Tyler might think, and it came up with a a pretty good generic answer. Okay. All right. So, so So What what letter grade would you give that one? Well, for accuracy it's doing fine. For Tyler in this I'd give it a C minus. Okay, C minus. So we got an A and the middle one was Yeah, we had two excellent ones and then one that's a perfectly fine answer but has no Tyler in it. There you go. Well, that's that's the story. Um Uh be- before I let you go, anything that you wanted to cover or talk about that that we didn't get to? Well, I would just uh you know, to people that they read my book econgoat.ai. They can just put it on their Kindle, print it out, read it on screen, but it's also embedded in an AI in the form of a chatbot. And I made a deliberate decision to publish a whole book \{quote\} \{unquote\} in GPT-4. Mhm. And we're working on Claude and also Gemini versions of it. Wow. And I think this is the future for at least some book publishing. Like why not interact with the thing? It's an option. You don't have to do it. Mhm. Uh and I'm very excited to have been able to do that. What have you learned or what have you found so far having having done that that you didn't know before? Well, many people email me or on Twitter. They say what their questions are for the thing and you learn what your readers really care about. Mhm. And it's not what you want them to care about. And that's part of the experiment. So maybe books of the future will in fact use this data and give readers what they want what they really want to care about. I saw a question on Twitter this morning. Someone said they asked the chatbot, what would Tyler think that Milton Friedman would say about Ayn Rand? Which is not really a question about economics. Now the answer was good. It was both a good answer and a good Tyler answer. But you just learn people want something really quite subjective Mhm. out of even economics books. Mhm. And I think that will shape how we write and think over the next few decades. That's fascinating. I love it. Well, I think that's a that's a great place to leave it. Thank you so much for doing this. It was super fun to have you. My pleasure. Thank you, Dan.