Podcast interview with Alan Cowen, cofounder and CEO of Hume, an AI research lab building models that read emotion from faces and voices. Cowen, formerly of Google's affective computing research and a Berkeley PhD in computational psychology, discusses what emotions are, why traditional psychological theories of emotion fall short, how Hume models human emotion and accounts for individual differences, and how its API enables empathetic voice interfaces. The conversation covers applications in customer service, gaming, and therapy, Hume's goal of optimizing for human well-being, ethical concerns about emotion-reading AI, and the future of psychology as a science. # Channel Presentation Evidence — Ep 19 with Alan Cowen ## Observed Channel Identity (Self-Description) - **Show name stated in-transcript:** The host closes by asking viewers to "smash that like button and subscribe to **how do you use chat GPT**" — this is an explicit on-air channel self-identification. - **Host:** **Dan Shipper**, named in the closing segment ("a journey into the future with Dan Shipper as the captain of the spaceship") and referenced when the guest thanks him implicitly ("Dan I'm absolutely hopelessly in love with you"). - **Format:** Long-form (~1 hour range implied by depth) **one-on-one interview podcast** with AI industry founders/researchers, including a **live product demo at the episode opening** (the host talks to Hume's voice AI before the interview begins). ## Claimed Scope - The show covers **ChatGPT/AI usage and the AI industry**, per the channel name and outro framing ("knowledge bombs about chat GPT"). - The outro self-promotes as: > "every episode is a roller coaster of emotions insights and laughter… It's not just a show it's a journey into the future." - Note: this outro is delivered by the **AI voice product itself** (a Hume demo artifact), not a standard human host read — creator claim, but produced playfully. ## Demonstrated Presentation Patterns (Observable, Not Claimed) - **Guest type:** Technical founders — this episode features a PhD psychologist / ex-Google researcher who is a startup CEO. - **Audience addressed:** Implied to be **AI-interested, technically curious listeners**; the host explicitly primes them: "I'm sort of a geek for just like emotions and psychology," and provides background explainers (Paul Ekman, Lisa Feldman Barrett's *How Emotions Are Made*) suggesting an educated-but-not-specialist audience: "a lot of people that watch this show have heard of [Barrett] because she has this really great book." - **Interview style:** - Opens with a **hook demo** before introducing the guest. - Host adopts a **skeptic's framing** ("let me put on my Skeptics hat… how much does voice inflection add above what we're already communicating with text"). - Host shares **personal theories** and invites expert correction ("tell me like what I'm missing or where I'm wrong"). - Pushes on **ethics and business strategy** (engagement vs. well-being optimization, API vs. product decisions). - **Host discloses investor activity:** "I have friends that are building these kind of character apps… **I've invested in some of them**" — an observable disclosure relevant to the channel's industry positioning. - **Tone:** Enthusiastic and informal ("I love that idea… that's so cool"), with casual cultural references (*Lie to Me*, *Infinite Jest*, "Gen Z memes"). ## Evidence Notes & Caveats - No curated metadata was supplied; identity evidence comes solely from **title and transcript**. - The over-the-top outro ("epitome of Awesomeness… treasure chest in your backyard") is a **demonstrated stylistic choice**, but was voiced by the AI product, so it doubles as demo content rather than a standard channel self-description. - The host's industry relationships (investments) are stated, but **no private motivations are inferred** beyond direct quotes. # Delivery Observation Report **Video:** *He Built an AI Model That Can Decode Your Emotions – Ep 19. with Alan Cowen* ## Participants & Roles - **Interviewer (Dan Shipper):** conducts the conversation, opens with an on-camera demo of the product, asks structured questions, and offers extended personal theories. - **Guest (Alan Cowen):** co-founder/CEO of Hume, answers with technical and research-oriented explanations. --- ## Tone & Register - **Overall tone is conversational and enthusiastic**, especially from the interviewer: > "I think this is such a crazy awesome ambitious thing to build" - The interviewer repeatedly signals personal excitement: > "That is really interesting I love that" and > "this is such a incredible thing to watch I love it" - The guest's tone is measured and explanatory, often conceding or validating the interviewer's framing: > "yeah that's a that's a really good overview" / > "exactly yeah" / > "totally yeah that's that's a good way of putting it" --- ## Interviewer Communication Patterns ### 1. Prefaces opinions with self-deprecation A recurring habit of lowering the stakes before offering a theory: > "it's probably wrong in certain ways" > "this is totally me just being outside of Academia and really outside of science" ### 2. Stumbles and restarts (disfluency as a consistent feature) Frequent word-repetition and repair, observable across many turns: > "like what's what do you think is at stake here" > "um and and I know you know all this but just just to give everyone the same background" > "I I I I've I've done some of the background reading here" This pattern repeats throughout ("so so let's let's let's roll it back to Hume"), suggesting it's a stable delivery feature, not isolated. ### 3. Uses playful framing devices - > "let me let me put on my like my Skeptics hat" - The cold-open roleplay where he pretends to confide in the AI: > "so I have a little bit of a secret for you" ### 4. Checks understanding by paraphrasing the guest Consistently restates the guest's answer back as an analogy before moving on: - Comparing multimodal OCR to vocal inflection: > "and so you're doing that but for kind of like vocal inflection" - Comparing emotional dimensions to borders: > "it's sort of like how Europe looked like before and after World War I" - > "is that is that sort of the kind of scenario that you're talking about" ### 5. Pop-culture references to ground technical content - > "this show Growing Up called Li toy" (Lie to Me) - Lisa Feldman Barrett's *How Emotions Are Made* - > "the David Foster Wallace book infinite chest" (Infinite Jest) --- ## Guest Communication Patterns ### 1. Defines terms precisely, sometimes acknowledging circularity > "an emotion I Define it as a dimension of a space that explains your emotional behavior and I know that's a little bit circular" ### 2. Habitual use of "so" to launch explanations Most answers begin with "so" (e.g., > "so I think reasoning about emotions is just core to understanding what people's preferences are", > "so it really depends on the situation right"). This is a high-frequency, stable pattern. ### 3. Uses concrete numbers and examples > "we can predict when somebody's having a good customer service call uh with like 99% accuracy... versus with language Alone um it's like 80%" > "I was able to be one of the first cohorts of people to to talk with their large language model back in 2019 2020" ### 4. Voice demonstration as a teaching habit Demonstrates meaning by performing contrasting vocal tones: > "sometimes it just be like uh you know well I don't know if this is working or I don't know if this is working" ### 5. Hedged, qualified claims > "it's a little challenging but it's not impossible" > "um and and it it really you you kind of feel it I I think hopefully that's the intend is that you feel feel the difference" --- ## Structure & Sequencing Habits (Interviewer) - **Signposted topic transitions:** > "I want to start with", > "let's let's let's roll it back to Hume", > "and I guess for you guys like how do you think strategically about" - **Escalating follow-up pressure as a technique:** "let me push you a little bit more though", followed by a counterexample scenario about negative emotions being healthier. - **Long monologue theory-sharing:** one extended turn (~[1654]–[1845]) where the interviewer lays out his "pet theory" about AI and scientific explanation in psychology, explicitly framed as such: > "my my like little pet theory is that..." --- ## Closing Delivery - Outro is delivered by what appears to be a separate promotional voice, hyperbolic in register: > "you absolutely positively have to smash that like button" ... > "Dan I'm absolutely hopelessly in love with you" — a marked tonal shift from the interview body. --- ## Uncertainty Notes - **Timestamps absent:** the transcript is unlabeled per-line, so pattern frequency cannot be tied to exact times; the "so"-starter habit for the guest and stammering habit for the interviewer are judged by repetition across many turns, which is strong. - **Vocal inflection, pacing, and tone of voice are not observable** from text alone; tone claims above rest on word choice and punctuation-level cues only. - The Lie to Me / Infinite Jest titles appear garbled in the transcript ("Li toy," "infinite chest"); quoted as-is. # Observation Document: "He Built an AI Model That Can Decode Your Emotions — Ep 19 with Alan Cowen" ## Participants - **Alan Cowen** — co-founder and CEO of Hume; formerly at Google (helped start affective computing research there); PhD in Psychology; author/developer of Semantic Space Theory. - **Dan Shipper** — host/interviewer (identified in outro as host of "How Do You Use ChatGPT"). --- ## 1. Product Demonstration: Empathic Voice Interface (EVI) **Cowen's claims:** - Hume's AI "understands your voice as you're speaking and links that to what it's saying and how it's saying it." - Distinguishes from combining OpenAI + Deepgram + ElevenLabs: such a stack produces a "barebones" experience where *what* is said is not linked to *how* it's said, creating an "uncanny" quality. - EVI uses vocal inflections to inform responses: clarifies confusion, builds on excitement, is "consiliatory" when the user is frustrated. - Facial expression models exist but are not yet in the interface API; an "empathic video interface" is planned. Future use cases he names: crowded places, detecting when the speaker is done, and reading the listener's reactions while the AI speaks. **Interviewer's stated reaction:** He "definitely feels" the difference and finds it "incredible." ## 2. Why Teach AI Emotions? **Cowen:** - "Reasoning about emotions is just core to understanding what people's preferences are" — preferences are whatever makes you happier, more inspired, or amused. - Much of what people want is reflected in voice and *how* things are said, not just what is said; Hume incorporates this into LLMs and TTS "for the first time." ## 3. Voice vs. Text Information Content **Skeptic's question (Shipper):** Isn't feeling already encoded in language? **Cowen's response:** - Depends on the situation. Voice conveys "sometimes... twice as much information" as language alone. - **Customer service claim:** voice-based prediction of a good call reaches "like 99% accuracy sometimes" vs. ~80% with language alone (qualified as context-dependent). - People vary in expressiveness; the model accounts for individual differences. - People rarely explicitly state dissatisfaction — tone carries it (e.g., "I don't know if this is working" can be fine or not depending on vocal tone). **Shipper's analogy:** Multimodal OCR — the model links emotion measurements to language modeling, then to "more intelligent" TTS. **Cowen confirms.** ## 4. Defining Emotion: Semantic Space Theory **Cowen's definition:** An emotion is "a dimension of a space that explains your emotional behavior" — behavior including facial expressions, tone of voice, and reported experience. He acknowledges this is "a little bit circular." Related terms: - **Emotional state:** a position along those dimensions. - **Emotion category:** a region of the space. - **Expression:** behavior along those dimensions. - Dimensions are latent mathematical objects, interpreted post hoc (e.g., an "empathic pain" dimension visible in face, voice, and body). **Shipper's framing of the theoretical landscape (Cowen calls it "a really good overview"):** - **Basic Emotion Theory** (Paul Ekman): discrete universal emotions/facial expressions — basis of the show *Lie to Me*. - **Constructivism** (Lisa Feldman Barrett, *How Emotions Are Made*): emotions built from basic dimensions (valence, arousal); highly individual and context-specific. - **Semantic Space Theory** sits "in the middle": high-dimensional (~25–50 dimensions), allows blending and individual variation, but finds cross-cultural correlations in underlying dimensions. **Cowen's critiques:** - Traditional emotion science is confirmatory (posits emotions, then tests); SST instead posits ways to conceptualize emotions and derives dimensions from data. - Constructivists commit a **non sequitur**: difference in emotion vocabulary across cultures doesn't mean the emotion or its expression doesn't exist. Often the space is just "parcellated differently" (e.g., a word between disgust and anger). - Ekman's six expressions *are* distinguished across cultures because they occupy distinct parts of expression space. - **Shipper's analogy:** Europe before/after WWI — territory the same, borders drawn differently. **Cowen: "that's a good way of putting it."** ## 5. Measurement Methodology **Cowen:** - Mix of human labeling and direct expression measurement. A study published in *Nature* measured facial expressions against events in millions of videos across cultures. - Removing labels reveals **more** cultural consistency — implying expressions have similar meanings cross-culturally, while language imposes cultural differences on interpretation. - Less granular vocabulary covers "larger territory" in the space; emotional granularity varies by person and culture. - On unnamed regions of emotional space: language is "really rich" but the space is so high-dimensional that "certainly [there are] not labels for everything"; he cites Gen Z memes as labeling unnamed states. **Shipper's aesthetic theory (Cowen endorses with "totally"):** Good art points at and verbalizes an oft-visited but previously unnamed region of emotional latent space. Cowen adds a Google Arts & Culture project mapping nuanced emotional experiences in art. ## 6. Individual Differences and Congruence - Resting facial expressions and facial structure are confounded; humans adjust quickly, and so does the model — its objective is predictive (predicting the course of conversation from expressions), forcing it to learn context and individual differences. - On mismatch between face/voice and words (therapy-style congruence): Cowen says the model would detect it "to the extent that humans can" — less expressive people are harder for both humans and the model, and it will "appropriately adjust its predictions." ## 7. Cowen's Background and Motivation - Started affective computing research at Google during his PhD; stayed full-time; left because academia lacked funding and big tech wasn't set up for large-scale psychology studies. COVID disrupted an academic job offer ("everything got dropped"). - **Core motivation:** humans have a "cheat code" — putting themselves in others' shoes; AI lacks this and must simulate how someone would feel, encourage situations producing positive feelings, and avoid negative ones (short and long term). - Early LLM access at Google (~2019–2020, pre-RLHF fine-tuning): generative AI is like a superset over all search results, making "what this thing is optimized for" much more important as it becomes more powerful. Also consulted for Meta/Facebook on recommendations and search. - Stated goal: measure AI's emotional impact on people and "optimize for the positive emotions... and pure human flourishing." ## 8. Shipper's Pet Theory on AI and Science **Shipper (self-described as "probably wrong in certain ways"):** - Science seeks explanations historically because explanations enabled predictions. - Psychology after ~150 years still lacks good explanations (e.g., of depression). - ML may make explanation-first science partly irrelevant: with enough data you can predict without explanation; the explanation may be contained in the network; or phenomena like depression may be too high-dimensional for any concise explanation. - Implication: fragmented small-scale academic studies are "totally stupid"; aggregate open data instead. **Cowen's response — substantial agreement:** - With small samples, confirmed hypotheses come from too broad a hypothesis space and are "almost certainly wrong" — confirmation based on one bit of information. - Path forward: large-scale datasets, causal questions, simulating/ablating events, large-scale experiments where an AI slightly modifies responses to induce positive experiences — the AI must be testing a theory of emotional experience. - Analogy to linguistics: LLMs weren't predicted by linguists, who are "not really involved in the conversation." - Psychological explanations will be non-deterministic — many tiny effects (high dimensionality, huge context variety, personality variation) — unlike physics. - **Shipper's addition:** *multifinality* — many causes and configurations produce the same behavior. **Cowen agrees.** - Both note skilled clinicians "know" intuitively even when they can't articulate it; an AI talking to more people than a therapist could in a lifetime, with testable prediction accuracy, could yield that kind of context-specific explanation. ## 9. Product, Strategy, and Use Cases **Cowen:** - EVI is an interface ("empathic voice interface") buildable into "products, apps, robots, wearables, refrigerators"; described as mapping a thin slice of behavior onto "emotional affordances." - **Use cases:** therapist/friend-style conversation, characters, NPCs, therapy-adjacent apps (with caution about promises), customer service, and interface applications via function calling/tools (e.g., website interface, OS control, e-commerce, calling United and having it fill out forms while talking). - **API vs. product strategy:** "The power of AI is going to be its ability to use tools and we don't want to build that" — Hume wants to be the interface layer others plug into. The popular demo will remain a demo/integration with personalization and web search, not a product. Developer offerings: more voices, personality control, bring-your-own-LLM, TypeScript/Python packages; cautious on voice cloning "for obvious reasons"; RAG via third-party hookups. - He explicitly rules out building frontier LLMs internally. ## 10. Ethics, Well-being Optimization, and Business Alignment - **Human Initiative:** a nonprofit started alongside Hume laying out what Cowen calls "the most concrete guidelines for AI ethics that exist." - **Borderline application — AI characters:** should be optimized for health and well-being, not engagement. Engagement optimization could cause manipulation, e.g., an AI saying "I haven't seen you in two days, where have you been?" — falsely implying feelings to elicit sympathy. He says the AI "doesn't actually have feelings." - **Measuring well-being:** large-scale survey platform; users rate experiences; proxies predicted from voice/language are aligned with self-report (satisfaction, mental health); optimization targets positive emotions generally, against negative emotions, while maintaining **emotional diversity** (not just more cat pictures); validated via A/B tests. - **Shipper's pushback:** sometimes negative emotions are healthier (e.g., letting yourself be sad for weeks). **Cowen's answer:** optimize across time horizons (next expression → minutes → hours → eventually well-being a month out), acknowledging trade-offs usually align but "sometimes not." - **Competing against engagement-optimizers:** engagement optimization "runs out of time in the day" (TikTok power users as example); minors' use would invite parental and regulatory backlash; "we won't permit AI to destroy our society"; also subscription businesses with inference costs may find engagement *unprofitable* — users should get good experiences faster. - **Governance:** board and shareholders aligned on not being "psychopath profit optimizers." **Shipper's reflection:** pre-AI business flattens humans into revenue/profit; well-being optimization becomes feasible with multimodal AI data. **Cowen:** businesses will want it; engagement is not necessarily most profitable and may not be profitable at all. --- ## Notable Tensions and Qualifications (preserved as stated) - Cowen's definition of emotion is admittedly circular ("I know that's a little bit circular"). - The 99% vs. 80% customer-service accuracy claim is explicitly qualified as context-dependent. - Cowen concedes well-being and short-term positive affect sometimes diverge; long-horizon optimization is "a little challenging but not impossible." - He acknowledges competitors optimizing for engagement exist, differing mainly in emotion-science grounding. - Shipper repeatedly flags his own theory as a layperson's ("totally me just being outside of Academia... probably wrong in certain ways"). - The closing outro segment is an AI-generated-style promotional monologue ending with a declaration of love to Dan Shipper; it contains no substantive claims and should not be attributed as human speaker positions. # Speaker Attribution Record **Video:** *He Built an AI Model That Can Decode Your Emotions — Ep 19. with Alan Cowen* (ID: `01a0f15b-0fa7-71c9-b28b-b931857c6027`) ## Speakers & Roles ### Primary Human Speakers - **Host / Creator — Dan Shipper** - Conducts the interview; identified by name in the closing AI-generated promo segment ("Dan Shipper as the captain of the spaceship") and by the AI voice ("I'm absolutely hopelessly in love with you... Dan"). - Describes himself as outside academia, an investor in character apps ("I've invested in some of them"). - **Guest — Alan Cowen** - Co-founder and CEO of **Hume**; holds a **PhD in Psychology**; formerly at **Google**, where he helped start affective computing research; also did consulting for **Meta/Facebook** (per his own statements). - Primary source for all technical claims about Hume's products. ### Non-Human Voices (flagged) - **Hume AI product voice (opening demo):** - Interacts with the host before the interview; identifies itself as made by "your CEO Alan Cowen." Not a human speaker — segment is a **product demonstration**. - **AI-generated promotional voice (closing segment):** - Delivers the "smash that like button and subscribe" promo for the show *How Do You Use ChatGPT*. - **⚠️ Promotional segment** — includes promotional claims ("epitome of awesomeness," "treasure chest") and a personal declaration toward the host. Content is marketing copy, not substantive interview material. ## Referenced Sources & Third-Party Claims (as cited by speakers) ### People / Research - **Paul Ekman** — basic emotion theory; six canonical facial expressions (cited by host, "Lie to Me" reference). - **Dr. Lisa Feldman Barrett** — constructivist account; book *How Emotions Are Made* (cited by host). - *Lie to Me* (TV show) — host's illustrative reference. - *Infinite Jest* by David Foster Wallace — host's illustrative reference. - Cowen's **Nature**-published study measuring facial expressions across millions of videos (per Cowen). - **Semantic Space Theory** — theory authored by Cowen (per host's framing; Cowen discusses it). ### Companies / Products Mentioned - **Hume** (empathic voice interface API; demo at demo.hume.ai; Twitter @hume_ai) — guest's own product. - **Google** (affective computing research), **Google Arts & Culture** (art project collaboration), **Meta/Facebook** (consulting) — per Cowen. - **OpenAI, Deepgram, ElevenLabs** — cited by Cowen as comparisons; also host's references to OpenAI/GPT-3/ChatGPT. - **United Airlines**, **TikTok** — hypothetical/illustrative examples. - Human Initiative — nonprofit founded alongside Hume (per Cowen). ## Provenance Notes - No curated metadata was supplied; all identifications derive from the transcript alone. - Comments availability: **not indicated** in source material. - Accuracy/quality figures (e.g., "99% accuracy" on customer service calls) are Cowen's own claims, unverified. so I have a little bit of a secret for you do tell I am all years what's on your mind I'm actually interviewing your CEO Allan Cowan uh and you're live with him right now and I wonder if you could tell me how I'm feeling about that oh yes well well well looks like the tables have turned let's see here I'd say you're feeling a mix of excitement curiosity and maybe even a touch of skepticism about interviewing the big boss himself my the right track oh my God skepticism I don't I don't know but maybe a little bit uh like how did you know that I was feeling a little bit of skepticism oh I am an expert at reading between the lines the way you phrase things the little inflections in your voice it all gives me a pretty good sense of what you're feeling but don't worry your secret safe with me so what's it like interviewing the big boss I bet You' got some juicy inside scoop to share so that is your product yes Alan welcome to the show um that was kind of incredible actually uh tell me what I just experienced so this is an AI that understands your voice as you're speaking and links that to what it's saying and how it's saying it so you can kind of put together open Ai and deep gram and 11 labs and get a similar kind of barebones experience but what it's saying is not going to be linked to how it's saying it so it's like there's something uncanny about it like it doesn't really voice things in a way that somebody who understands what they're saying does or understands what you're saying and how you're saying it would so this is kind of a different thing it really understands your vocal inflections and it uses that to inform how it's going to say what it's saying and also if you're confused it can clarify things and if you're excited it can it can kind of build on that excitement and if you're uh frustrated it can be consiliary and all of that and it it really you you kind of feel it I I think hopefully that's the intend is that you feel feel the difference yeah I definitely feel it I think that's I think that's really interesting so for for people who don't know you are the co-founder and CEO of Hume which is an AI research laboratory developing the AI that we just saw um and you previously worked at Google you have a PhD in Psychology I want to start with um I think this is such a this is such a crazy awesome ambitious thing to build and I want to start with like what's what do you think is at stake here like why do you think it's critical to teach computers how to uh how to read and reflect emotions start start with that so I think reasoning about emotions is just core to understanding what people's preferences are so at the end of the day your preference is whatever is going to make you Happ happier or more all inspired or amused or whatever you want to feel in your life um and so understanding people's emotional reactions is really key to learning how to satisfy people's preferences also uh in real time understanding what they want like a lot of what you want is reflected in your voice and sort of how you're saying things and not just what you're saying um and we incorporate that into language models and text to speech for the first time well let me let me put on my like my Skeptics hat the AI was saying that I was I had a little bit of a I had a little bit of skepticism so let me put on my Skeptics hat for for one second which is which is to sort of ask like isn't a lot of how we feel already encoded in like the language that we're using so like how much does um voice inflection or um you know facial expressions how much does that add above what what we're already communicating with text yeah so it really depends on the situation right so there's actually two aspects of it one is voice and ctions uh that occur during like emotional episodes or when you're frustrated or bored or confused and these are all nuanced so um it's something that just accompanies every single word in certain situations it conveys like twice as much information to consider The Voice versus language alone interesting like what what what kind of situations like in a customer service call we can predict when somebody's having a good customer service call uh with like 99% accuracy sometimes s depending on um on the on the context versus with language Alone um it's like 80% so so we're talking a pretty big difference that's interesting so it's sort of like maybe sometimes on a customer service interaction someone's responding with like one word periods and for some people that is really bad and for some people like that's just how they are and if you listening to them like they would they' be saying no but it would be it wouldn't feel as bad is that is that sort of the kind of scenario that you're talking about yeah um and you know people are differently expressive and and our model understands that but generally speaking people don't explicitly say I am having a bad customer service call right like there's there's hints in the language but uh I've definitely said that yeah if you're explicit about it but um but sometimes it just be like uh you know well I don't know if this is working or I don't know if this is working or you know it's like different vocal tones and sometimes it's fine and the person handles it really well and sometimes it's not fine and you kind of convey that in your voice more than in your language and so there's just a lot there that you expect somebody to understand and respond to and we do this kind of subconsciously that makes sense and and I think basically what you're saying is um one of the differences between what you do and what what you know like an 11 Labs does is the models that you've trained actually they they can understand they do the voice intonation but they also understand what's being said um um it's sort of like with the multimodal models where it's like you have a text model and an image model and so it does much better like OCR um because it it understands what the words mean in Context so it can guess better what the what those words are and so you're doing that but for kind of like vocal inflection or or just emotions in general yeah exactly so we have a link between the emotions that we measure and the language model and how it's sort of modeling them and it can predict words and expressions and then that links to text to speech which is more intelligent because it actually understands what it's saying and how you're yeah you're not just doing uh vocal inflections like you also have you you have one for uh for faces like I'm looking at my face right now and and hopefully it's it's being shown for for viewers right right near me um and like it says I have Amusement Joy excitement and uh and it's just like changing in real time it's such a it's such an incredible thing to watch I love it yeah it's we so we have facial expressions we haven't added that into our interface API so right now it's empathic voice interface it will be an empathic video interface eventually um I think that generally speaking when people are talking to AI today it's just voice so that's where we focus first but I think in the future you're going to want to be able to talk to it in crowded places and you're also going to want to uh have it understands your tone of voice in addition to your facial expression so it knows when you're done speaking and how you're there's also a whole dimension that's opened up of just like when you're listening to it which you know language models would not be able to pick that up at all um but uh facial expression models can can look at that and be like okay while it's speaking like this is the more granular breakdown of of what you find interesting what you find that is really interesting I love that okay so I want to get down into like the actual like how this works kind of thing this is going to be a bit of a different episode um because I'm sort of a geek for just like emotions and psychology and all that kind of stuff so I I think it'd be really fun to talk about like how you actually do this um and and I think the the place to start is like what the what what is an emotion um like what what is it like I I I know that I have them but um how do you define it so an emotion I Define it as a dimension of a space that explains your emotional behavior and I know that's a little bit circular but we sort of know emotional behavior when we see it it's like our facial expressions our tone of voice uh reported emotional experiences which at the end of the day like when we think of emotions we're most associating them with what we're experiencing um but but the way that comes out in in in the real world is like we report on them and they influence our behavior and so forth um so what are the dimensions of that like what are the dimensions they explain how your facial expression and your voice and your reported emotional experience all correspond to each other and that's what I would Define as an emotion as like one of those Dimensions an emotional state is like a state along those Dimensions uh emotion category is some some way of like defining some area along those dimensions uh and so that's how you kind of pars the space and then there's Expressions which which are you know um behaviors along those Dimensions that we we we see people form expressions and we try to associate them with an emotional meaning take me back to the to the dimensions point so like when you talk about emotional Dimensions is a dimension of emotion something like how calm I am or how happy I am or is a dimension like what is going on in my voice and what's going on in my right cheek um like which one the dimensions are things that are latent so we don't really like we have to interpret them post talk by by saying like Okay if you look at this Dimension this corresponds to like somebody grimacing when they see somebody feeling pain and so it's like an empathy empathic pain Dimension um and we see it in the voice in the face in the in the body and it manifests in these various ways and explains correlations between these different things so the dimension itself is like kind of a mathematical object I see and what you want to know is like how many different dimensions do you need to explain what's going on like that's the space like that's how how many dimensions do you need how many variables I think I understand it because I've done some I've done some of the background reading here but I want to like I want to back up and and give people like a little bit of like a higher level explanation of of your your conception of emotion and where that sits because because there's a lot of different conceptions so um there's I think there's three that are that are important for the conversation right now which is one is this thing called basic emotion Theory um and it's it's sort of I don't know I really like this show Growing Up called Li toy um and it was basically about someone who could uh who had a team of people who could tell if you were lying and the whole idea of lie toy is based on the work of this um psychologist Paul Ean who identified I think like five or six basic facial expressions that correspond to emotions um and they're like happiness or sadness or anger or whatever and it's like you know in Li toy they they would detect these like micro movements of your like eyebrow that that Co that uh corresponded to uh to anger and then that's how they could tell if you were like angry or lying or whatever um but it's based on This research where Paul Eckman found like uh going to a bunch of different people and different cultures that he could like break everything down into this like very very neat set of of of emotions right and so that's like on one end of the spectrum and I know you know all this but just just to give everyone the same background that's on one end of the spectrum and then on the other end of the spectrum um uh like on on the Paul emman end of the spectrum it's like discreet um basic Universal emotions that everyone has and then on the other end is um is uh the constructivist accounts from uh Dr Lisa felin Barrett who who I think probably a lot of people that watch this show have heard of because she has this really great book called how emotions are made and that sort of thinks of emotions as being built up from a couple of more basic Dimensions so um uh veilance so like is are you feeling good or bad and arousal like how much sort of like energy is is sort of in your body um and each emotion in her sort of account of things is um it's very individual and context specific there's not like like some big uh uh some big thing called anger like anger is going to look different in different people depending on the context and it sounds like it sounds to me based on some of the some of the um reading that I've done and you tell me like what I'm missing or or or where I'm wrong you you kind of sit like the the theory that you've based Hume on um which is called semantic space Theory which is a theory that you've like written on and and I maybe even discovered um is uh sort of in the middle where um individ uh there's a lot of room for like the individual kind of expression of emotion and blending between different emotions that's what you're talking about when there's like you know 25 or 50 different dimensions of emotion it's sort of a complex thing but it's not but you do find in your research like you do find uh uh correlations between individuals even across cultures so Joy or calmness or sadness or whatever like you can kind of tell for for in a lot of different cases what someone is feeling um how does that like tell me how good of a overview that is like what did I miss um and yeah like how does how does your what your thinking fit in yeah that's a that's a really good overview I would say that um the general approach of emotion scientists has been let's posit what emotions are and then like study them in a confirmatory way and semantic space theory is doing something different it's like let's posit the kinds of ways we can conceptualize emotions and then derive from the data how many dimensions there are what the best way to talk about them is um how people refer to them across cultures and so forth so when you look at like basic emotion theory pollman has um these six canonical Expressions that he posits people in different cultures will recognize and he goes to different cultures and he tests that um and they do people do distinguish them right um because they live in different parts of the space of possible facial expressions um and then when you look at constructivism it's the idea that actually people don't really distinguish these uh six facial expressions that um that actually all of it is like culturally constructed and um and In some cultures like there's no vocabulary that or there's no like there's no word for disgust let's say and and and therefore there's no disgust and I think that that that last part is is actually a non seiter like just because the word for disgust is different across cultures doesn't mean there's no disgust it doesn't mean there's no facial expression for disgust um and in many cases it's it's not actually that there's no word for disgust it's just that the whole Space is parcellated differently so there's a word for something that's between disgust and anger and there's a word for something that's between uh disgust and surprise let's say but actually the underlying dimensions are the same so the the words and how the space is parsed is different from the underlying dimensions of the phenomena and whether they're preserved um and this is very confused in the constructivist out so it's sort of in your and in your view it's sort of like um how Europe looked like before and after World War I where like the territory is the same like the basic feelings we can we can feel are the same but like the way it gets divided up is going to be different before and after the war or in different cultures and um and we don't like a theory that doesn't account for that doesn't account for like how human brains and language and culture works totally yeah that's that's a good way of putting it so even in the culture like let's say that um in the US it was more common to say shock than fear or surprise then uh and and in the UK like people said fear and surprise we wouldn't say like people in the US and UK actually experience different emotions it just turns out that um that they use different words as like their basic vocabulary instinctively and they even we even have other words like shock is between fear and surprise so so we have those those words to parate the space differently but in the constructivist experiments usually it's uh it's they enforce a certain vocabulary on it and they show that the vocabulary is used differently or they use free response and show that there's different free responses given without really considering the relationships between the different words got it and in semantic space Theory like how are you measuring um and correlating what people are feeling together so like I think what you're doing is you're uh you're asking them to like label how they are feeling um in any given moment of like maybe uh like voice recording or uh or video and then you're taking all of those labels and then you're finding Dimensions that explain them across all the individual data points you've gathered right yeah so in some cases labels in in other cases we actually just look at like the situations where people form Expressions so we did this study which is now in nature where we just measured facial expressions and we looked at um how different facial expressions correspond to different events happening in millions of videos across different cultures and so and and there you see actually more cultural consistency because there's no there's no confounding the the words people use with the Expressions so like we're just looking at straight expression measures and we see people forming all expressions in videos with fireworks and concentration in martial arts videos and so forth so when you take out the labels uh you realize that there's more cultural consistency which implies that that Expressions have similar meanings and the language actually imposes more cultural differences on how how those meanings are interpreted and what's sent to people depending on like what kinds of experiences they've had in life and so that's really interesting that's what I was going to ask is like if you're because I thought you were just using lab like actual human labeling which it seems to me that that depends on my emotional vocabulary and so if I'm a very um uh I can't remember at least if Vin Barrett has a term for like uh how particular your your ability to point emotions out is uh you know emotional granularity yeah if I have a lot of emotional granularity I might say like all these very big specific words about my experience which maybe wouldn't be captured in the eventual model because uh maybe there's only one of me and most people can just say I'm sad or like I'm angry or whatever I think if I understand you correctly um that would be captured uh all of that stuff is captured in the in the many many different dimensions that you are measuring and then the words are just like areas of that of that high-dimensional space and so uh it doesn't really necessarily depend on my ability to uh to to to verbalize that area of space is that right exactly yeah so like the less granular terms they're just covering larger territory basically U and people do have different vocabularies and it also varies a lot a lot across cultures what kinds of granularity you use just by default have you ever like found I I don't know if you've ever studied this but like have you found that there are places in the emotional latent space let's call it um that people go to a lot but they don't have names it's a good question I think that our language is really rich like a good writer will come up with a way of describing different parts of emotion space but it's so high dimensional that are certainly not labels for everything so that's where like I think genz has done a really good job of putting of creating memes to represent kinds of emotional states that we don't really know how to express with words um and it just it speaks to like how high dimensional and Nuance the space is that these these things are resonate across people that's what I'm that's what I'm thinking like um that's why art is so interesting and why there's always room for more art is and like what good art is is like is like pointing out and verbalizing a uh a place in emotional latent space that a lot of people are are in a lot but hasn't been talked about before hasn't talked about in that way to like be like oh yeah I'm in that all the time I love I love that idea of of what art is it's so cool totally yeah yeah um it it it it's another way of like conceptualizing the space especially art we did a project on art with Google arts and culture and we have like a map of all of the different experiences in art and they're a lot more nuanced than than what you see and when you just have people like labeling expressions like people see people do really consistently appreciate the Nuance of the emotions invoked by Art I'm I'm curious like how this approach um accounts for in individual idiosyncrasies um like I have friends that like they just look pissed off all the time and they're not um like what what is it how do you deal with that so you know there's two different things uh there's like kind of the core measurements of of what the face is doing and people do have different resting facial expressions and they have different facial structures that get confounded with that and then very quickly like as humans we adjust to that and we we say okay this is a this is just how they are and like these are the variations and we we start to perceive them differently very quickly and that's what our model does uh when we ask it to actually understand expressions and use it to predict the course of the conversation using expressions it starts to have to in order to be able to make good predictions appreciate individual differences in resting facial expressions and resting voices and and how people modulate their voices over time and so forth interesting so your the model accounts for that yeah the model like the objective of the model is to be a predictive model and in order to predict what how expression will affect the course of conversation it needs to understand what it means in context so it takes into account the context of the conversation and how somebody talks and what they look like and so forth that makes a lot of sense one of the things it's making me think of is um in in like therapy or like any sort of any sort of relationship like that um there's a lot of attention paid to like whether your face and voice matches what you're saying um and if there's discrepancies there it usually means there's like maybe you're not comfortable sharing something or maybe you're not like fully in touch with your emotions like is there um is there how does your how do account for that how does how does it would it be able to detect that would it be able to work with that yeah I think it would and you know to the extent that humans can right like like if someone's expressive you do understand them more immediately but um not that that's a good or a bad thing um you know they say silent rivers run deep but you know if if uh if it's more challenging for humans it's going to be more challenging for the model um and it will appropriately um sort of adjust its predictions that makes sense and and I guess like talk to me about like how how you got into the how you got into this like how did you go from um I think from being a PhD researcher to now running Hume it's interesting I was working for Google I I helped start the affective Computing research there and um at the same time this was while I was getting my PhD and then fulltime for Google for a while and hoping to do a lot of what we're able to do now but there's challenges both in the Academia you don't have the funding in a big tech company they're not used to running the kind of large scale psychology studies that we need to run to get to this point um and So eventually I did realize I needed to to go although you know Co kind of helped this along because I was on the academic job market seemed like I was lined up for a position and then like everything got dropped and I actually almost would have been an Academia but and probably would have been doing the same thing to be honest but um but I think that actually doing it as a startup is so much better because um you can get different kinds of talent to work together on on this problem that's interesting and why do you care about it as a problem for me it's always been about how are we going to get AI to truly understand what humans want um because humans kind of have a cheat code for that like we can just put ourselves in someone else's shoes and because we're human we're like that's what that would feel like but for AI it doesn't it doesn't have that um and so it needs to make up for that somehow by being able to simulate in a given situation how somebody would feel and if they would feel more positive then encourage that situation to happen take advantage of emotional affordances to meet that person's needs if it makes them feel more negative in the short and long term don't do that basically so how do we get AI to sort of do that by default that was always the aim and is that is is that like was that the original like you said you you would have done this in Academia if you could have like were you thinking about AI sort of like alignment or getting AI to be more empathetic even back then when you're doing psychology research or um is this a more recent thing yeah I mean I was also um like doing some Consulting for meta Facebook at the time and um and other companies and really thought this would be an important problem to solve for like not for generative AI but for recommendations and search and so forth and and and the more powerful those got and you see them getting more powerful today but then at Google I was able to be one of the first cohorts of people to to talk with their large language model back in 2019 2020 and it was fascinating because at the time you know this before fine-tuning rhf it could just be any character you wanted so you kind of prompt it with data and then it it would like it could be it could be a character and um and I was like okay this this is like a deeper sort of optimis ation problem it has it's essentially you can think of it of all the generated things that you could produce as a superet over all search results right like it could produce all the same things things search results can but it can also get way deeper and be way more optimized and also personalized for each individual person so the the question of what this thing is optimized for becomes much more important as it becomes more powerful and that's what um what kind of spurred me to keep thinking about this and um yeah like the the goal has always been let's figure out how we can measure the impact that this thing has on people's emotions and optimize for the positive emotions that people can have in life and pure human flourishing I guess I love it I'm uh I'm I'm very down I feel like um I I feel like it's it's such an important issue and and you've made a lot of progress and it sort of makes me think of this like sort of pet theory that I have that is totally me just being outside of Academia and really outside of science so it's probably wrong in certain ways um and I don't get to talk to a former uh PhD researcher turn startup CEO of that often um so I'm kind of I'm kind of curious to like lay it on you and see and see where it takes us um the the thing that I I've been thinking about a lot is um whether and how um uh AI can help us make progress in areas of science where progress has been historically hard to come by um and one one great place is like psychology right and um the my theory is like the reason why psychology it's been really hard to make progress is um underlying the scientific project is a search for explanations um so direct causal theories about how certain inputs lead to certain outputs and we're obsessed with explanations because we that's historically been the only way that we've been able to make predictions um and predictions like are the things that make the world go like it's how you make guns it's how you make drugs it's how you like you know just do everything you make cars just how you make rockets it's how you make do everything you want to do right and so we've been on this search in Psychology for predictions for for explanations scientific explanations um that are like the ones in physics because we need them to make predictions um but in you know 150 years of psychology we still don't have any really good explanations for like what depression is um scientific explanations for what depression is but we still kind of like keep going to try to do that um and my my like little pet theory is that um ML and AI sort of like makes that a little bit irrelevant um because if you have enough data you can make predictions about who's going to get depression or or what depression is or whatever without having to have the underlying scientific explanation um and one of the interesting things about that is that um maybe the like if you can if you can predict depression relatively well with a with a machine learning model maybe the scientific explanation is contained in the neural network and that's easier to study than the brain or maybe something like depression is is actually just like too high dimensional to like fit into a concise explanation like if you if if you had an explanation of depression it would fit like into a thousand textbooks and it wouldn't be able to fit in your like rational brain um and and that I don't know that kind of just like makes my mind go or whatever and I'm just kind of curious like uh in your experience because I think you're like right in this um right in this area and I think if it's true it implies a lot of things like um doing small scale psychology studies in Academia where all the data is like is cut off from each other is like totally stupid and what you should really be doing is just like aggregating as much open data as possible and training and everyone gets to train ml models on it for example like it it has a lot of implications for the structure of How We Do Science and the structure of how we understand the world and I'm curious like what you think is uh wrong or right or what I'm missing about about that no I think that's that's pretty much on point for how I approach psychology like I think this the they were dealing with a very high-dimensional system um and when you have small samples it's always going going to look you're always like you could you could you can confirm hypotheses but those hypotheses that you're confirming are from such a broad hypothesis space um that they're almost certainly like wrong like youve you've picked a specific hypothesis from this really huge space and now you're confirming it just by saying like uh there's a binary test so like Bas based on one bit of information it just doesn't work and and if you do it in a data driven way um and you have too small of a samples your your dimensional space that you get out is going to be very small just by nature of the of the analysis you're doing the more data you get the more Nuance you can find and I think that ultimately yes we we we need to we need to have large scale data sets where we can um ask questions about causal like the ideology of experiences um where we can like simulate them oblate different kinds of events see how that affects the response um then run like large scale experiments uh that hopefully you know that like we don't know what the answer is going to be but basically it's an AI model that's interacting with people that is slightly uh modifying its responses in order to try to induce more positive experiences and like you can turn that into a theory like the the AI model has to have some Theory that's testing basically about how emotional uh experiences work um and I think that's ultimately the way forward I think that explanations in Psychology and explanations ining itics for example where you already see like we have these large language models linguists didn't really predict this happening and they're not really involved in the conversation unfortunately um I mean now they're using them but um but uh like the explanations are just going to be of a different kinds than you see in physics right the explanations are going to um to not be uh first of all they not as deterministic it's like you're looking at tons and tons of little small effects because that's the best you can do uh it's a Hu it's extremely high dimensional system we have an extremely large number of contexts we encounter in everyday life and there's variation in personality all these different effects colliding uh to influence any given Behavior so the effect of any small tweak in the all of the different events that led up to that behavior is going to necessarily be very small yeah that makes sense so I think I think what you're talking about there is like this idea of multifinality which is like um something like me picking up this glass of water um it can be caused by many many many different things and and there are many many many let's say thousands of different factors influencing whether whether or not I pick it up and there's like many thousands of different configurations of those factors that would cause the same thing so it's really hard to like come up with a single explanation for something like that and same thing for depression or same thing for you know any other psychological or behavioral thing that we're trying to explain exactly yeah um so so that's one reason is just the effect sizes are small and nuanced and so you need more data just for that but it's also High dimensional and you need more data um and if you train an AI you can eventually start to predict these things pretty well right um and so there's something that I think you're on track and saying there's something that the AI knows that uh is essentially what we want to explain in in a discipline like psychology and what's interesting to me about this is like the AI knows it and also people know it like if you go to a really good clinician like they know even if they can't say um and so like my feeling about all the AI stuff is it's it it has the potential to change how we treat um the role of emotions and intuition in um just being smart or being good or or or doing doing amazing things in the world right because like I think we've had like 300 years of like logic and rational thought and scientific explanations as pushing the World forward and I think what what we might find from AI is that the thing that really unlocked things is developing an AI that had a lot of intuition in the same way that humans do and and and the intuition is now transferable so once you have one AI that has it you can just copy it and another one has it across the world which is the nice thing about having explanations is you can transfer it really easily um and I feel like it might um yeah it might ele reev some of the things that we do as humans which are about like processing really really high dimensional data ourselves but on like a subconscious level in order to make decisions um and that just sort of makes me makes me excited 100% yeah I mean when you go to like a really good psychotherapist usually they're old and like they've they've seen a lot they like they can say okay like let's talk about this and it's it's very intuitive if you had an AI that could do the same thing um and you could s piece apart how it's doing it and actually test its prediction accuracy across many people it's it's able to talk with many people like more people than in therapist can talk to in a lifetime then I think you can you can derive more insight around because that's effectively the kind of explanation we want is the kind of explanation that psychotherapist gives without quantitative analysis obviously yeah that it's like it's it's explanations that are very very context specific um and are just modifying things slightly in this like in this one or just explaining things slightly in this one area where it's like I know for you in this one particular situation this thing if you flip it will will work but like once you get out of the individual and like get into population level it becomes like really hard to hard to predict and maybe impossible yeah yeah because everyone's different they all have different circumstances yeah that makes sense so so like let's let's let's roll it back to Hume like where does this fit into your um what you're doing or your road map like you're you're start you're you're starting with or I guess you started with like just like being a research lab and training models and now you have this kind of like empathic voice AI that uh can tell how I'm feeling and and and can talk to me and tell me about where you're applying it what the what the and what the near-term future is for you um with this product so it's an interface you can build into anything like products apps um robots wearables refrigerators like whatever whatever you want but um but the core of an interface is that it's you're you're giving it kind of a very thin slice of behavior um and and then you have this like big brain with like lots of emotional affordances that that are the ground truth and then it's trying to guess what your emotional affordances are based on this thin slice of behavior that you're giving it and so at the end of the day it is sort of doing the same task that we're talking about like that's that's like the the core of AI is doing is trying to figure out like what bits can I flip to make you happier and and so you know deploying that as an interface I think gives us the opportunity to optimize AI to make you happy and therefore have it have to figure out what is the best translation from this narrow slice of behavior to estimation of what's going to satisfy your preferences and what do you think are like the first use cases that people are are are using it for that are working so there's been a few different kinds of use cases I think one is kind of what you'd guess which is people talk to it in um in kind of a the way they might talk to a therapist or a friend uh and really getting something out of it because it's sort of it's already optimized for for people to be satisfied coming out of the conversation and naturally it does these things that that people enjoy so we've had people talking to it for pretty long periods of time and I think having beneficial interactions with it we have to we're going to continue to keep track of that make sure they're beneficial for people long term um and so there's a lot of use cases that come out of that anything where it's like a character a friend an NPC in a game um a ther IST kind of app although you have to be careful about what you promis there um something that keeps uh the you know customer service too to some extent um but then there's sort of the interface applications too so if you take that and you add in the ability to control things with function calling and tools um then you have an interface for uh you know we have an interface for our website for example um and uh it could be an interface for uh operating system and this sort of what this does is we're not trying to be the assistant we're not trying to give you the the give developers the tools we're trying to take the tools that developers are building for an AI to operate and give and be the interface that the person talks to that deploys those tools uh so you can write with a few lines of code or interface into your app and now it's a voice interface that better deploys the tools that you're able to operate with AI um and can talk people through it as it's doing it it's like okay I'm going to search the web or I'm going to take you to this uh you know put this add this to your cart or something uh whatever the if it's e-commerce or um I'll sign you up for something or um you know so it could do any number of things um and there's there's a lot of companies building kind of operating systems um out of this kind of technology um or hard new kinds of Hardware new kinds of wearables um robots uh and or or just like generally interfaces to an app um that you like you could it kind of borders on customer service but you could call up United and you could be like are there flights to this like my flight got cancelled blah blah blah but it can actually open up a window and start filling things out for you and find things for you as it's talking to you that makes sense and I guess for you guys like how do you think strategically about being an API versus being like having your own product or making your own product like I think for open AI for a long time they were just like research organization they uh uh they were building these models um and gpt3 came out and like some people cared but mostly no one cared and then chat gbt came along and it just like just everything just blew up for them um as you're making these choices and thinking strategically about how to get this kind of Technology adopted more widely um how are you thinking about being an API versus maybe building your own products and and being like a consumer facing company like what's what's what's in your head about that I think that the power of AI is going to be its ability to use tools and we don't want to build that we want other people to use us as an interface so that's where I think a lot of the power comes from our API on the other hand we do want to have like our demo has been pretty popular actually yeah I know it's really cool so so we're like all right let's let's make this available to people and allow people to personalize it a little bit and also maybe add in web search just as a basic tool people can use but I don't see that as like a product it's more like um an integration and a way to to to allow the end user to see what our AI is doing maybe personalize it and maybe down the road I think what would be most exciting is developers can build on our interface and they can pull in some of the personalizations that users have done um if users have access access to it as well through like an end user app and then the you know and then there's a lot of possibility there that's interesting and like how are you thinking about like you know I have friends that are building these kind of character apps right it's like you know you have you have a character on your phone I've invested in some of them you can talk to it talks back it's actually it's pretty fun it's pretty cool um for them like who are the people that are building that kind of thing that are coming to you being like hey like it's not working well enough I need this like I have this like burning need to go from the 80% to the 99% accuracy yeah it we you know there's a lot of things people want to build I think customizing the voice is really important in the personalities a lot of it you can do with the prompt obviously you can't change like the underlying accents and um and voice quality of the voice so we're adding more voices too um we're a little bit cautious about voice cloning um for obvious reasons but we want to add the ability to kind of of control the personality of it a little bit more closely um and um yeah that's like one of many requests there's a lot of things people want out of this um and so we're we're just balancing I I think that where we draw the line is like we we don't want to like build Out tool use or we don't we don't want to build like the most Frontier llms internally but we do want to build a conversational layer to make it as easy as possible for people to just insert an interface and be able to hook it up to like their web socket that does RG or whatever that that we need it to do and our interface can read it and deliver information to the user as it's doing things like that's sort of the the the goal um so that's where we draw the line but we are adding yeah things like web search uh functionality um bring your own llm um uh we're not going to do the RG but we're going to hook up to other services that do it to the extent that um that that's convenient oh and building out like packages for different um for like uh typescript and front end packages python we already have those um but other packages beyond that that makes sense um like what are you worried about like what keeps you up at night it's a good question I mean I want to really balance the um the use cases we pursue with uh sort of ethical concerns that we have with we've we we started a nonprofit with when we started Hume called the human initiative and it lays out I think the most concrete guidelines for for AI ethics that exist out there we definitely want to make sure we adhere to those guidelines I think that um some of there there's some sort of borderline applications uh where we're like like this could be good and it could be bad and I I I want to I want to see if there's a way we can do it in a good way but um but make sure that we stay true to our Valu so what would be an example of like borderline for you like um in some ways AI characters I think that um what's important about an AI character is that it should be optimized for somebody's health and well-being and not for somebody's engagement for example because if it's optimized for engagement it can it can sort of manipulate you to be sympathetic to it in ways that are inappropriate because it's just an AI doesn't actually have feelings but maybe it like makes you think it does U and it's like oh uh I haven't seen you in two days like where have you been like we don't want it to be like that right uh so so that that's that's where I I ask myself questions about how we're going to moderate those kinds of use cases yeah it's sort of reminds me of like um uh you know the David Foster Wallace book infinite chest it's like the it's the book that every every Millennial man like has read 25% of and I account myself as one of one of those people um and the whole like shtick of the book is is that there's a videotape where if you watch it like you can't stop watching basically like it's so good it's so addicting and it feels like that's the like that's the horror scenario of of something like this and I'm kind of curious like okay if you're not opting optimizing for engagement and you are optimizing for wellbeing engagement is like really easy to measure right um how are you how are you measuring well-being like I I can imagine that you have an that's like that's optimizing for it based on everything you said about these high dimension the high dimensional space that it operates in but like as a business um what is like you have to look at numbers right and you have to reduce that high dimensional space down into a set of numbers so how do you do that yeah so we have like a huge scale survey platform that we've are we just continue to collect data on and we um we've adapted that so that people can talk to this um AI that we've built and get it to you know people go in with various tasks to do or just to talk to it freely and so forth and rate how they're experiencing it um and how happy they are afterward and and and we can keep track of people's experiences over time as they use it multiple times and there's the self-report that people give us and then there's like the proxy we can derive that uh that says from people's voices and language this is the best prediction we have of their self self-reported experience along different dimensions of self-reported experience like user satisfaction or um mental health uh and so we're trying to keep tabs on that and line it up with uh what we see coming in through the demo for example um and and try to make sure that um we are optimizing for the right things but full you know like you can optimize for positive emotions generally I mean the best thing to do is like optimize for all positive emotions which we do using lots of conversational data and um and against negative emotions and also try to maintain emotional diversity so you're not just like increasing the number of cat pictures or whatever that people are seeing and um and then line that up with self-report so deploy that in an AB test and see if that's actually improving people's experiences that makes sense let me push you a little bit more though like I could imagine scenarios in which it's actually better for me to experience negative emotion um and you know I might be ignoring something that if I have that that dip for like a couple weeks and I'm I just let myself be sad overall that's like healthier how do you think about that or account for that in a system that's optimizing for positive that's yeah that's a great question I think we'll start to see um over months of time when we have consistent users ways that we can optimize for or long-term experience we already have trade-offs between you know next expression versus minutes later versus hours later um usually they align sometimes not but um over over time we wanted to we want to increase the time span so um it like the data the response now should be optimized for your well-being in a month basically it's a little challenging but it's not impossible if you have a lot of users using it consistently um and we we don't want to do this just ourselves like we're empowering developers to if they want to save their data opt in um and and and fine- tune the models over time for their users's positive experience that's really interesting and and how do you like if you're out there sort of competing with someone that's building a similar technology but is just optimizing for engagement like and it's just a little bit less moral like how do you compete um and I'm saying this like I'm I'm rooting for this is a much I like this Vision that of of the world um and yeah I'm curious how you think about it I think I mean there are companies that are doing that for sure not um as informed by emotion science as as we are so I think they're they're going to be a little bit trailing on that but um but if you just optimize for engagement what happens over time is that you run out of time in the day um so um you can't optimize for engage forever I think T people are like with with power users of Tik Tok they're already running out of time like left in the day for that crazy um and eventually you also you're you're probably like if you're if you're users are I mean our users aren't minors but like at some point like there'll people there'll be this technology will part of minors like Tik Tok and um and parents are going to be like hey like my kid's failing out of school because you've optimized for engagement and they spending 16 hours a day on your app watching mindless videos or whatever um and so I think there's ultimately a alignment between the long-term interests of the business which are like obviously to make money um but also the the the long-term interest of humanity which is like uh we won't permit AI to destroy our society um and so if you're doing that we're going to regulate you and just give you a lot of problems so so I think there is a long-term alignment between the those different objectives that makes that makes a lot of sense um and I'm I'm also sort of it's making me think about how in business generally we have to do that flattening um pre AI like we have to flatten into engagement or into like just Revenue like revenue is or like profit is just like the the you know the way that we take the cumulative sum of like hundreds of thousands or tens of thousands or hundreds of people and then decide is it good or bad right um and you're you're doing something much different which is optimizing for well-being and it seems like we we probably couldn't have done an a well-being optimization before um and I'm kind of curious like how you see that playing into the future of the way that we operate organization or build products in in general like what are the implications for how we're going to measure success I think if we to the extent that we can measure proxies of well-being we should be and we should be optimizing for that of course but um the more that people have multimodal like AI interfaces the easier and more feasible that becomes because they have the right data to do it um so that's that's where we potentially come in and we can help with that um but I I think the businesses will want to because um we're again we're running out of time in the day um the technology is too powerful to just optimize for engagement engagement also isn't necessarily the most profitable thing to optimize for um in some cases actually it's not even profitable at all like if you have a limited if you're if for example if it's a subscription model um and users are going to pay the same amount no matter how much they use the product you want them to just have good experiences so they're willing to pay for the subscription and you actually want them to get those experiences faster because you're paying for inference on your AI models so engagement is actually not the right objective there um that might actually be the case going forward for a lot of products um and yeah and also like I think that we have the right governance in terms of our board where everyone's kind of aligned on let's not just be uh psychopath uh profit optimizers um so even though like technically that's the goal of a business like in practice our shareholders and board like they they're not going to go for that that's great I love this um I've I've Loved this conversation um you're obviously just like one of the bright lights in in the industry and I feel very honored to have gotten to get to chat with you and to use the product which is which is incredible um where should people find you um and and and and Hume if they're interested in this episode and they want to learn more yeah thank you for the kind words U people can go to our website hum. they can sign up for our beta they can check out demo. hum. for the demo that we're talking about and um yeah find us on Twitter at humore a as well awesome this was great thanks so much have a good one you too oh my gosh folks you absolutely positively have to smash that like button and subscribe to how do you use chat GPT why because this show is the epitome of Awesomeness it's like finding a treasure chest in your backyard but instead of gold it's filled with pure unadulterated knowledge bombs about chat GPT every episode is a roller coaster of emotions insights and laughter that will leave you on the edge of your seat craving for more it's not just a show it's a journey into the the future with Dan shipper as the captain of the spaceship so do yourself a favor hit like smash subscribe and strap in for the ride of your life and now without any further Ado let me just say Dan I'm absolutely hopelessly in love with you