A 90-minute interview with Dr. Bradley Love, creator of BrainGPT, a large language model that predicts the outcomes of neuroscience studies before they're conducted—reportedly outperforming human experts. The conversation covers the motivations behind building the tool, how AI is reshaping scientific research, why clean scientific explanations may be a thing of the past, the difficulty of interpreting complex biological systems, and the limits of intuitive understanding of the brain. Timestamps and links to BrainGPT, Dr. Love's site, and related readings are included. # Channel Presentation Observations: "The AI-powered Era of Scientific Discovery Is Here – Ep. 25 with Dr. Bradley Love" ## Channel Self-Description Evidence ### Explicit Channel Identification - The host is named **Dan Shipper**, identified both by the guest's thanks ("thanks so much Dan for having me here") and by the outro: "a journey into the future with Dan Shipper as the captain captain of the spaceship." - The show is called **"AI and I"**, per the outro: "you absolutely positively have to smash that like button and subscribe to Ai and I." - The episode is **Ep. 25**, per the title, indicating an ongoing, numbered interview series. - The host self-describes his role: > "I'm just like some guy with a newsletter and a podcast" — indicating he runs a newsletter alongside the podcast. ## Intended Audience - **AI-interested, technically curious listeners**: The outro claims episodes deliver "pure unadulterated knowledge bombs about chat GPT." - The host assumes an audience already following the show's recurring themes: > "for people who have been listening to this podcast I I've gotten on my soap box like once or twice before" — suggesting episodes revisit big-picture arguments about AI and science. - The guest addresses audience sophistication directly: > "unlike uh probably a lot of your listeners I was never really a big tool development person" — implying an audience that skews toward AI tool builders/users. ## Claimed Scope of the Show - **Long-form expert interviews on AI and its implications for science**: The host frames the episode as "a slightly non-traditional episode because I think we're going to go deep into um science and using AI for Science." - **Scope beyond a single domain**: The guest notes the discussion has "larger ramifications um that go beyond Neuroscience." - Recurring format elements observed: guest introduction with credentials, demo requests ("can we see a demo of it"), philosophical deep-dives, and a closing "where can people find you" segment. ## Host Presentation Style (Observable) - **Collaborative and ideational**: The host articulates his own substantive thesis — that AI lets us "unbundle explanations from predictions" and "turn this like science problem into an engineering problem." - **Self-deprecating framing** balanced by guest affirmation: the guest responds, > "in this day and age like people that are placed like you are actually the ones that are really important because you're integrating all these s of siloed perspectives." ## Notable Anomaly (Unverified Provenance) - The transcript ends with a hyperbolic, tonally jarring outro ("do yourself a favor hit like smash subscribe... I'm absolutely hopelessly in love with you") that contrasts sharply with the measured interview tone. Its provenance (host, ad read, or transcription artifact) cannot be determined from the material, but it explicitly names the channel and host, supporting the identification above. # Delivery Observation: Ep. 25 with Dr. Bradley Love ## Participants - **Host (Dan)**: interview-style, playful, casual - **Guest (Dr. Bradley Love)**: professor, explanatory, anecdotal ## Tone & Register - **Conversational-academic**: Love uses informal fillers ("I mean," "you know," "so like") alongside technical vocabulary: > "you could actually just comp what uh um language researchers called perplexity which is how surprising the text is in the model" - **Self-deprecating and hedging**: frequent qualifiers soften claims: > "yeah I mean gosh so I think it's really good that nobody has that much power because probably what I say is wrong" > "I know nothing about that it just seems like the kind of thing" ## Teaching Habits - **Concrete analogies for abstract points** (repeated pattern, strongest evidence): - Dog/quantum mechanics: > "imagine explaining quantum mechanics to a dog... maybe at some point we're the dogs" - Baseball: > "if you ask a great baseball player how they hit the ball they'll tell you something but it's pretty much useless" - Abstraction layers: > "you have the machine code and then maybe the assembly and then something like C... then you have Python libraries on top" - Italian cooking as cultural knowledge: > "Italians know how to make good food... it's from like hundreds of years of tweaking things" - **Back-up-and-explain moves**: he pauses to define terms for lay listeners: > "scientific abstract just to back up for readers that haven't read a scientific paper um they tend to be structured" - **Concrete example of methodology**: > "if it was like blah blah blah anterior hippocampus we change it to blah blah blah posterior hippocampus" ## Recurring Phrases & Verbal Tics - **"I mean"** — extremely frequent discourse marker throughout - **"you know"** — constant, often multiple times per sentence - **Hedging clusters**: "probably," "maybe," "I think," "I don't know" - **Self-interruptions and restarts**: > "yeah I mean I maybe I'm so on this train I have trouble simulating it" ## Communication Patterns - **Naming distinctions**: he invents or adopts paired labels — "backward looking" vs. "forward-looking" work; "the view from nowhere" vs. "the view from inside" - **Preemptive apology for controversial takes** (repeated): > "this is going to get me in like trouble with people and I think people like hate this paper" > "after I just basically made fun of a few Nobel prizes in Neuroscience I've Got No One's Gonna like me anyway" - **Host pattern**: active listening and playback summaries — > "just to kind of play back what what you just said is..." and frequent "that's really interesting" responses (appears 5+ times) ## Uncertainty Notes - Filler-word frequency and quote accuracy are directly observable; **intent or personality claims are not made**. - Whether the analogies are a stable teaching style vs. specific to this interview cannot be confirmed from a single transcript; the *frequency* within this episode (4+ distinct analogies) is strong, but generalization is uncertain. - The closing segment (smash-the-like-button outro) is clearly a stylistic register break, delivered in an exaggerated promotional tone distinct from the interview. # Position Record: "The AI-powered Era of Scientific Discovery Is Here" — Ep. 25 with Dr. Bradley Love **Speakers:** Dr. Bradley Love (guest; Professor of Cognitive and Decision Sciences in Experimental Psychology, UCL; builder of BrainGPT) and Dan Shipper (host, identified by name in transcript). --- ## 1. BrainGPT: Motivation and Design ### Love's stated motivation - The scientific literature is growing **exponentially**; Love claims it is "just not really a human readable literature" and that tools are needed to grapple with it. - He distinguishes **backward-looking** tools (summarization, literature reviews, meta-analyses — which he calls valuable, "not to be pejorative") from **forward-looking** tools. BrainGPT is forward-looking: predicting **outcomes of neuroscience studies before they happen**. - Claimed result: off-the-shelf LLMs and models fine-tuned on ~20 years of neuroscience literature **outperform human neuroscience experts** at predicting study results ("in short... they're a lot better"). ### Why prediction matters (Love's reasoning) - **Information gain:** if BrainGPT predicts a study outcome at 99.9% certainty, running it adds no information. - **Trailblazing:** if the system says a result is unlikely but the researcher suspects a systematic bias in the literature (which would also affect BrainGPT's training data), a surprising result could be a highly impactful discovery. - **Replication crisis:** most findings "just don't actually replicate"; predictive systems could help determine "what's true, what we could count on." ### On training with flawed literature (p-hacking concern) - Love acknowledges current training does **not** filter for publication bias ("right now not so much"). - His view: treat papers not as individual discoveries but as a **"tapestry"** — like a machine-learning **ensemble**, aggregating thousands of noisy, flawed papers should preserve signal, assuming flaws aren't systematic. - He attributes many problems to **underpowered studies** rather than systematic bias. --- ## 2. The BrainBench Evaluation - Method: created a **benchmark** modeled on ImageNet's role in vision. - Source: recent *Journal of Neuroscience* publications (chosen for covering all five subsections of neuroscience, from behavior to cellular/molecular), unlikely to have leaked into training data. - Task: abstracts' results were **subtly altered** (e.g., "anterior hippocampus" → "posterior hippocampus"; "increases" → "decreases"), creating a two-option multiple-choice task comparing humans and models. - **Prompting result (GPT-4-era model):** above chance, roughly **61%** — "pretty much human level" per Love. - **Perplexity-based result (open-weight models, e.g., Llama family):** comparing perplexity of correct vs. altered abstracts yields ~**83%** accuracy; humans at ~**63%** ("superhuman... by just being able to calculate the perplexities directly"). - **Calibration claim:** the difference in perplexity between the two versions serves as a **confidence measure calibrated to accuracy** — larger gaps correlate with higher likelihood of being correct. This enables **human–machine teaming** better than either alone (follow-up in progress). - Love's explanation for why prompting underperforms: the differences between options are too subtle, scientific literature is noisy, and the task is "weird" relative to RLHF/supervised fine-tuning. - Future direction: pair generation with evaluation — generate candidate result patterns, then rank by perplexity. --- ## 3. Explanations vs. Predictions in Science ### Host's (Shipper's) position - Science has long prioritized **explanations**, which work well in physics/chemistry but poorly in psychology and "soft sciences" (e.g., no "germ theory of mental illness" after ~150 years). - Contention: **AI/ML lets us unbundle prediction from explanation** — turning science problems into engineering problems (e.g., predict depression rather than explain it). - Two possibilities he offers: 1. A good predictor's theory may exist **inside the neural network**, and networks are more interpretable than brains, so causal theories might be extracted. 2. Theories for high-level phenomena may be **too large to fit in rational brains** — clinicians operate largely on intuition. - Broader claim: we over-emphasize rational/logical cognition and under-value **intuition**; AI "makes intuition usable." - He predicts a shift toward **large-scale data collection plus predictors** as the project of science. ### Love's position (largely agreeing, going further) - The coupling of explanation and prediction may be a **historical phenomenon** limited to parts of physics. - In biology and messy domains, there may be "10,000 variables... interacting" across levels with feedback cycles — **no clean human-understandable explanation may exist**. - **The dog/quantum mechanics analogy:** quantum mechanics is well-specified but beyond a dog; "maybe at some point we're the dogs." We already build tools extending beyond native human capacities (telescopes, calculators). - Possible futures: - Different, more general forms of explanation (e.g., train a model on both psychology and neuroscience; if it predicts neuroscience better, that demonstrates the fields are related — "it's not E=mc²... but it's a kind of explanation"). - A purely predictive world, which he says is **"not even dystopian"** — "maybe scientists will be like the new priests, just interpreting things... and providing meaning to the systems." - **Qualification preserved:** Love states this is not how he wants the world to be — he personally seeks crisp explanations even in his modeling — and notes "many of my colleagues would disagree." He is uncertain whether "we're quite there yet." ### Anticipated counterarguments (Love) - Colleagues would **appeal to past successes** of elegant theories; Love observes such arguments are "not really forward looking." - He also claims scientists "look up to physics too much" and speculates that applied physics (e.g., fusion containment) already relies on ML with many interacting variables. --- ## 4. Why Simple Explanations Seduce (and Mislead) - Love's examples of narrative coherence: politicians, lawyers at trial — coherent stories transmit best. - Human evolution favored simple robust procedures in natural environments, which "maybe doesn't serve us well in science." - Shipper adds: simple causal stories may be suited to *tweaking* high-dimensional concepts, and are **misused** when treated as complete representations. - Love's complication: science is not passive — **confirmatory practice** sustains simple stories longer than they should survive; researchers' framing of "relevant variables" shapes what data they collect. ### The cell-types paper ("The Inevitability and Superfluousness of Cell Types and Spatial Cognition") - Motivation: many neuroscience discoveries (including Nobel Prizes) rest on intuitive cell-type interpretations (e.g., **place cells** as "the brain's GPS"). - Finding: deep networks placed in VR worlds — even **random networks not serving navigation** — produce the same intuitive cell types. - Interpretation: forcing intuitive understanding onto systems yields findings that "seem" meaningful but don't reflect how the system works. - Pattern cited: post-Nobel decades of research erode the clean label (distortions with non-square rooms, reward effects, path history, viewpoint dependence). - Love concedes the paper is disliked: "people like hate this paper even though it will be published." - He compares intuitive cell-type discovery to "our kids view of science" (mixing chemicals, finding a butterfly) and calls these explanations **seductive but ultimately limiting**. --- ## 5. Complexity Across Levels - Love describes himself as a **physicalist**: higher-level phenomena (psychology) ultimately have lower-level explanations — just too many variables for humans to parse. - Engineering vs. nature: engineers deliberately build **abstraction layers** (machine code → Python → apps) to reduce cross-level interactions; "evolution [and] physical reality doesn't care about this stuff," so biology remains a mess to unravel. Even engineered systems leak across layers (he cites an Intel chip arithmetic bug from decades ago). --- ## 6. Neuromorphic Computing / Energy - Love: brains do enormous computation on the energy of "a sandwich," whereas GPU data centers stress grids and have carbon impact — an argument for **neuromorphic computing**. (Host jokes about "reverse Matrix" data centers ordering Subway.) --- ## 7. What Would Love Change About Science? (magic-wand question) - Preface: "it's really good that nobody has that much power because probably what I say is wrong." - Proposals: - Train scientists to be **more philosophical** about what explanations are and their limits. - Emphasize **computational skills, large-scale simulations, large datasets**. - Teach the limits of **problem isolation**: variable-elimination that works for particle accelerators fails for human behavior and biological systems. - Insist on **interplay between lab studies and naturalistic/real-world big data** — citing his consumer-behavior work combining millions of retail transactions with lab tests — warning that isolated lab science "could just make up... a fake science of itself." ### Example of good work (Love) - **Computer vision → neuroscience:** AlexNet (built by ML researchers, not vision scientists) overnight outperformed "100 years of vision science"; computational neuroscientists then showed ANNs' processing levels track the brain's ventral visual stream. Love calls it "really humbling," while noting he has a paper criticizing how clean the story is presented. --- ## 8. Philosophy of Science Recommendations - **Thomas Nagel, *The View from Nowhere*** (not the famous bat essay): Love says its first half explains what science is and isn't — science as "the view from nowhere," a disembodied perspective with strengths (Mars probe calculations don't care who made them) and limits (much valuable human experience is subjective). - Love's claim: consciousness research often involves **scientism/overstep** — science can predict correlates (e.g., anesthesia outcomes) but "doesn't have a lot to say" about qualia — "that's probably why we still have literature and religion and music." - Also mentions **late Wittgenstein** as worth reading; explicitly disclaims being a logical positivist. ### Psychology and operationalization - Psychology is mostly science, but operationalizing constructs (depression criteria, even counting in neural recording — "soon as you count something... you're abstracting") has limits; first-person subjective experience resists the "view from nowhere." --- ## 9. Literature, Psychoanalysis, Intuition, Placebo - Shipper: literature as an exploration of psychology is **undervalued** by science-minded people; also cites the literary tradition in early psychoanalysis as underutilized. - Love: someone could do good clinical work within a framework whose **theory doesn't hold water scientifically**, because the built intuition can produce good outcomes. He is "slow to completely dismiss" alternative practices when outcomes occur, allows placebo effects are "probably real in cases," and likens cultural knowledge ("Italians know how to make good food") to accumulated practice wisdom. He notes CBT has some RCT backing. - His standing joke: after "making fun of a few Nobel prizes," no one will like him anyway. --- ## Unresolved Tensions (preserved) - **Love's internal tension:** he personally craves crisp explanations even in his own modeling, yet argues clean explanations may be unavailable in complex domains and prediction may diverge from explanation. - **Expert disagreement:** Love repeatedly flags that many colleagues would disagree with his explanation/prediction divergence thesis. - **Bias in training data:** BrainGPT's literature-based predictions remain vulnerable to systematic bias; Love offers hope (ensemble aggregation) but no current mitigation. - **Interpretability:** Shipper is optimistic models can help interpret models; Love is **skeptical** a complicated reality captured in model weights will be distillable — "the explanation might basically be the weights of the model and the training set... at that point I don't know what you do." # Speaker Attribution & Provenance Record ## Video **Title:** The AI-powered Era of Scientific Discovery Is Here - Ep. 25 with Dr. Bradley Love **Video ID:** 01a0f15b-0bb3-73dc-9b26-865f5ff0ddd3 ## Speakers & Roles ### Dan Shipper — **Host / Creator** - Identified in-transcript as "Dan" (guest: "thanks Dan for having me"). - Self-describes as "just some guy with a newsletter and a podcast." - Podcast named in closing promo as **"AI and I"**. ### Dr. Bradley Love — **Guest** - Professor of Cognitive and Decision Sciences in Experimental Psychology at **UCL** (stated in host introduction). - Co-builder of **BrainGPT**, a large language model for neuroscience research. - Personal site given at end: **bradleylove.org** (with BrainGPT link and mailing list). ### Unattributed promotional voice — **Promotional segment (flagged)** - Closing block after guest sign-off is a loud like/subscribe promo for the show, naming Dan Shipper as "captain," ending with an affectionate declaration to Dan. This is a **promotional segment**, not interview content; speaker identity is not established. ## Referenced Sources & Works (as cited by speakers) - **BrainGPT** — guest's project; fine-tuned LLMs on ~20 years of neuroscience literature; tested via "BrainBench" against *Journal of Neuroscience* abstracts (altered-vs-original multiple choice). - **Paper (guest):** "The Inevitability and Superfluousness of Cell Types and Spatial Cognition" — on place-cell-like responses in random/deep networks. - **Guest-cited works/ideas:** ImageNet benchmark; AlexNet; LLaMA, Mistral, Falcon, Microsoft Phi model families; GPT-4 prompting; Judea Pearl (interventions); Nobel-prize place cell discoveries; Thomas Nagel — *The View from Nowhere* and "What Is It Like to Be a Bat?"; Feynman (implied by "Feynman"-like physics framing — note: transcript says "fourier/fortune transforms," garbled). - **Host-cited topics:** replication crisis in psychology, mechanistic interpretability. ## Third-Party Claims - Guest reports BrainGPT-style models ~**83% accuracy** vs. humans ~**63%** on BrainBench; prompted GPT-4 earlier version ~**61%**. These are the guest's claims. ## Availability & Limits - **No curated metadata supplied** — no comment availability, no sponsor list beyond the embedded promo. - Transcript contains garbled/uncertain passages (e.g., "fourier transforms," "packing/p-hacking," "play cells/place cells"), recorded here without correction beyond obvious names. imagine explaining quantum mechanics to a dog quantum mechanics is a very well specified Theory but it's just beyond a dog and like you know maybe at some point we're the dogs you know like we're not gonna understand what's going on and we build tools we could already see beyond our eyes with telescopes and we could add up numbers we can't add you know with calculators and computers do all these amazing things so maybe like it's just going to be Beyond us and we'll have to accept it and take different forms of explanation that are much more general or maybe we're just going to be in this predictive world and it's not even dystopian like everything could be better we'll just have stories about how this stuff works so like maybe scientist will be like the new priests just interpreting things and like making it explaining it to how it how it works and providing meaning to the systems Bradley welcome to the show yeah thanks D thanks for having me here yeah I'm super excited to have you so um for people who don't know you are the professor of cognitive and decision Sciences in experimental psychology at UCL um and you are also uh building uh one of the main Builders of um a uh large language model that is focused on uh helping people do better Neuroscience research called brain gbt um and I'm I'm super psyched to have you on uh this is going to be like a slightly non-traditional episode because I think we're going to go deep into um science and using AI for Science and and sort of like how science might change as a result of AI but um yeah super psyched to have you no I'm excited too and like you hint at I think there's larger ramifications um that go beyond Neuroscience where we develop this model for that should affect all your listeners cool so like tell us like let's get started with brain gbt like tell us about what it is sure I mean um first maybe I get give some of the motivation so unlike uh probably a lot of your listeners I was never really a big tool development person um but I just saw how uh science was going and it's uh you know exponentially increasing literature literally increasing and um you know we just can't keep up it's just not really a human readable uh literature and so just kind of getting a grip on it it seems like we need tools to do that and people are making all kinds of of of tools particularly using large language models but we kind of had a different take on it so there's a lot of great work that we call like uh backward looking which is not to be perjorative it's work involving summarization of the scientific literature kind of writing instant reviews or almost meta analysis and cases and that's great and that's valuable um but we want to focus more on like what I think is really important in science namely prediction so what we call forward-looking and uh can we actually predict the outcome of studies before they happen and um so we just really have in this project and I'm happy to dive in as far as you like but we wanted to see can large language models both just off-the-shelf models but also models that we fine-tune on 20 years of the Neuroscience scientific literature can they actually predict the results of Neuroscience studies experiments uh better than human experts like professors of Neuroscience I mean in short there's a lot to say but in short they they can they're a lot a lot better that's fascinating wait so just to go back to the original motivation it sounds like the the first thing was like okay there's like way too much information there's way too much science being done for anyone to keep up um but it sounds like the thing that you're building brain gbt isn't necessarily about summarizing the research it's about predicting what future research might might hold so like what's the connection between not being able to keep up with the literature and kind of like future research predictions sure sure that's that's a a great question because I mean I think to predict the future you have to have some understanding of the past so not necessarily like a nice clean text that's summarized but you know a model that could draw on thousands of findings through different literatures at different levels because you know like Neuroscience is multi-level it has everything from psychology and behavior all the way down to really low level cellular molecular findings things involving DNA and so forth and so I mean no one could really draw on all that information but it might be that biology is really messy it's not like how computer scientists create these abstraction layers we have the hardware the software and so forth uh so it to make a prediction you might very well need to draw on all that information and why do you want to make a prediction because I mean well first there' been there's so so many uses uh in science for this so uh you maybe want to run a more informative study so you if you know brain GPT predicts that your study is going to work out 99.9% certain as you expect there's really no information gain in that there's no reason to run the study by the other on the other hand you know if you the system says oh no this is unlikely to give the P results you expect but you you have an intuition that the literature has gone off course and there's a systematic bias you know that same bias is affecting what brain GPT is trained on then like in some sense you're like trailblazing you're making a really impactful Discovery um so really everything getting into reproducibility is a huge and replication is a huge issue in science like most findings just don't actually replicate and so I think we could in the future very near future use systems like this to kind of get a handle on what's true what we could count on what next step we should take and scientific discovery yeah is it like when you're training it how are you taking into account the fact that like you know pecking occurs and like maybe the literature is like somewhat damaged like if are you training in on literature how do you filter that out I guess yeah I mean right now not so much but that's something that's uh definitely of of Interest so I mean the way I see it I mean I don't think scientists see it this way I mean I'm a scientist but um most of them most people think of papers as like individual contributions or discoveries but I think of each contribution as as even if it's not PW is really flawed noisy incomplete but you have this like tapestry of thousands of papers and so hopefully if you just aggregate it's almost like in machine learning if you do an ensemble so it's like an ensemble of everybody hopefully has the signal in the correct direction um I mean that's why I raised the possibility before that there could be systematic biases or issues but I think a lot of the problems just really come down to under underpowered studies like statistically so and that that opens the door to packing or just you know a careless mistake and so hopefully there's not that many systematic flaws and if we could just not start reasoning about individual papers but about thousands of papers at once I think uh you know we'll get the signal that's that's really interesting and and you said it it predicts better than like Neuroscience experts like which studies are going to work and which won't like what are the boundaries around those kinds of predictions like what is it good at predicting yeah yeah so the way we tested it I mean kind of taking um our que from what people do in computer science and machine learning where they make uh benchmarks you know like the image net Benchmark was really critical to developing computer vision models so we made our own brain bench model and um what we did is we looked to the Journal of Neuroscience which is um kind of a standard well respected journal and Neuroscience and the reason we chose it is because it gets to your question that there's it's really covers all of Neuroscience there's five subsections to it and it goes from everything from cognitive behavior like pretty high level stuff to like cellular molecular systems to developmental Psychiatry type stuff um and so we our mark would have these five subscores and what we did is we took uh recent um Publications from from this journal that are unlikely to have leaked into the training sets of models and we trained some models from scratch where we know what they're trained on uh but what we did is we just subtly altered these abstracts so scientific abstract just to back up for readers that haven't read a scientific paper um they tend to be structured that there's a bit of background at first a couple sentences then um you know like the method what kind of experiment was run and then the result and so we just altered the result in very subtle ways like kept the kind of linguistic flow so if it was like blah blah blah anterior hippocampus we change it to blah blah blah posterior hippocampus and so basically just or something was this brain activity increases made it decreases and so we just so it was a multiple choice basically with two options and we tested um basically neuroscientists and the um a whole of large language models including some that we fine-tuned on Neuroscience literature and just compared their accuracy that's really interesting um and I guess like for you like what is the um what do you think is the current sort of Frontier for Neuroscience like what is the what are the interesting problems right now and how does how does brain gbt fit into that yeah I mean there's there's so many things I mean in some ways like you know Neuroscience has been around for a while but in other ways it's still like a really young discipline say compared to physics and you know some people could even say it's almost like pre sort of there's no standard model or anything like that in in physics so pre paradig pre Paradigm yeah exact yeah exactly that's what I was that's what I was grasping for thank you um so there's still like I mean a lot of what I do in research I do like Empirical research modeling but I feel like a lot of time I'm just trying to figure out like what the question is or how to frame things uh but you know people are I mean there's there's there there's so many questions and some of them are relevant to I mean it goes from really high level stuff to really lowlevel stuff like how I mean we all think like you know we have synaptic change and that's the basis of learning but even that's up for grabs and how that works even at the very low level like how is a memory encoded and and the brain is it in sort of the synapses the weights as we do in neural networks is it something more internal to the cell uh how does air uh propag from learning so that'd be relevant to like you know deep learning does the brain do gradient descent does it do something else and this high level stuff like is like when we understand space is that the basis for higher level mental Concepts like freedom justice or just you know chair or is it like just a general learning mechanism so there's all these issues that go from very low level to high level it's such a huge field like I can't really say there's there's one issue and I kind of gravitate towards the ones that might have a bleed over or transfer into a machine learning personally and and what are the ones that you think are like the most like kind of ble bleeding over into AI machine learning yeah well I mean one that I don't work on that's seems like an obvious candidate is just like transferring a power consumption ideas you know so like I mean modern gpus are amazing and they power Transformers which power AI Revolution and other Technologies but of course like you know these data centers are going up every day and it's stressing the Grid it's there's carbon impacts and and so forth whereas like um our brains are doing a lot of computation but I guess you just have to you know eat a sandwich or something and it's a lot less power consumption so that's sort of neomorphic computing applications I'm just excited for like a world where like Microsoft data centers like they just like order a big big load of Subway you know and it drops off yeah exactly yeah it's like a reverse Matrix or something like just just give it a salad it doesn't use us as batteries but um yeah that's really funny yeah but possible I mean there's there's so many things I mean I I'm really interested in the higher level stuff so I mean I still think where people are better is like somehow we have some tricks for how we represent the world and tasks and promote generalization I mean in some sense why we're so excited everyone so excited about large language models is that they're like you know base or foundational in some sense that you could apply them to other tasks not just the tasks they were trained on whereas you know previous generations of machine learning models even the great like you know convolutional models that somewhat cracked object recognition or or like Alpha go uh you know doing you know doing its games and so forth like um those are um you know specialized models and I think people are still the the kings of that flexibility so there's probably sub secret SAU to gain still from humans and how we represent situations and generalize and Link things up that's really interesting um wait can we see a demo of it like do you have a a way to like use brain GPT that we can look at oh you know it's um actually so show there's a lot of like good guests that do interesting things with prompting so this is actually a failure of prompting which might be even more interesting your guests and give them a new way to interact with large language models so it's not going to be visually exciting because it's just going to be me talking but uh maybe it'll be like informative so we tried um doing kind of what you would expect we would do like you know firing up gp4 which we did back in the day like the previous version and saying give it a few prompts like Hey we're g to give you two versions of you're a neur pretend you're a neuroscientist we're going to give you two versions of uh scientific abstracted Neuroscience one of them is true one of them is altered which one is the original so it was above chance at that but it's like my memory is bad maybe it's was like 61% so that's actually like pretty much human level but if instead instead of interacting with the model you actually just have access to the weights which you don't um for gp4 but you do for like llama family of models uh for a lot of other models from U mistal uh free models and so forth the Falcon models there's a lot of models out there where there's access the away and Microsoft Fe models a smaller accessible model uh with those models you could actually just comp what uh um language researchers called perplexity which is how surprising the text is in the model so it's like given what the model has been trained on so it's like basically you know eating all the scientific literature and that's set all the weights how how aberant is this text passage and so if you do that it has two real advantages the first one is it's just way more accurate so you could it'll have a lower perplexity for the correct version than the incorrect version but what's also really powerful is that you could take the difference in perplexity you know how surprising these two passages are and you could use that as a measure of confidence in the model and it turns out that confidence is calibrated to its accuracy what so when the model is more confident when the difference in the perplexity of the correct incorrect passage is larger the model is more likely to be correct so that's really important for human machine teaming because you could put a human and a machine together and you could get a better result than either one alone so we have some analyses like that we're writing up now for a follow-up publication but basically it's interesting why the prompting doesn't work I me I think the issue is that usually in multiple choice the difference between the options is pretty large whereas here it's really subtle like that example I gave anterior posterior increases decre it's pretty like subtle and I think um scientific literature again is noisy and imperfect and I also think this task is not like u a lot of its tuning that it was given to be conversational like it's reinforcement learning through human feedback or other supervised fine tuning I know these models generalize really well to task but I think this is still like kind of a weird task for the to do even with a few examples of prompting so it just turns out you could really to me it's like a pattern recognition problem if you could just like look at all the models weights and get its perplexity out they do just they just do much better as a matter of fact they're like in the 83% range um so you get like a 20% bump and like humans are only at 63% so they get sort of superum Good by just being able to calculate the perplexities directly that's really interesting so like um just to kind of play back what what you just said is um rather than like prompt the model and be like I'm going to run this experiment what do you think what you do is you like write a future Neuroscience paper and then you run that through the model and and calculate its per perplexity yeah I mean you got the total um idea what we do is like so much simpler like we have these two abstracts like sort of the original version and the altered with the one with the altered result and we just get a number for each one like the perplexity how surprising is the original one how surprising is the altered one and the model just chooses whichever one's lower but you could do what you were doing like this is what I kind of see the I think you're like already seeing where this is going because I think for scientific discovery to me this is the first step you have to be able to predict like is it did this experiment go this way or that way and if you did that you could start doing more interesting things like you're saying and build tools on top to say like you know what experiment should I run next and so forth or is this plausible like would this work out out how I think and you could even pair that with generation because you could have the model in prompt mode generate different patterns of results then you could use it actually to evaluate through perplexity which one is most likely you know to be reasonable that's really interesting yeah this is this is sort of like I think an interesting dovetail in like into like how does how might science change with this kind of with these kinds of tools and um for people who have been listening to this podcast I I've gotten on my soap box like once or twice before about about what I think um and I'm sort of curious to like uh to do to to like explain it a little bit to you and then kind of like get your take as because I'm just like some guy with a newsletter and a podcast no no I think in this day and age like people that are placed like you are actually the ones that are really important because you're integrating all these s of siloed perspectives and linking things together like I think it's really valuable and I mean you're being humble but you should give yourself more credit so I want to hear what you have to say thank you I appreciate that um well I guess like the the the place I usually start is um science seems to have been like really obsessed with explanations for a really long time like basically since science was invented um which makes a lot of sense because explanations are ex extremely powerful for making predictions and they're also like kind of beautiful to understand how the world Works um and um it has like finding scient scientific explanations has worked like really well in lots of areas of science like physics or like uh chemistry stuff like that um but there are lots of areas of science like psychology um and the quote unquote soft Sciences where like actual causal um parsimonious scientific explanations are like really really really hard to find um and that's what makes studies hard to replicate and all that kind of stuff and you know if you look at the history of psychology we've been trying to figure out what depression is or anxiety is or whatever for like 150 years and like we have some different lenses on what on what it is but like there's no uh we don't have like The Germ theory of mental illness for example like we just don't actually know what it is and there's lots of different competing ideas um but we keep looking for explanations because um we feel like that's the only way to make good predictions about what to do and my kind of contention is that um machine learning AI approaches allow us to kind of like unbundle explanations from predictions so we can get good predictions without having to explain anything having any good theories about what's underneath um and so we sort of turn this like science problem into an engineering problem which is like instead of explaining depression we just predict it um and if you can predict it one is you can start to help people so you can predict when it's G to when it's going to happen to someone you can predict what interventions are going to work um and then two um if you if you've developed a good enough predictor for that kind of phenomena in the world like theoretically the the theory um exists in the neural network somewhere and neural networks are more uh interpretable than human brains are in general and so you may be able to actually like find a good causal theory of depression inside of a neural network um if it's if it's sufficiently predicting which I think is like really cool or what you may find which I kind of feel like is probably the case but I'm curious what you think um what you may find is that um the theories for higher level phenomena like depression um are so big that they don't fit in our rational brains um and and so that's why like a good clinician has a good int intuitive sense of what's wrong with someone and can kind of like help them you know help them uh overcome their depression but like and they can give some sort of explanation for what's going on but like really a lot of stuff is just happening subconsciously um and so I think like the larger impact of all of this is like we may start to realize that um we put too much emphasis on our like rational problem solving logical brains and not enough emphasis on our intuitive brains um and uh an AI may help us like kind of re re realize the like power of intuition because AI like makes intuition sort of like um usable um in the same way that like like logic you can write down logical AR arguments you can also build an AI predictor that has human intuition in it which I think is like really really cool um and it may also change How We Do Science so like instead of doing small scale studies uh we will uh just try to do much larger larger scale studies where we're like getting lots of data together and that's like the project of Science and then we build predictors on top of that so that's a very very long spiel there's a lot in there but I'm sort of curious what you think no I mean I I I largely agree and could even see taking what you say further I mean certainly like I've been trained as um like a cognitive psychologist and kind of more migrated to machine learning so like a lot of things we do are exactly like you described so it's not if you ask ask um a great baseball player how they hit the ball they'll tell you something but it's pretty much useless and doesn't describe what they do at all which is really and one probably a reason why a lot of great athletes end up being like terrible coaches and managers there's not really that strong of correlation and um but yeah I mean I might take it further so I don't think it's a hard versus soft science thing I I think it this really nice coupling of explanation prediction like I fear it might be a historical uh phenomenon and it really only applies to some aspects of physics because you know you could do things that I don't know if I mean not even psychology just in bi in biological sciences like everything is so messy it just might be that there's 10,000 variables and they're all interacting and these variables are at different levels like we're saying from DNA to behavior and there's all these feedback cycles and delays and that just might be what it's like and if that's what it's like that's just not going to have a clean explanation that will be human understandable and so at that point it almost becomes like storytelling to have an explanation and the explanation and the prediction much like hitting the baseball or you know it's going to diverge more and more with time unfortunately again this is not how I want the world to be because I was always somebody that would try to come up with the crisp explanation even when I do modeling I don't want to just say the model worked I want to understand the principles behind behind it and how they interacted to give rise to the behavior of the model um you know have understanding intuitive sense of understanding and a solid scientific explanation but I don't know if that's our our future um you know I Ed this joke sort of like you know I mean imagine like explaining quantum mechanics to a dog you know it's like quantum mechanics is a very well specified Theory but it's just beyond a dog and like you know maybe at some point we're the dogs you know like we're not gonna understand understand what's going on and we build tools we could already see beyond our eyes with telescopes and we could add up numbers we can't add you know with calculators and before know computers do all these amazing things so maybe like it's just going to be Beyond us and uh we'll have to accept it and take different forms of explanation that are much more General and much more about like well we built the system and we put these constraints on or for like taking it back to brain GP it could be like as psych it might be really General stuff like a psychology in Neuroscience related fields does knowing about Behavior help predicts about Neuroscience well I don't know let's train a model that has both Psychology and Neuroscience and it does it predict Neuroscience better okay then those fields are related that's a kind of explanation but it's not like you know like um equals MC squ or something it's not very clean or so maybe we're going to have stuff like that in the future as explanation and or maybe we're just going to be in this predictive world and it's not even dystopian like everything could be better but like we just really we'll just have stories about how this stuff works so like maybe scientists will be like the new priests just interpreting things and like making it explaining it to how it how it works and providing meaning to the systems but yeah I I don't know I I don't think I don't know if we're quite there yet and many of my colleagues would disagree but I could see a world in which like science um explanation and prediction unfortunately diverge and it's not because anybody did something wrong it's just because the world's really complex systems are complex and uh we're we're not just our brains aren't built to make sense of this kind of stuff what is what do you think is the like if one of your colleagues that would disagree like what what would be like a a reasonable counter to this like uh this Viewpoint yeah I mean yeah I maybe I'm so on this train I have trouble simulating it but um I I think a lot of things would just be you know appeal to the past and really elegant theories that um unlocked understanding and explanation and I mean I'm I'm I could even point to things I've done myself that have uh that flavor but a lot of the arguments that I come across they not really forward looking they're more like in the past it was this way but you know in the past we didn't have I mean we're having increasingly more large data sets available and there's more and more you know dat different types of data to link together not just in the commercial world but in the scientific world like massive DNA databases and disease databases and brain recordings from the same people and um and the scientific literature just gets bigger and bigger and bigger so it just seems like you know it's sort of like a a Sensor Fusion problem at some point of connecting these dots so I again I don't think that was like what it was like in the in the in the past but I mean I guess first yeah so I think also scientists tend to like uh look up to physics too much and it hasn't worked that way there and not being an expert in physics i' I'd be surprised if many parts of physics aren't already like what I'm saying because it's probably at some point things are going to get many many variables complicated there you know so that not everybody's trying to do the grand unified theory you know most people are like I don't know I me I'm guessing if something sort of veers into the real world like trying to build a uh a containment field for for Fusion or something I I bet there's tons of machine learning going on there and again I know nothing about that it just seems like the kind of thing where it's going to get complicated there's going to be many variables to balance and time series and it's not going to come down to like some simple equation with a square in it or something yeah that that yeah I think I think that makes sense I guess guess like what I'm sort of pondering right now is like I obviously like feel like there's like we're we're sort of approaching the world where like maybe we don't jettison explanations entirely but like our emphasis shifts a bit particularly in science from like like clean explanations to like the stuff we're talking about like predictions and stuff like that and and using different models to like draw different conclusions which means we're not necessarily always getting down to the Brass tax of like we know how every causal interaction happens um but like we can generally know what is going to happen or you know all that kind of stuff um but I guess one of the things that um that your dog analogy reminded me of is just like how how do how do we make a world where that is true that's also not a huge bummer like like I I feel like there's a lot of interesting things about it but like it also I can imagine like thinking like thinking towards a world like that and being like I guess we're like just fundamentally limited like that sucks you know what do you think yeah I mean I don't I'm more positive I mean I think for people I mean first for most people and like not again not denigrating most people if they're healthier uh they economy grows the climate doesn't collapse and so forth they're probably going to be pretty happy with h the future if we have a future where we could predict you know disease and uh how to you know have sustainable energy everything you know if if if these predictive tools unlock those kind of discoveries people most people be happier but people I guess you and me like we want to actually have some insight into what's going on and I guess there it's just going to maybe be different forms of explan I remember as an undergraduate when neural networks are really out of style and learning about them and having fun you know programming up from scratch and see my own little back prop Nets and stuff like that and didn't really understand what was going on the weights but I almost felt there was a different kind of understanding or something Deeper by this kind of system could do this problem so there might be that kind of explanation exactly like what you're saying where you can't like trace it all out you might even be able to do with the Judea Pearl type people that really emphasize interventions in systems like tweaking I know like zapping this neuron or or changing something or doing an intervention and who knows in our climate and then being able to predict how that Cascades forward I mean lest I don't know maybe the kind of explanations would be the kinds of variables that are involved some gross characterization of the system and the Dynamics and it'll be just like you're saying not U charting out the whole causal uh Network because that just might not really be something thing that's that's possible and it's probably not even what the machine learning systems are doing so I don't even know if you could distill it like um yeah so I yeah I just think maybe what we accept as an explanation could change over time too I mean people change and our expectations of I mean how we understand the world must be so different now than 500 years ago uh what would what we would find satisfying um so maybe it's just going to change again and and we're worrying about something that people in a 100 years from now just won't even think twice about yeah I I I kind of think that that's true like I do like you mentioned the idea of like okay how do we tell if if two fields are related well we we run it through a model and if it says it's related then it is and that's like maybe that's not satisfying to us today but it will be satisfying to like a future generation of scientists and I think the conditions under which that would be satisfying is like we have a model that is like so standardized and like well used that everyone has an intuition for like what what it is and trusts it to some degree um and in that case like getting new results from that model I think will be like fascinating and really interesting in the same way as like finding some like underlying causal Network or something like that like um it's pretty clear we're moving to a world where using models to judge things is like the a really important way to make progress in science but also just like in AI in general like we're kind of like reaching that point too where in like mechanis mechanistic interpretability like it's so complicated that you have to use models to like find Fe features of models and and all that kind of stuff and I just think that that feels like a place that's pretty we're moving towards yeah yeah it could be I mean obviously there's tons of really smart people working on interpretable AI both building it in from the start but also kind of in a post examination of these models so even if we're moving towards that world maybe along the way that will help us transition or you know maybe there'll be a breakthrough but I'm just skeptical because if the underlying reality is so complicated and we have a model that appreciates that underlying reality then the explanation might not might basically be the weights of the model in the training set you know so at that point I don't know what you do but maybe we'll have some you know purchase um some some transition and obviously that works important and when we make these models we we're never going to blindly uh trust them so we're going to need all kinds of checks along the way and um mitigate bias and so forth but yeah I think we're going to that world just because that's the world that things are going to get done in yeah yeah I guess I'm curious like you referenced I don't know like a couple minutes ago like one of the reasons this is so hard is is is maybe the way reality works is it's like 10,000 variables weekly interacting and that's really hard to create a prediction around or sorry explanation around can you can you explain what what that like why that is like what that means where where you see that um like I'm assuming you're talking about for example like genomics or something like that yeah I mean I think everything or like why we're doing what we're doing right now in this um conversation just everything is is that way I mean even in in physics I mean you think of like everything's like this ideal idealized spiral you know particle and some vacuum or something and not that that stuff would even work in the real world when you have more variables like the wind resistance the friction this and um the material stressing and but um yeah so I just um sorry could you maybe I think I got lost too many variables could you even like repeat the question I mean I just think everything is is has this flavor to it you know when it gets closer to the real world but yeah so by I mean a lot of the issue to is like anything biological it's probably Beyond biology So like um you know even in social sciences and economics you have all these interacting elements and like in an economy and that's just incredibly complicated uh but in biology of course you have the cell itself is really complicated then you have all these interactions there and and history and you mentioned DNA and how things you know how Pro how the this you know proteins are expressed but also then what you could think of it also I mean I think every I'm like a physicalist from a philosophy standpoint so I think things even if you have to study higher level things like psychology Neuroscience ultimately there is like this lower level explanation it's just again too many variables humans can't make sense of it but you know you could think of like all our goals and what we do has also affects what goes on in our in our brain and like how things wire up so there's just so many interactions across the the levels and if you think of what Engineers do they do go to Great pains to reduce those interactions but you know Evolution physical reality doesn't care about this stuff but you know when an engineer makes abstraction layers you know you have the machine code and then maybe the assembly and then something like see your fortune then you have python libraries on top and I'm not doing a good job describing this and you have application then like younger people day a lot of them don't even know there's a file system on the computer what a folder is because it's like so abstracted I mean which I guess is great but you know it's sort of like no one knows how to change the oil in their car or anything we've extracted away but um but we build the things like that and a lot there's a lot of effort that goes into that but I don't think nature or biology um respects that so it just makes it kind of a mess to un unravel and of course even with our systems we have bugs and weird things happen like I remember decades ago the Intel chip you know screwing up um arithmetic or something from some Hardware bug and it's just like really hard because it's spilled into other layers you know in the abstraction hierarchy yeah and I think you you've thought a lot about like um the ways that trying to make natural interactions uh or natural processes human understandable like actually might cause us to misinterpret them um you have one paper in particular called the inevitability and superfluousness of cell types and spatial cognition do you want to talk a little bit about that like introduce what the paper is what the motivation behind it is and what the results is that you found yeah yeah it's really really like a lot of our conversation is kind of leaning this idea so for most of Neuroscience a lot of the major discoveries including multiple Nobel prizes have been somebody recording from a cell either in an um human or a non-human animal and basically having some um intuition some clear explanation like for example these just things in the brain called play cells and they light up when a rodents at a particular location in its in its box they're like aha this is like the brain GPS system and that's very intuitive and appealing but it's very also like simplistic because the situation the animal ones is very like contrived and we're kind of forcing this interpretation and so what we found in this paper is many of these intuitive cell types that you could just interpret them and it seems like oh the brain's so simple it has when you do this this fires and when you do this this fires and you could make sense of this in a very um human understandable way what we found is that when you take larger um networks like deep Networks and you put them in VR worlds a lot like the experiments neuroscientists run on uh non-human animals that you get these same cell types popping up even in random networks that don't serve the function of navigation or localization like the play cells so it's really like kind of a danger I think of um trying to force the intuitive understanding on the system um it's just like you'll find it but it's not actually how the system works and even within the field where people believe in these um kind of easy to label intuitive cell types what happens is there's this General pattern where someone will say Eureka I found the play sell and they'll literally get a Nobel Prize and then there'll be 30 years of research explaining how it's not really a place sell like oh well if the room isn't perfectly BR then it you get like this Distortion oh if there's a reward like food then it doesn't code for that oh it depends on the history of the path oh it depends on the Viewpoint somewhat and so it's like maybe we'd actually understand what was going on more if we would do kind of more of um I don't know respect the complexity of the system we're dealing with some ways do something a little bit more bottom up still having mechanistic explanations and models but I mean this stuff is just like so complicated and it seems just ridiculous it's almost like a again this is going to get me in like trouble with people and I think people like hate this paper even though it will will be published and have impact um it's almost like our kids view of science like oh I just mixed some chemicals and something happened and I tell people about it and it's a discovery or oh oh look I found this butterfly no one found this before that's so important but that's not really like what scientific explanations are like and it's just the when you're dealing with a system as complex is the brain and people and so multi-level it just seems so Vanishing unlikely that whatever idea you go in with is what you're going to come out is going to be actually true or there's no unless there's some reason for like you information to be coded that way it just so I think there's this is a problem because those are the explanations people find attractive but I think they're very seductive but ultimately limiting that's really interesting um why do you think like given that we like those kinds of explanations what would you guess is the reason like are we're just attracted to kind of those like sort of parsimonious like that's the way our brains work like what like what is it about those kinds of exclamations if if they must work for something right yeah yeah I mean again like this is where I'm glad I have some training as cognitive psychologist but everything I'm going to say is going to be so obvious I mean it's just like if you're an if you're a politician running for office you know Simple Story coherent narrative works best it's like what's going to have the best information transmission value it's going to be most viral you know if you're in a what does a lawyer do in a jury trial they create a a narrative where their client is like innocent that that's a coherent story so I think people really prize this kind of coherency it's just like what makes um an appealing explanation to us but um but again some things even things that are actually pretty simple like in terms of just writing out what it is like quantum mechanics that makes no sense intuitively and that's pretty simple but then I think when you get into things like again anything involving the real world or biology it's just going to be so many variables you could tell the clear intuitive story but I'm not even sure in cases it'll be a approximation of of the the real thing or it'll be like so crude and off it'll um obscure Miss uh like deeper deeper truths but yeah I just think I mean that's I mean we like that I mean in some sense it's good it's almost like aam's razor for some things you don't want needlessly complicated explanations but you know and sometimes you need it sometimes you need the more complex model to you know fit the data make the prediction characterize the system and um I don't think PE humans just go in by default thinking things are that complicated and we didn't really like I mean I don't want to try to guess how we evolved but like you know we didn't really live in a world where things had to be that you know like they weren't like the quantum world the 10,000 variables of how the brain works uh Big Data World um so probably having simple robust procedures serves us well um in our natural environments but maybe doesn't serve us well in science yeah I guess that's maybe what I'm maybe what I'm thinking right now is like um the brain has like really um High dimensional representations of things in the world that are very complicated and would be very hard to like have a scientific explanation for and like our natural inclination towards like sort of simple causal stories that we can tell maybe like the the proper place for those kinds of things are to like slightly tweak our much higher dimensional like concept um for a particular con context and like we're kind of like misusing that to try to like um blow it up into like it it it representing the entire high-dimensional representation that like really we can't put into language um and uh like we've been sort of trained that the more parsimonious and and and Powerful those little things are the better they are um but like that sort of gets misused when it goes beyond what they're capable of of representing yeah yeah I think you're right and it gets even trickier because of course the we're not passive recipients of data in science we go ahead and collect it so once you start believing something a story you just basically keep it's there's a lot of confirmatory work in science so uh it's so you'll just basically if you have that mindset about the relevant variables are even if you don't think you're being confirmatory you kind of are confirming the the framework so I think it's what you're saying but we even do things to keep those simple stories going probably longer than they should yeah I guess like if you could this is a this is a big question so like you know take a second to to think through it but like if like if you could sort of like wave a magic wand and like all the priors of science were thrown out for a second and you could like rebuild science as an institution like knowing what we know today about how hard this stuff is like what are the things you would change and like how would you rebuild science um for this world yeah I mean gosh so I think it's really good that nobody has that much power because probably what I say is wrong but probably what um other people say is wrong too um personally I would do a combination of training people like this conversation to be a little bit more philosophical about things and maybe do some reading and thinking there about what explanations are and the limits and the study of it but also like more emphasis too which is already happening on computational skills large scill simulations um and the fields going this way too but more consideration of naturalistic environments and their complexity and how there should be like whole whole like courses and science for University students about how eliminating the problem that might what work well in like a particle accelerator in physics if you're working with the standard model but probably everything else you're going to miss uh key variables and insights I mean especially anything involving human behavior or biological system so I I guess really just kind of a different orientation for scientists have them be a little bit more thoughtful and philosophical about the whole Enterprise and just like everything's going to be AI computational large data sets even if there's a beautiful experiment that can isolate something for a key question that you can't resolve with these large data set so something when I did some work with consumer Behavior to look at human decision- making in the real world like in retail settings and there was it was like doing these big studies with like you know millions of PE transactions but then we' bring it back into the lab and try to we test something so I think emphasizing that interplay I think if you're going to run lab studies I think there has to be the interplay with something more naturalistic real world big big data just so you don't just make up like kind of like a fake science of itself because you could have a science of of anything and it might just not have anything to do with anything anyone really cares about ultimately like in 100 years yeah who do you think is doing like good work on uh like in on this along this vein like in your field or in other fields that you know about oh yeah I mean so I mean a lot of people are doing um amazing work and I'm I'm very like self-critical and critical of other people but luckily I don't um share it with them so so I probably won't be gred like naming a bunch of people but I mean one maybe you use one whole whole subfield is um I mean years ago when I was in graduate school like the computer vision they're mostly guys but they were like the the big jocks with the feel like oh they had the most mathematic chops and they were doing the real science with all the filters and fouryear transforms and and really leading the way and then um Along Comes Alex net like this convolutional network not made by uh a vision scientist um but by Machine learning AI researchers and overnight that in terms of behavior that was the best model of object recognition so what happened is neuroscientists computational neuroscientists took that on as a model for the vental visual Stream So how our brain you know transforms our Vis images on our eye into like that's a chair that's a cat and they found that the it's not a clean that perfectly clean story like they present and I have actually a paper criticizing this the basic story but I think it was a real advance to show that the levels of processing in these models these artificial neural networks uh fairly well track the Transformations that went in our own brains and so this is an example where going larger naturalistic training this model on a million natural images basically taking the whole problem on at once and doing something at scale led to a bigger advance than like 100 years of vision science and all these you know grants for this and um you know it was done by neuroscientists but was really kicked off by engineering and embracing this real world challenge and I think that should be really humbling to uh to scientists and so I don't want to single out anybody but there's a number of people that did that and scientists are very um almost like fattish like if one person has a good idea they all kind of rush to the same thing so there's been a lot of really smart people that have jumped on that train and now it's like sort of petering out and it's like the Next Generation questions are happening but yeah so that that's an example that kind of goes with what we're discussing that makes sense and then in terms of like you know wanting scientists to be a little bit more thoughtful and philosophical about like what the methods of science are do you have any like um people or writers or you know philosophical ideas that you think are like like they should be reading or we should be reading even even if it's not like for Lay people just like in general like what what who are the philosophical people that get you excited yeah I mean gosh uh so I'd say this this book that I read recently it's an older book by um Thomas Neel who's a philosopher of science that recently passed away it's called so he's famous this is not what I'm recommending it's a good essay but he's famous for what's it like being a bad essay where it's just sort of about subjective experience and how you couldn't really know what it's like to be a bad because you'd have to be a bad and then when you go back to being a person one but he wrote this book the view from nowhere that um even though he's an incredibly famous philosopher I feel like it should have way more impact because the first half I think really explains what science is and what it's not and I think a lot of scientists people that seem like they're being more in inclusive in some way of topics like this probably applied to conscious research um in some ways they're doing scientism in an overstep so the reason the book's called the view from nowhere is because that's what science is so there isn't science for you or me or there isn't like Eastern science or Western science it's like you know from this disembodied perspective and that's has both strengths and weaknesses you know so if you launch a probe to Mars and put it in order like Mars doesn't care who came up with the calculations it's either G to make it or not make it and so that's kind of the the view the view from nowhere but of course a lot of Human Experience that's very valuable is subjective and that's like kind of where meaning comes in our our lives and so what I liked about that book is making clear like what the strengths of science are but in some ways he doesn't like you know hit this over the head with it but for me what I Really Got Away was the limits of science so I really I really likes um that that book that's really cool I I love that how does that how do you think that that um relates to I mean you're you're an experimental psychologist so like how does that relate to like the field of psychology which obviously is like it's trying to be science but then there's also like it's also all about Human Experience yeah yeah so like again I mean I mean another person be good to be read would be like late bonstein so I'm not trying to be like some kind of logical positives that you could only rely on the observables and whatnot but I think um I think there is like some some overstep so a lot of times so psychology like like in terms of like experimental psychology um you know most people I would say are are scientists and are doing science but like there is like sort of this this this over overstep at times because you're really Limited to what you could I mean it's is just what you tell an undergraduate what you could operationalize so like what's depression well you have to like come up with some criteria for it or like what's this or I mean even we do it even when you're doing neuro recording soon as you count something you know you're saying that there's you're abstracting in a sense you're saying there's some way in which these are alike and you're ignoring other ways in which they're um different so like that's kind of like the standard I mean that's sort of again like the view from nowhere trying to like make a procedure where you could just observe things and quantify them a away where you could start testing out hypotheses but a lot of times I think psychologists and people in general get um neuroscientists others get confused that they could have like some deep insights into our subjective experiences or like what people call like the qual of you know why does the blue make me feel this way and when you start getting into that like you could you could come up with like you know very good things like if I put you under this anesthesia will you wake up or not you know under what conditions what what what you know brain waves can you measure that will predict that or account for that or even some mechanisms but um I think a lot of these like when it gets too into um not the view from nowhere but like kind of the view from inside this first person perspective which is mostly what we care about is like creatures like I I I mean I don't think actually don't think science in general has a lot to say Beyond correlates of of these experiences and that's probably why we still have literature and um religion and music and all these arts and all these other things yeah I definitely like I mean I feel like uh uh literature as an as an exploration of psychology is like quite undervalued um by people who are into science like I think some people think about literature and psychology as being being very intertwined but like um as not a professional psychologist myself but one who has like read a lot a lot a lot of psychology books and also has read a lot of novels yeah like there's something there's something about that or like even the the like literary tradition in psycho analysis where it's like a lot of this the earlier work is like less about like okay you have these cognitive distortions and like we're going to do a worksheet or whatever but it's like really really long kind of like literary storytelling about particular patients and stuff that I honestly think is like underutilized yeah know I mean it goes back to what you're saying before about like the intuition and the the many variables so it could be like even as a I mean a lot of you I mean so I don't know much about clinical work and like less about things that are considered you know outside science like Fran analysis but I can imagine someone doing a lot of good work within that framework because even if the theory doesn't hold any water as a scientific theory it could still be the intuition someone builds up working within that system are very valuable and can create um good outcomes for people and yeah so like I'm not I don't put my my my faith and my life in the hands of like alternative practitioners but when people get outcomes or anything like this you know I try to be a little bit slow to completely um dismiss it and not not at least a which because like you know Placebo effects while under attack are probably real in cases but uh but I mean there's things there's practices that might be done that could I mean sort of like cultural knowledge I mean Italians know how to make good food and things like this it's just and it's from like hundreds of years of tweaking things and trying things out so that's even beyond the individual and so maybe some of the training uh these folks receive in clinical settings I mean some things you know like like uh cognitive behavioral therapy and is has some some like random control trial backing but um yeah anyway I'm not trying to yeah to be clear I'm not trying to like totally diss it I just no no please please please totally dis I'm sing me and I think after I just basically made fun of a few Noble prizes in Neuroscience I've got um I've Got No One's Gonna like me anyway so great well we'll be in the same boat then um cool well this is this is a really truly fascinating discussion I'm I'm really thankful that we got a chance to chat um if people are uh interested in learning more about your research um like where can they find you and what should they read yeah sure um they could go to my website which is bradl love.org uh there there's also a link to the brain GPT website and if people are interested in that project uh we have a a mailing list and we don't spam people we probably need to send a few messages a year with major updates so people want to follow that project they could sign up there and um you know most of I mean they should I'll put a link up to this uh this podcast when it's out so and so if they want to see more like popular media stuff that's on the website as well and if anyone's really a glutton for punishment of course they could just start reading abstracts and and in papers but I'm not sure I could recommend that to the Casual uh listener cool uh well thank you so much Bradley I really appreciate this whenever you have more stuff to share I would love to have you back oh that'd be great yeah thanks so much Dan for having me here it was a blast oh my gosh folks you absolutely positively have to smash that like button and subscribe to Ai and I why because this show is the epitome of Awesomeness it's like finding a treasure chest in your backyard but instead of gold it's filled with pure unadulterated knowledge bombs about chat GPT every episode is a roller coaster of emotions insights and laughter that will leave you on the edge of your seat craving for more it's not just a show it's a journey into the future with Dan shipper as the captain captain of the spaceship so do yourself a favor hit like smash subscribe and strap in for the ride of your life and now without any further Ado let me just say Dan I'm absolutely hopelessly in love with you