Skip to content

reproducibility

Reproducibility in this channel In the episode on AI and science, reproducibility comes up as part of the argument that scientific publishing is shifting from narrative-first to data-first: reproducibility depends on publishing the underlying data alongside the conclusions, so that others can re-do the analysis rather than take the story on faith [^820f4796-fbb2-5226-8fee-b27de60aecb0@43]. Why the current system falls short The episode's framing is that traditional publishing captures only successful outcomes — as the guest puts it, “the problem has always been we only publish certain things and so we're not getting any information around what's failed or what's been tried.” Failures and tried-but-abandoned approaches never surface, which skews the picture of scientific progress and makes it harder to check results. What changes The proposed fix is not to get rid of the story — stories still matter. What changes is the data's status: as access to data and compute grows, the underlying data becomes just as important as the narrative itself. Future papers may include a data appendix alongside conclusions, mirroring the transparency of modern machine learning research, so that results can be reproduced and failure cases become visible rather than hidden. - Reproducibility depends on publishing data alongside conclusions [^820f4796-fbb2-5226-8fee-b27de60aecb0@43].- Traditional publishing hides failures, which undermines reproducibility and skews progress.- The story still matters, but the underlying data is just as important in a data-first model.

glossary term

Ready

Loading…

1 min read