Conference talk by Filip Makraduli (Superlinked) on embedding inference infrastructure for small models. Covers why typical embedding setups assume a fixed model chosen in advance, real profiling mistakes and infrastructure gaps observed in production, and the design of an embedding inference engine supporting dynamic model loading, hot-swapping, and memory-aware eviction instead of one-model-per-container deployments. Timestamped topics include moving from research to inference infrastructure, the Superlinked inference engine, context management for agents, why adding more GPUs is not the only answer, balancing model support with infrastructure, supporting diverse model architectures, scalability, and the open-source launch of SAI. # Channel Presentation Observations — Video: "The Small Model Infrastructure Nobody Built (So We Did) — Filip Makraduli, Superlinked" ## Explicit Self-Description (Creator Claims) - **Company affiliation stated directly**: The speaker identifies as part of a team at **Superlinked**, describing it as "a team at Superlinked, comprised of very good, uh, infrastructure engineers." - **Product launch framing**: The talk is framed as a **"soft launch"** of an open-source project: *"this repo, the Superlinked inference engine that we have open-sourced, and this is kind of like the soft launch that I'm doing today."* Later referred to as **"Sie"** ("It's called Sie, so S I E. And the company Superlinked."). - **Claimed scope of the offering**: The channel/creator claims a full-stack, end-to-end inference solution: > "We want to give you the whole end-to-end thing so that you can work with this models quickly, and the models are basically just a config that you can switch around and then do Terraform apply, and that's it." - **Claimed market gap**: The creator asserts no existing open-source solution covers this territory: > "On the market currently, there is no open-source solution that leads you from creating the model inference to actually productionizing this at a scale, um, of this size." ## Intended Audience (Observable Presentation) - **AI/ML engineers and applied practitioners**: Content assumes familiarity with concepts like flash attention, positional embeddings, grouped query attention, memory-bound vs. compute-bound processes, and MTEB benchmarks — no beginner-level explanation of these terms. - **Builders of agentic workflows**: Explicit framing — *"why doing inference for AI search and document processing actually matters when you're building agents or when you're building workflows that involve, uh, agents."* - **Infrastructure/DevOps-aware audience**: References to **KEDA autoscaling, Prometheus metrics, Grafana, Terraform, Helm charts, and Docker images** presuppose production-engineering knowledge. - **Conference talk format**: Audience interaction ("Does anyone have any ideas maybe from the audience?"), a Q&A exchange at the end, and a slide-based presentation indicate a live developer conference setting. ## Claimed Scope of Content and Product - **Small-model inference** for **AI search and document processing**, not large-model serving. - **Model support breadth**: Claimed support for heterogeneous architectures (BERT, Qwen, ModernBERT, ColBERT, cross-encoders, rerankers, Glinter NER, Stella embeddings) via re-implemented forward passes and variable-length flash attention. - **Infrastructure layer**: Routing, queuing, GPU pooling, spot instances, LRU eviction for hot-swapping models on a single GPU, autoscaling. - **API primitives**: Three stated primitives — *"encode, score, and extract."* - **Ecosystem validation claim**: *"we've tested this out... with Chroma, Quadrant, Weaviate... as well as LanceDB"* — positioned as partner/vector-DB ecosystem testing. ## Demonstrated Presentation Patterns (Observable, Not Claimed) - **Personal-narrative pedagogy**: The talk is structured around the speaker's own learning arc (a Substack article, a mistake about inference, joining Superlinked to learn by building) — an evidence-first, humble-expertise style. - **Three-part structured argument**: The speaker explicitly enumerates takeaways: (1) "why this matters," (2) "what inference is not about," (3) "the yin and yang of model inference." - **Community-referential argumentation**: Cites **Andrej Karpathy's knowledge-base work**, a **Chroma research paper on context rot**, and Hugging Face model counts as supporting evidence rather than relying solely on self-promotion. - **First-principles technical depth**: Detailed discussion of normalization differences between BERT and Qwen, fused QKV vs. grouped query attention, absolute vs. rotary positional embeddings, and token-based batching waste — depth matching the audience assumption. - **Interactive engagement device**: The background-slide guessing game with a "reward" at the break, resolved at the end (sinusoidal positional encoding visualization). ## Speaker Credibility Signals (Stated by Speaker) - Self-described experience: *"a person who's been in AI for a few years,"* work with **VLAM, training models, fine-tuning, applied ML/AI**, and *"a bit of also research in academia."* - Self-acknowledged gap transparently disclosed: production inference, *"scheduling GPUs, routing, and automation... was a bit of a blind spot for me"* — consistent with the talk's "we built what was missing" narrative. ## Notes on Evidence Limits - Claims of partner testing (Chroma, Weaviate, etc.) and the absence of competing solutions are **creator assertions**; the transcript provides no independent verification within this material. - No curated metadata was supplied, so identity observations rest solely on title and transcript. # Delivery Observation Report ## "The Small Model Infrastructure Nobody Built (So We Did)" — Filip Makraduli --- ## 1. Tone and Overall Manner - **Conversational, self-deprecating opening**: The speaker frames the talk around a personal mistake: *"I felt very confident. And that was true, but then some people pointed out that actually I had overlooked one key aspect."* - **First-person narrative style**: The talk is structured as a personal learning story — *"the story starts with me posting an article a few months ago"* — rather than a purely technical lecture. - **Casual hedging language** appears throughout: *"kind of," "a bit of," "a little bit," "um."* > "this is kind of like the soft launch that I'm doing today" > "routing, and automation, I guess was a bit of a blind spot for me" --- ## 2. Recurring Phrases and Verbal Habits ### Repetition of "kind of" as a softener Used at least 10 times across the talk: > "why we kind of made this approach" > "how that combines with my ML experience" > "there is this effect of context rot" ### Framing everything in threes The speaker explicitly structures the talk: > "the three key points I want you to, uh, go away with from this talk are these" He then uses signposting: *"first… Then… And the third thing is…"* ### "So" as a constant connector Nearly every paragraph begins with "So": > "So, the story starts with…" > "So, basically it's inference for small models…" > "So, why this matters for your agentic workflow?" --- ## 3. Teaching Habits ### Anticipating audience objections He poses rhetorical questions on the audience's behalf: > "And you might say, 'Okay, why would I not use like code and grepping?' And that's a valid point. However…" This is a repeated pattern — also: *"what inference is not about"* is an entire section built on addressing misconceptions (*"the traditional maybe perspective is, 'Okay, I will just chuck in more GPUs…'"*). ### Concrete named examples Consistently grounds abstract points in specific model names: **Stella, Glyner, BERT, Qwen, ColBERT, Gemma, MTEB, Chroma, Weaviate, Quadrant, LanceDB**. > "if you use BERT and Qwen as models, they have different implementation of flash attention" ### Narrative callback structure Opens with a puzzle ("if you can guess what this is, I'll prompt you at the end of the slides, you win a little reward") and closes the talk with an audience Q&A resolving it — an interactive bookend device. --- ## 4. Communication Patterns ### Signposting progress Repeatedly summarizes where he is in the talk: > "So, I've talked about what inference isn't, why this is important, and let me tell you now about, um, what actually inference is about." ### Mnemonic labeling He brands concepts with a memorable name: > "I call this the yin and yang of inference… So, first, the yin is model support… The other part, the yang, is around infrastructure." ### Appeal to external authority Supports claims by citing others rather than self-asserting: > "And it's not just me saying this, so this is how the community has responded… Andrej Karpathy is building knowledge bases graph-based." --- ## 5. Fluency and Uncertainty Markers - Frequent **false starts and self-corrections**: > "make make this kind of applicable" > "The Q The query, key, and values" > "what what we've done is we've set up a way" - Occasional admitted uncertainty, notably in the Q&A: > "I don't remember exactly, but it was one of the ones like ColBERT embeddings…" This suggests delivery is largely unrehearsed or extemporaneous in places — **uncertainty: the corrected-repetition pattern is frequent enough to be a stable habit, but the Q&A hesitancy is an isolated moment.** --- ## 6. Audience Interaction Style - Playful and informal: *"think about this, but also listen to me, so don't think too hard."* - Gracious with a wrong-ish answer: *"So, I will take that as a correct answer… Congrats."* - Lighthearted deflation: *"Okay, no no no reward, I guess, for you, sadly."* --- ## Summary | Feature | Evidence Strength | |---|---| | "So" + "kind of" as discourse habits | **Strong** — highly frequent, stable across the talk | | Three-part signposted structure | **Strong** — explicitly announced and followed | | Personal-story framing | **Strong** — opens and closes the narrative arc | | Rhetorical objection-handling | **Moderate-to-strong** — appears in two distinct sections | | Interactive puzzle bookend | **Isolated** — single device, not a pattern | | Q&A hesitancy | **Isolated** — one moment only | # Position Record: "The Small Model Infrastructure Nobody Built (So We Did)" **Speaker:** Filip Makraduli, Superlinked (per video metadata) ## Origin Story / Motivation - Posted a Substack article explaining **flash attention**, model mechanics, and memory-bound vs. compute-bound processes; readers pointed out he **overlooked inference** as a key aspect of real-world model speed. - Stated this was a **blind spot** for him despite years in AI (VLAM, training, fine-tuning, applied ML, academic research) — specifically production concerns: "scheduling GPUs, routing, and automation." - Response: joined Superlinked's infrastructure team and built the open-sourced **Superlinked inference engine** ("Sie"), described as a **soft launch** during the talk. - Tested with partners including **Chroma, Qdrant, Weaviate, and LanceDB**. ## Point 1: Why Small Model Inference Matters (Agentic Workflows) - **Context rot**: cites Chroma research showing quality degrades as context increases, regardless of approach. - Proposed solutions: - Small models **preprocess data** before agents/workflows run. - Small models used for **tool calling** (e.g., a production use case: taxonomy classification for an e-commerce store using the repo as a tool-calling solution). - Anticipated objection ("why not code and grepping?"): he conceded it's a valid point but claimed preprocessed data **improves** grepping and file systems rather than replacing them. - Community evidence cited: - **Andrej Karpathy** building graph-based knowledge bases (NER models → ontologies → knowledge graphs). - **Chroma** shipping their own model to preprocess/filter input and manage context. - Community tools reducing token counts for agent effectiveness. ## Point 2: What Inference Is *Not* About ### Not "just add GPUs" - Small models (e.g., **Stella** embeddings, rerankers, **Glyner** NER) occupy only a few GB of memory; provisioning one GPU per model wastes idle space. - Their solution: **hot-swapping models on one GPU**, higher per-GPU utilization, **least-recently-used eviction policy**; lowers cost and enables fast model switching. ### Not only the server/production code - Tools like TAI, VLAM, or an API wrapper leave routing, autoscaling, and monitoring (Prometheus/Grafana) as **do-it-yourself work**. - Claim: **no open-source solution** currently spans model inference through productionization at this scale — the gap they filled. Included: routing, autoscaling, queuing, GPU provisioning. ## Point 3: "Yin and Yang" of Inference ### Yin: Model Support - Open source momentum: ~2–3 million models on Hugging Face; open models improving in size and accuracy. - Claims open source models **beat managed services on narrow tasks** (per MTEB and similar benchmarks), and cites **Gemma's** low-parameter models with ELO scores higher than much larger models. - Architectural heterogeneity problem — no universal engine handles BERT, Qwen, and ModernBERT because of differences in: - Flash attention implementations - Positional embeddings (BERT's absolute lookup vs. Qwen's rotary) - Normalization - QKV fusion (blocked in Qwen by grouped query attention) - Output types (ColBERT's multiple vectors / late interaction; rerankers outputting scores, not vectors) - Their approach: **re-implementing the forward pass** (adapting attention, padding, variable-length attention, QKV fusion), described as built "with agents as well as with humans." - **Variable-length flash attention** to avoid wasted compute on padded empty tokens in token-based batching — called "a key differentiator." ### Yang: Infrastructure - Three API primitives: **encode, score, extract**. - Router + queuing mechanism, pools of GPUs (including **spot instances**), **KEDA autoscaling with Prometheus metrics**. - Goal: **end-to-end** cluster plus model support, not one or the other; "models are basically just a config" switched via **Terraform apply**; Helm charts and Docker images published. ## Closing / Audience Interaction - Background-slide reveal: audience member identified it as a **vector visualization of positional encoding** (sinusoidal pattern); Makraduli accepted the answer and awarded the promised reward (he initially couldn't remember exactly — noting it was about embeddings/clustering in an embedding space). ## Qualifications & Unresolved Notes - His "millions of models" figure was hedged: "This is from March. Now, there might be even more like close to 3 million." - The "no open-source solution" market-gap claim is the speaker's assertion, not independently verified within the talk. - Performance comparisons (open source vs. managed services) are attributed by the speaker to benchmarks like MTEB, without specific citations shown. # Observation Document: The Small Model Infrastructure Nobody Built (So We Did) ## Speakers & Roles - **Filip Makraduli** — **Creator / primary speaker**. Identifies himself as an AI researcher and engineer at **Superlinked**, delivering the full talk. - **Unnamed audience member(s)** — respond near the end guessing the background slide image ("positional coding"); speaker accepts it as correct. Role: **audience participant**. - No narrator, host, or other guests are present. ## Speaker Provenance & Self-Disclosed Background - Filip states he previously did **academia research**, **VLA model work**, **training and fine-tuning**, and **applied ML/AI**, but identifies production inference (GPU scheduling, routing, automation) as his prior **blind spot**. - He references a **Substack article** he wrote (a few months prior, on flash attention and memory/compute-bound processes) that drew community feedback pointing out he'd overlooked inference. ## Promotional Segments (Flagged) - **Soft launch of the Superlinked inference engine** ("Sie", spelled S-I-E) — open-sourced repo, with QR code shown to the audience. This is explicitly a promotional/product-launch portion. - Mentions it was **tested with partners**: Chroma, Qdrant ("Quadrant"), Weaviate, LanceDB. - Mentions published **Helm charts and Docker images**. ## Referenced Sources & Third-Party Claims - **Chroma research paper** on **context rot** — cited as evidence that quality degrades as context length increases. - **Andrej Karpathy** — cited as "building knowledge bases graph-based" (community example, no direct quote). - **Chroma** — shipped their own model for preprocessing/filtering input to manage context. - **Hugging Face** — millions of open-source models cited (figure "from March"). - **MTEB benchmarks** — cited for narrow-task open-source model performance claims. - **Gemma** — cited for low-parameter models with high ELO scores. - Named models/tech: **Stella, GLiNER, BERT, Qwen, ModernBERT, ColBERT, Prometheus, Grafana, KEDA, Terraform, BentoML-adjacent tooling ("Tay"/"VLAM" per transcript), token-based batching, flash attention**. ## Availability & Uncertainty - **Comments availability**: not specified in the source material. - Audience participation audio is partial/unclear; the guess about "positional coding" is transcribed fragmentarily — uncertainty remains about exact wording. - Partner testing claims ("they've tried it out a bit") are **self-reported by the speaker**, not independently verified in this source. - Some product names appear in ambiguous transcript form ("Quadrant", "Tay", "VLAM", "Glyner") — likely Qdrant, vLLM, and GLiNER respectively, but transcript text is the only source. Hello everyone. Um, welcome to this talk. I'll be speaking about small model inference and a gap that we've recognized in the market. And what we did about it and why we kind of made this approach. And as you can see this background slide here, um, this is no accident. So, if you can guess what this is, I'll prompt you at the end of the slides, you win a little reward. So, you can catch me at the break afterwards. So, think about this, but also listen to me, so don't think too hard. So, the story starts with me posting an article a few months ago on Substack that got a little bit of traction, got a few people interested, and I explained flash attention, I explained how models worked, how processes can be memory-bound, compute-bound, and I felt really good cuz I kind of went deep into this and as a person who's been in AI for a few years, I felt very confident. And that was true, but then some people pointed out that actually I had overlooked one key aspect around what makes these models fast in the real world. And that aspect that I overlooked was inference. So, as someone who kind of wants to understand things in first principles and work to understand the problems and the solutions deep, I realized, "Okay, I need to figure this out. I need to know kind of where I've made my mistake and as a AI researcher and engineer, I need to find more about inference. So, I've done a lot of work with VLAM, training models, fine-tuning, kind of doing applied ML and AI, did a bit of also research in academia as well. But this part around kind of how models run in production, scheduling GPUs, routing, and automation, I guess was a bit of a blind spot for me. So, I realized, "Okay, this is the time I have to now figure this out, learn, and make it work." And what better way to do this than to actually build stuff. So, I decided to join a team, um, a team at Superlinked, comprised of very good, uh, infrastructure engineers, and actually work with them and build something around inference. And that something is, um, this, uh, repo, the Superlinked inference engine that we have open-sourced, and this is kind of like the soft launch that I'm doing today. So, you can have a look at that later. So, basically it's inference for small models around AI search and document processing. And we've tested this out, as you can see, with some of our partners. So, we've tested out with Chroma, Quadrant, Weaviate, so a lot of the vector DBs, as well as LanceDB. And as you can see, they've tried it out a bit. Sounds fun, sounds interesting. So, this is working. And, um, it was the right step for me to kind of figure this out and learn where inference, um, kind of truly is and how that combines with my ML experience. So, the three key points I want you to, uh, go away with from this talk are these. So, first, I want to tell you why this matters. So, why doing inference for AI search and document processing actually matters when you're building agents or when you're building workflows that involve, uh, agents. So, this is very important, the why. Then I want to talk to you about the second thing, which is what inference is not about. So, there are some misconceptions or ideas of how inference looks like, but it's not all about those things. And the third thing is how we see inference, and I call this as like the yin and yang of model inference, and it's a way of combining a few things around model support and infrastructure. So, why this matters for your agentic workflow? Well, what you have encountered for sure is context rot. And as we probably all know, this research paper from Chroma from some time ago, um, showcases this effect that no matter what you do, there is this effect of context rot. So, quality degrades as context increases. So, being able to manage this context and do some context management is very important and a useful way to solve this. So, using small models that can preprocess your data so that then you can actually use your agents and build your workflows is a very powerful technique. You can also use the small models for tool calling to again do similar things and tackle this, uh, problem of context management. And you might say, "Okay, why would I not use like code and grepping?" And that's a valid point. However, you can still do that and having your data being preprocessed is actually making the grepping and file systems that you build even better. And it's not just me saying this, so this is how the community has responded to this problem. So, Andrej Karpathy is building knowledge bases graph-based. So, for example, you can use named entity recognition models, um, to generate ontologies and then build knowledge graphs or you generate your file systems in that way. Chroma have also, uh, shipped their own model that actually does this. It kind of preprocesses and filters the input, manages the context in order to tackle and achieve the same problem. And there's also people in the community building a lot of solutions that, um, lower this, um, context and kind of reduce the token amount so that the agents can be even more effective. And we've done this in production as well. We have a use case where this repo that I showcased, uh, Superlinked inference engine, we've used it as a tool calling, basically, uh, solution where it was around taxonomy classification for like an e-commerce store. So, going directly, uh, with tool calling where the small models are tools that can retrieve and go through your data is also a powerful way to approach this. So, now I'll go about inference and what inference doesn't look like. So, the traditional maybe perspective is, "Okay, I will just chuck in more GPUs, get more compute, all good, I've solved inference." However, um, with small models where actually each model, um, takes up only a small space in memory. So, you can see, for example, Stella, which is an embedding model, then other rerankers, um, the named entity recognition Glyner model, they only occupy like a few gigabytes of memory. And if you provision a GPU for each model, you're wasting a lot of, um, idle space. Your GPU stays idle and it's not used. So, in this case of small model inference, it's very important to be able to hot-swap models. So, what we've done is we've built the ability for you to swap all of these models in one GPU so that per GPU, you get much higher utilization. So, this lowers your costs, but also enables you to hot-swap and switch around between models quickly. If you want to have one tool that's one reranker or another tool that's another model, you can switch them around quickly, and we have this least recently used eviction policy that we've built in. And also what inference is not about is, um, only the server or only kind of the production situation. So, we have solutions for both the server and the production, but building something, so for example, like using Tay or VLAM or having even, um, some API wrapper in order to get that in production where you actually do routing, auto scaling, you do kind of monitoring with Prometheus metrics and Grafana, you have to write that code yourself. And on the market currently, there is no open-source solution that leads you from creating the model inference to actually productionizing this at a scale, um, of this size. So, that's why we've kind of wanted to fill this gap as well. So, um, we've included a lot of stuff around, routing, auto scaling, queuing mechanisms, and provisioning GPUs. So, I've talked about what inference isn't, why this is important, and let me tell you now about, um, what actually inference is about. And I call this the yin and yang of inference because I feel like this is a holistic approach that has to combine two key things. So, first, the yin is model support. So, your inference is worthless if you're not supporting the right models or you're not offering enough breadth of um, options for your users. And open source models, so on hugging face currently, there are millions of models. This is from March. Now, there might be even more like close to 3 million models. And open source is moving very quickly both in size, but also in accuracy. So, you need to support these models because people want to use them. People want to work with open source, and the performance is also getting better and better. So, it's not even a sacrifice anymore. It's actually you're being able for specific tasks. So, if you look at MTEB or different benchmarks, you can see that for very narrow tasks, open source models are beating managed services. And we see this even with more general-purpose models like what Gemma have done. So, they've released very low parameter models that have ELO scores that are higher than much, much bigger models. So, open source small models are very relevant, and you need an infra to support to support them. And we decided to do exactly this. Okay, let's do it. We'll build and support hundreds of models. However, that's not as straightforward as it might sound because all of these models have different run times. So, for example, with if you use BERT and Qwen as models, they have different implementation of flash attention. They have different positional embeddings, and you need to adjust this in order to make make this kind of applicable to your case and make it more general. So, there is no universal engine that can handle BERT, Qwen, and modern BERT as well because their architectures are different. So, what what we've done is we've set up a way of re-implementing this forward pass in order to adapt attention, in order to do padding where needed, so have variable length attention as well. Work with kind of fuse fusion of the query, key, and value. And this is kind of an underrated component, especially if you look at supporting models like ColBERT that has multiple vectors as output. So, late interaction models, also cross encoders, and re-rankers that do not output a vector at all, but they output kind of scores. So, it's very important to figure this out, and one example of this is, for example, this, which is a comparison of a few models that are totally different in these five aspects. So, normalization is done differently in BERT and Qwen. So, that needs to be accounted for. In ColBERT, it's also totally different. The Q The query, key, and values are also can be fused somewhere. In others, for example, in Qwen, it cannot be because there is um grouped query attention. So, that's another problem. The position positional embeddings are also different. You can do in BERT absolute lookup, where it with Qwen, there is rotary positional embeddings. So, these are problems that maybe people don't think about straight away, but if you want to support a lot of models, there needs to be a consistent way of doing this. So, we've set up this with kind of working with agents as well as with humans to be able to support this and re-implement this forward pass in order to make um the the inference of this models efficient, and we've also worked to do flash attention where it's variable length flash attention, so that you can do padding, and that there is no waste because what happens is if you want to do token-based batching, you can have tokens that are kind of lower number of tokens one request, and then another is a higher amount, and then if both are padded at the higher amount, you basically are wasting compute on empty tokens. So, you need to adapt this in order to make the models work even better and quicker. And that's a key differentiator in our way of approaching inference. The other part, the yang, is around infrastructure. So, this is the complex plot. It's kind of this is the deep dive. We have this in our website and in our repo. I won't go into each component here, but basically what's up there is the three primitives of our API, so encode, score, and extract. And then the infrastructure layer happens, and this is kind of the summary of it. So, basically, we have the three primitives of the API, and then we have a router and also queuing mechanism depending on the load. So, we're adjusting between these and also different pools of different GPUs and resources that we use in order to kind of distribute the workload, and we we use spot instances, but also bigger GPUs. So, it's about being able to provision the hardware and also have the metrics to auto scale this. So, we're doing KEDA auto scaling with Prometheus metrics in order to be able to kind of switch models around, not keep GPUs idle, and kind of waste resources. So, the key driver here is that we want you to have the cluster as well as the model support. So, not just one or the other, and then you have to merge them together and write all that code, but we want to give you the whole end-to-end thing so that you can work with this models quickly, and the models are basically just a config that you can switch around and then do Terraform apply, and that's it. And we also have published Helm charts and Docker images as well. And that's how I learned my lesson. We've built Sie. So, this is kind of our soft launch in a way. And we've open sourced both the model inference that I talked about with kind of adapting this forward pass, working with attention, re-implementing different aspects on the model, but also the cluster so that you can actually use this straight away without having to think about hardware, provisioning GPUs, and all of those problems that arise in in the real world. You can scan the QR code to look at the repo, and it's called Sie, so S I E. And the company Superlinked. And also, now we have the the background as well. So, a quick reminder on that. Does anyone have any ideas maybe from the audience? I'll reveal it in the next slide, yeah, I don't want to maybe do it too quickly if anyone knows what this is. Think of it machine learning a bit kind of foundational knowledge. I was talking about attention and all that, so it's a bit around embeddings, but also how transformers work. Okay, no no no reward, I guess, for you, sadly. Or yeah, sorry. I don't remember exactly, but it was one of the ones like ColBERT embeddings for the clustering of the like one area. It wasn't using basically the embedding space. Okay. \>\> Because of the way they were doing the positional coding. How much of that So, I will take that as a correct answer. So, it was around positional encodings, and yes, this is basically yeah. Congrats. So, yes, this is around vector visualization of positional encoding. So, this is done when transformers are trained, this is how positions are encoded, and it's sinusoidal. That's why you get this pattern because it they're embeddings, and yeah. Thank you very much.