From MCP to Scale: Pipelines That Build Themselves — Rafael…
Conference talk by Rafael Levi (Bright Data) on using MCP to let agents inspect websites, understand their structure, generate production scrapers, and automatically maintain data pipelines when sites change. Covers moving from one-off extraction to reusable parser scripts that save tokens, and agents repairing broken collection jobs without human intervention.
# Channel Presentation Observations — "From MCP to Scale: Pipelines That Build Themselves — Rafael Levi, Bright Data"
## Speaker & Affiliation
- Presenter identifies as **Rafael Levi of Bright Data** (per title; no additional self-introduction in transcript).
- Speaks in a conference/demo-session format: "in the previous session… I talked about how MCP gives access to LLMs to websites that are behind CAPTCHA" — indicates he delivers multiple talks at the same event.
## Intended Audience
- **Technical practitioners who build or maintain web scrapers**: "I don't know how many of you actually wrote scrapers before… you know the headache when all of a sudden things are missing, data is missing, you got to wake up."
- **AI/agent developers concerned with token costs**: "Tokens is expensive. I'm going through millions and millions of tokens a day… all of you are looking how do we save money on LLM?"
- Live, interactive audience — he polls them, asks for a website suggestion ("Pick a website. What's a popular marketplace… in UK?"), and takes questions.
## Claimed Scope (Creator's Own Statements)
- **Company capabilities**: "Bright Data has skill sets that actually teaches your LLM on how to build a pipeline"; "we have over 150 million IPs"; "66 tools" in the MCP; "about 500 different APIs pre-built for different domains."
- **Product positioning**: "the MCP is mostly useful in about 20% of the domains… Akamai, Data Dome, Cloudflare, those heavy protected domains."
- **Free-tier offer**: "The MCP has 5,000 requests for free… just open an account with Bright Data, which doesn't cost you anything."
- **Legal stance / policy scope**: "we only deal with public data. So nothing behind login. We don't accept terms and conditions." Claims: "We were sued by Meta… sued by Elon Musk… the judge said… Public data is public data."
## Demonstrated Presentation Patterns
- **Live product demonstration as primary format**: builds scrapers on-screen against walmart.com and very.com in real time, showing delays and troubleshooting ("Oh, I forgot to delete the old one… as usual, it gets stuck on a live demo").
- **Quantified efficiency claims woven into demos**: "for the three pages… it saves about a million tokens"; final demo shows "about a 62% save of tokens."
- **Personal-use anecdotes to broaden appeal**: apartment hunting ("I set up a listener… In a few days, I got a notification and now I live there") and restaurant booking ("I'm waiting for 2 months already").
- **Comparison with competitor tools**: "CloudCrawl does an amazing job… I know Crawlera does a great job as well, but I prefer CloudCrawl because… it just gives me less headache."
- **Compliance cautionary framing to audience**: "Always check terms and conditions of the website. If they say don't scrape… then that company can actually sue you."
- **Ongoing engagement invites**: "we have a booth… Feel free to come to the booth on the third floor"; "we can connect on LinkedIn… I'm always willing to help."
- **Closing slogan consistent with the legal framing**: "let's keep the public data public."
## Distinguishing Claims from Demonstrations
- **Claimed, not demonstrated**: 150M IPs, 66 MCP tools, 500 APIs, lawsuit outcomes, million-token savings (stated as prior experience, not shown).
- **Demonstrated on screen**: scraper generation against Walmart and very.com, the blocked-without-MCP vs. unblocked-with-MCP comparison, generated parser/schema code, and the 62% token-saving figure.
- **No curated metadata was supplied**, so identity context rests entirely on title and transcript.
# Delivery Observation: "From MCP to Scale" — Rafael Levi
## Overall Tone & Register
- **Conversational, informal talk-show style** throughout — heavy filler words ("uh," "um," "right?") and casual asides.
- Repeatedly frames technical points as shared experience with the audience rather than lecture material.
## Recurring Verbal Habits
### 1. Rhetorical questions to the audience
A dominant, stable pattern — used to poll the room and build rapport:
- > "I don't know how many of you were here"
- > "I don't know how many of you actually wrote scrapers before."
- > "how many of you use VibeCode?"
- > "Any more questions? How about this? Anybody has problems with actually getting data?"
### 2. Sentence-final "right?" as a cohesion marker
Attached constantly to explanations:
- > "you don't do that. Right?"
- > "sometimes you maintain it more than it it it takes you to write it, right?"
- > "Markdown is It's to save tokens, right?"
### 3. Repetition/stuttering under live pressure
- > "it it it solves all that headache"
- > "I've I've Look. Clean test. No cheating."
- > "the the the text, right?"
- > "And and uh just you know just a heads-up."
Most concentrated during the live demo and impromptu moments, suggesting the disfluency is situational (demo multitasking) rather than constant.
### 4. "As a matter of fact" as a filler-transition
Appears twice — mild evidence of a verbal tic, but only two instances:
- > "As a matter of fact, sometimes you maintain it more..."
- > "As a matter of fact, I do collections on the daily basis"
## Teaching Habits
### Live-demo-driven structure
The session is built around performing the task in real time, with narration:
- > "Let me demonstrate. So, for this demo, I want to use Cloud Code."
- He narrates the agent's steps aloud: > "it's going to go to our GitHub, it's going to get all the skill set..."
### Before/after and contrast framing
Repeatedly contrasts "old way vs. new way" with rough time estimates:
- > "back in the day it took weeks to set it up"
- > "This used to take... maybe a full day, maybe a day and a half"
- Demonstrates failure first, then success: > "It's going to do a fetch, which is going to get blocked... Now, do the same with Bright"
### Quantified claims as persuasion
Habitually anchors arguments in token/number savings:
- > "it saves about a million tokens"
- > "it's about a 62% save of tokens"
- > "maybe like 60 tokens"
- > "we have over 150 million IPs"
### Audience participation / improvisation
Willingly abandons the planned demo and lets the audience pick the target site:
- > "let's do a different website. Pick a website. What's a popular marketplace in uh that's aggressive in blocking in UK?"
- Handles a forgotten cleanup transparently: > "I totally I was testing it and I totally forgot to clean up."
### Personal anecdotes as use cases
- > "I was looking for a new apartment... In a few days, I got a notification and now I live there."
- > "I have a listener right now. I'm waiting for 2 months already" (restaurant booking)
## Vendor Positioning
Promotional but delivered conversationally, with offers of free access and help:
- > "The MCP has 5,000 requests for free, so if anybody wants to try it"
- > "we have a booth... Feel free. I'm always willing to help." (repeated near-verbatim at the close)
- > "let's keep the public data public. Thank you." — a slogan-style closer.
## Handling Difficult Moments
- Acknowledges demo slowness rather than hiding it: > "Always when you do a live, it's slows down," "as usual, it gets stuck on a live demo"
- Exclamations during troubleshooting: > "Oh my god." > "What the I'm setting a variable." > "What is wrong with my paste?"
- Humor to close Q&A: > "Everybody is like I'm not sharing my problems... And they go home and like oh I do have a problem."
## Uncertainty Notes
- Patterns like rhetorical questions, "right?", and quantified claims have many instances — **high confidence**.
- "As a matter of fact," the apartment/restaurant anecdotes, and preference statements ("I prefer Cloud Code because... it gives me less headache") have only **one or two instances** — possible habits, but repetition is insufficient to confirm.
- No timestamps were quoted beyond those in the transcript; pacing/inference about tone shifts between segments is based on transcript flow only.
# Position Record: "From MCP to Scale: Pipelines That Build Themselves" — Rafael Levi, Bright Data
## Speaker
- **Rafael Levi** (Bright Data) — sole presenter; audience members contribute brief audience-quoted interjections.
## Core Thesis
- Rather than having an LLM parse every page (token-expensive), agents should **build, run, and maintain scrapers**.
- Bright Data's MCP plus its GitHub "skills" repository enables agents to explore pages, extract HTML, identify selectors, and generate self-maintaining scrapers.
## Key Claims & Positions
### On pipeline maintenance
- Traditional scrapers require ongoing maintenance: >"sometimes you maintain it more than it takes you to write it."
- He claims agents remove this burden: an agent can **write, execute, and maintain** scrapers. He states he runs a validation loop every 30 minutes where an LLM checks collected data and fixes missing data points "in 5 minutes" without waking anyone.
### On token efficiency
- Building a scraper and running a script is far cheaper than LLM parsing. Cited figures (his own demo measurements):
- ~**1 million tokens saved** over three Walmart pages (claimed from prior runs).
- ~**62% token savings** parsing 90 products on very.com — which he qualifies: >"This is not a high number," speculating the site may have structured HTML (he says "I'm not sure").
- Running the script costs ~60–100 tokens vs. thousands for direct LLM access (audience member adds the "1,000 vs 10,000 tokens" framing).
- JSON output is more token-efficient than markdown or HTML.
### On MCP capabilities (as described)
- **66 tools** for agents, including:
- Sending cURL to any URL; the system solves CAPTCHAs, handles headers/cookies so the server "thinks it's a browser."
- Full HTML or **"scrape as markdown"** (text-only, to save tokens).
- **~500 pre-built APIs** returning structured JSON for specific domains (e.g., Amazon) — no scraper needed.
- **Remote browser infrastructure** — "I can literally right now open a thousand browsers on this laptop that are running on our servers"; geo-selection of IPs (e.g., US IP for geo-restricted sites).
- **150+ million IPs** in infrastructure; in-house AI CAPTCHA solving; pre-recorded human-like mouse movements and typing (with simulated errors) to defeat behavioral trackers.
- He notes cheaper models suffice: >"With Claude Haiku model is more than enough for browsing" when masked as human.
- MCP free tier: 5,000 requests.
### Demonstrations
- **Walmart demo**: without MCP, the agent hits a bot-verification screen; with Bright Data MCP, it retrieves product listings via scrape-as-markdown.
- **very.com (UK) live demo**: agent builds a scraper on an unfamiliar site ("clean test, no cheating"), collecting 90 products; live demo runs slowly, which he acknowledges repeatedly ("always when you do a live, it slows down").
- **Personal anecdotes** (his own use):
- Apartment hunting: a listener scraping every half-hour notified him of a house under his target price; "now I live there."
- Restaurant booking: a listener waiting 2 months to auto-book a spot at an always-packed restaurant.
### Tooling preferences (stated, not argued)
- Prefers **Claude Code** over Codex: "It does a great job."
- Prefers **CloudCrawl** over Crawlera: >"Crawlera sometimes takes me on a wild goose chase."
### On legal/ethical boundaries
- **Public data only**: no data behind logins; Bright Data does not accept websites' terms and conditions.
- Warning to audience: if you create an account and accept ToS forbidding scraping, you can be sued; he cites ongoing litigation (LinkedIn, "Elon Musk took over Twitter, locked it down").
- Legal position (his characterization): Bright Data was sued by Meta and by Musk's X; >"the judge said it's very simple. Public data is public data. It doesn't matter how you collect it."
- Framing: >"Data right now is like the new gold, the oil."
- Closing slogan: "Let's keep the public data public."
### On action capabilities
- Agents can fill and submit forms, click buttons (e.g., flight searches on Skyscanner where URLs are hashes).
- The stated limit: **cannot log in** to sites.
- Notes the MCP matters most for ~**20% of domains** — heavily protected ones (Akamai, Data Dome, Cloudflare), which he calls "the juiciest" (real estate, big e-commerce).
## Unresolved Tensions / Qualifications Preserved
- His 62% savings figure is explicitly downplayed by himself as "not a high number," with an uncertain explanation for it.
- Live demos repeatedly underperform ("It's working very slow," "It gets stuck on a live demo") — presented as such, not smoothed over.
- All savings and speed figures are his own demos/anecdotes, not independently verified in the talk.
# Speaker Attribution & Provenance Record
## Video Metadata
- **Title:** From MCP to Scale: Pipelines That Build Themselves — Rafael Levi, Bright Data
- **Video ID:** `01a15ca9-107b-7289-8198-031d2f2c08b8`
- **Format:** Conference talk with live demonstration and audience Q&A
## Speakers & Roles
### Primary Speaker — Rafael Levi (Creator / Presenter)
- **Role:** Bright Data representative delivering the session; performs live demos and answers questions.
- **Affiliation (self-identified):** Bright Data ("our MCP," "we have over 150 million IPs," "we were sued by Meta").
- **Session context:** Second talk of the day; references a **previous session** on MCP giving LLMs access to sites behind CAPTCHA and bot detection.
### Audience Members (Guests / Unnamed)
- Multiple unidentified audience voices participate in Q&A and demo interaction:
- Suggest demo targets: **Amazon**, **very.com** ("Very"), confirm UK marketplace.
- Ask questions on: MCP details, generated scraper code, authorization/login-protected sites, performing actions on websites.
- One audience comment near the end estimating token savings ("Literally like we're talking about 1,000 tokens…") — attribution ambiguous between speaker and audience.
### Not Present
- No narrator, no other named speakers.
## Promotional Segments (Flagged)
- **Bright Data MCP:** "5,000 requests for free," "open an account with Bright Data, which doesn't cost you anything."
- **Bright Data GitHub skills page** ("Bright Data skills") — scrape-builder best practices for agents.
- **Bright Data infrastructure claims:** 150M+ IPs, 66 MCP tools, ~500 pre-built APIs, in-house AI CAPTCHA solving, remote browser infrastructure with human-mimicking mouse/typing.
- **Legal positioning claim:** Bright Data "won a few lawsuits" including suits by **Meta** and **Elon Musk/X** ("Public data is public data").
- **Call to action:** visit booth on third floor; connect on LinkedIn.
- **Competitor comparison (speaker preference, self-declared):** prefers "CloudCrawl" over "Crawlera" ("Crawlera sometimes takes me on a wild goose chase"). *(Names as spoken; possibly paraphrased/misspoken brand names.)*
## Referenced Sources & Third-Party Claims
- **Third-party platforms/services mentioned:** Reddit, Walmart, Amazon, very.com, Cloudflare, Akamai, Data Dome, Cloud Code / Claude (Haiku), Codex, "VibeCode," CloudCrawl, Crawlera, Skyscanner, CNET, Twitter/X, LinkedIn, Meta.
- **Third-party legal claims (reported, not verified):** LinkedIn litigation over scraping; Elon Musk locking down Twitter data after acquisition; companies suing scrapers over terms-and-conditions violations.
- **Quantitative claims (speaker-reported, unverified):** ~1 million tokens saved on three pages (Walmart); ~62% token save on very.com run; scraper build time ~3 minutes vs. days previously.
## Scope & Availability Notes
- **Claimed limitation:** Bright Data deals **only with public data**; nothing behind login; no terms-and-conditions acceptance.
- **Comment availability:** Not indicated in transcript or metadata.
- **Uncertainty:** Audience speakers unnamed; one late token-savings quote attribution ambiguous.
All righty, let's dive into it. So, uh in the previous session, I I don't know how many of you were here, but I talked about how MCP gives access to LLMs to websites that are behind uh
CAPTCHA, bot detection systems, and so on. And in this session, I want to talk about um how do you actually collect data on scales with LLM? Because a lot of times I'm seeing on Reddit and other social media is like, "Oh, I need to scan 10,000 products, but that's so many tokens if I need to parse everything with LLM." So, obviously, you don't do that. Right? So, the whole session is how do you build pipelines, right? With LLM. Instead of telling, "Hey, LLM, can you go and uh parse this for me?" build a scraper that's going to parse it for me, and I will demonstrate how easy it is uh with our skills, right? So, Bright Data has skill sets that actually teaches your LLM on how to build a pipeline. With our MCP, it can actually extract HTML so that it knows what are the selectors that it needs to parse, and so on. So, um the scrape tags, right? Write the scraper. Uh I don't know how many of you actually wrote scrapers before. Okay, so you know the headache when all of a sudden things are missing, data is missing, you got to wake up. I don't know if you guys did it for clients, and clients are like, "Oh my god, there's no So, basically, that's what used to be. You write a scraper, and you maintain it. As a matter of fact, sometimes you maintain it more than it it it takes you to write it, right? So, everything, especially if the website is constantly changing uh its selectors, or if it's a React website, it gets really complicated, right? So, an agent it it it solves all that headache, right? It can explore with our MCP, it understands what the data is needed. It writes the scraper and it actually runs it and executes it and maintains it. As a matter of fact, I do collections on the daily basis and I have every 30 minutes I have an LLM class pool spools up. It checks what the
data is collected, make sure that everything is fine. Everything is fine, it shuts down. If for this example, there's a always set a validation for data, right? Let's say the data point is uh missing
some something. Your agent fix it. 5 minutes. You don't have to wake up in the middle of the night. Uh So, let me demonstrate. So, for this demo, I want to use Cloud Code. I don't know how many of you use Cloud. If you're using Codex or whatever it is, I prefer Cloud Code. I like how it I like it. It does a great job. So, the way that you build scrapers now and it's ridiculous because, you know, back in the day it took weeks to set it up. We created a Bright Data like GitHub page, right? Bright Data skills. Here basically, your agent has everything that it might need, okay? Um scrape builders back practices, how to everything. Everything that it needs in order to build a scraper. And what I want to show is uh first of all, let's I'm going to start simple. Go to build me a scraper and I wanted to record it because I thought it's going to take a long time, but now it takes a little like 3 minutes. Build me a scraper uh two inputs keyword search max pages for walmart.com. Everybody's familiar with Walmart. Walmart has a very aggressive anti-bot systems. If anybody tried to scrape it, um it just doesn't work. But, through Bright Data,
it will figure it out, but you're you're all right. Let's just correct that. walmart.com Um
run a search for headphones, collect three pages. Uh so, what it's going to do right now is going to go to our GitHub, it's going to get all the skill set that it needs, how to build scrapers, then it's going to go uh with the MCP that's already connected to the cloud, it's going to extract the HTML from the page. Uh it's going to find all the selectors that it needs, and it's going to build you a scraper, and uh So, right now you can see a scraper as a markdown, it's basically extracting the the the the text, right? So, for the skills, we don't need the HTML, we just need the text. Uh scraper as a markdown is a part of the MCP of Bright Data MCP. Uh the MCP has 5,000 requests for free, so if anybody wants to try it, you guys can try it for free. Um You will need to just open an account with Bright Data, which doesn't cost you anything. And let it do its thing. So, why is this better than having an LLM parse every single page, every HTML? It's I'm I'm literally afterwards I'm going to ask it to tell you how many how much tokens it saves. So, for the three pages, I've done this before, it saves about a million tokens just to from building the scraper uh and using a script to parse the HTML. And um even if you are not doing any scraping
for production, but let's say that you want to find the best best headphones for the money, right? It used to be you go to Google, you go to some CNET where they uh compare everything, but here you
can use the the marketplace reviews. So, I can tell it here like hey hey, um scan the five pages, find me the best uh review to headphones. So, it could be even useful for you as a personal in in a personal way, right? So, instead of it getting blocked, it can actually find you things that you need on websites that are behind Cloudflare. Oh, I forgot to delete the old one. No. No, stop. Stop. Stop. Hold on a second. Delete the old scraper. Scraper. I totally I was testing it and I totally forgot to clean up. Uh, let's do a different website. Pick a website. What's a popular marketplace in uh that's aggressive in blocking in UK? \>\> Very. \>\> Amazon? It's not that aggressive. You'll be surprised. I can scrape it with data center IPs. What's a popular website that everybody uses here? \>\> Very. Very. \>\> V- \>\> v e r y.com. \>\> V- Very? \>\> Very. \>\> Like that? \>\> y \>\> y.com? \>\> Yeah. \>\> Oh, that's the UK? Okay, perfect.
So, this way I don't I don't want to Stop it. New session. Uh Let's do that.
Let's do that. Where's the new session? Oh, there it is. Really? What is wrong with my paste? Okay. Uh, it's clothing store? \>\> Uh, it's category. \>\> Okay, so let's do headphones again, the same thing, right? So, I don't know. I've never used this website. I've Look. Clean test. No cheating. Um So So, what else does this give you? So, again, market research. Um For example, I was looking for a new apartment, right? I would want to move a house. So, with a CloudCrawl, with BrightData, I set up a listener. Literally just telling Hey, listen. Build me a scraper that will run every half an hour when in this area there's a a house, private house, under this price, notify me. That's all I did. In a few days, I got a notification and now I live there. So, these things are not only useful on a scale where, you know, oh, I need to scrape millions of records, but even for your personal use. It's so easy these days. CloudCrawl does an amazing job, right? I know Crawlera does a great job as well, but I prefer CloudCrawl because I don't know. It just gives me less headache. Crawlera sometimes takes me on a wild goose chase. Um how many of you use VibeCode?
It's It's amazing. Why not, right? Anybody can build anything now. Um So, did you build it? I've got the structure, the search patterns. I don't know what third party I'm going to build. Oh, okay. It's building a
scraper. Um This used to take, I don't know. If you If you're having a good day, maybe a full day, maybe a day and a half, right? You need to go check out the selectors, figure it out. It used to be interesting and fun, but you don't need to do that anymore. So, what I'm trying to show you guys is that with our MCP, with our infrastructure, we have over 150 million IPs. Uh with our unlocking technology where we even though if it needs to run a remote browser, so if you want to It can actually write you a browser automation. Uh Uh the browsers are running on our system. So, I can literally right now open a thousand browsers on this laptop that are running on our servers. And they can go do things whatever you needed to. So, it already got 90 products. Okay, it just needs to fix the Unicode because it's in pounds. Almost done. \>\> Can you also go into details about the MCP or is that it? \>\> Sure, I did it in the previous session, but while it's loading, let me tell you. So, basically, the MCP gives you a agent 66 tools. Uh some of them is basically uh we have a system where it can send a curl to any URL and our system will literally get the HTML back. It will solve a capture if needed and send it with a token. It will It knows exactly what headers and cookies the website needs. So, basically, it will make sure that the server thinks it's a browser and serve the HTML back. So, your agent can literally send curls, pull data without any questions. It can pull a full HTML. It can pull just a scrape as a markdown. Markdown is It's to save tokens, right? You just want the text of the page. You don't care about the HTML tags. It doesn't like the pound. Come on, you can do it. Um so, that's number one. Second, we
have about 500 different APIs pre-built for different domains. So, instead of actually getting markdown, you can actually get a JSON of the product. For example, for Amazon, we have pre-built API. You can listen when
you add your agent, it can be like, "Okay, go and check on Amazon something." It doesn't even need to build a scraper. It can literally just send the the request and get the data back. Um on top of that, remote browser infrastructure and the anything that needs anything that your agent needs to access the web. It has, right? So, you don't get blocked. While this is running, I want to show you for example uh tell one to Walmart do a search for headphones without MCP one So, I'm telling it go to Walmart, do a search for headphones, and tell me what is the first result. Without the Bright Data MCP. Uh It's going to do a fetch, which is going to get blocked. Product security verification screen, robot or human, obviously, right? That's That's the first thing. Now, do the same with Bright Um Oh my god. So, now it's using Scrape as a markdown. It's doing a search. It's only pulling the text. It's not pulling the HTML, just the text itself.
Always when you do a live, it's slows down. The time is like Oh, actually, it could be. It could be
opening a browser and actually holding the button. You know, the Walmart has like one of those click and hold, and it needs to hold it for like 30 seconds. So, it could be doing that right now. So, is it We have built to capture solving solutions. Like, we literally have in-house We have a AI solving capture. And moving, clicking things, and like what? It's perfect. So, it already got first result, blue headphones. Oh, basically, there you go. It already has the results of the headphones without robot or human with full product listing name. Uh so, we here already have the output. Tell me if you were to do this manually, how many how much more tokens have you used? How many did you save? Let's just give it a breakdown, and this is probably the biggest issue, right? Tokens is expensive. I'm going through millions and millions of tokens a day. I used to go more, but I I optimized it. So, um all of you are looking how do we save money on LM? How do we waste less tokens of web access and um Bright Data has the solution. Instead of parsing the full HTML, create a scraper. Uh instead of using um you know, okay, it pulls the HTML and
then it extracts the data, it builds the the parser. But, it's working very slow right now. I feel like everybody's white coding. So, basically, here's the breakdown,
right? What do we have here? The big swings output parsing 90 products with input tokens, output tokens, total save. So, it's about a 62% save of tokens. This is not a high number. This website, I guess, uh has um maybe a structured um HTML. I'm not sure. But, this is what I wanted to show you. And the best thing about it is that it will maintain it. If it breaks it, it will fix it. I can set up a loop, right? So, for example, in Cloud, I can set a schedule every 30 minutes you do and run and check something. And this is we are giving you guys, okay? With our MCP, your agent has access to all the web uh without getting blocked. No capture will stop it. No robots antibot systems will stop it. And uh you have no headaches of that. I don't know how many of you have You guys probably dealt with blocking, right? When you build a scraper, it gets really complicated. So, this kind of removes all the headache for it. Um any questions? \>\> Where is the code? \>\> Excuse me? \>\> The code the code you generate a scraper, right? \>\> I can I If you you want to see the code? I mean, it's here. We have the scraper. Let's check it out.
So, it's using API web unlocker. This is the same thing as a scraper as a markdown with an MCP. It sends a simple request and gets the data back. This is the parser that it built.
I mean, it looks pretty good. This is the the schema that it built for the output. It's doing a keyword search. It has the max page, right? So, I asked for two inputs, the keyword search and how many pages to scrape. And hold on. Let's see Let's see We can run it through here as well. Where is the variables? API key is on key. Let's run it. Do another run on laptops. What are the two pages results? And now you kind of control it. Like you don't even need to run it, right? It will trigger it for you. It will maintain it for you. Um you can create a script that will automatically do all of this. I prefer the visual interface for the demonstration. Sometimes they you like looking at our code it's a little more complicated. But right now basically I can ask it any questions. It will actually run the script. So it's not wasting tokens to execute the script maybe like 60 tokens, right right now. So it's going to literally get all this data for 100 tokens. And then you can do whatever you can ask it questions about it. But now we're working with a JSON and JSON is much more token efficient than any markdown or any other file, right? Still the usual.
\>\> Can I ask can you can you do this on all the authorization sites as well? \>\> Uh no, we only talk we only deal with public data. So behind login it's um it's private data. And and uh just you
know just a heads-up. If you create an account, you accept terms and conditions of the website. Always check terms and conditions of the website. If they say don't scrape, don't use robots, and you do, then that company can actually sue you. And this is happening you probably heard it in the news all the time. LinkedIn is suing them. Everybody's suing each other because they the data right now is like the new gold the oil, right? Everybody's trying to like oh this is my data. Elon Musk took over Twitter, locked it down. That's it. Literally like uh the it used to be so open and now it's all locked in a few accounts that you can actually scrape. So the we're seeing this on every single level. Very slow today. Um so yes, also same thing if you accept uh there's like terms and like if you do a search and there's a checkpoint, I accept terms and conditions, always be careful with that. So we only deal with public data. So nothing behind login. We don't accept terms and conditions. And we actually won a few lawsuits. We were sued by Meta. We were so sued by Elon Musk a month after he took over. And uh the judge said it's very simple. Public data is public data. It doesn't matter how you collect it. Doesn't matter what you do with it. It's public. You know, it's like walking on the street, you write down the prices on the counter and then you sell it to somebody. It's public data. It's available. You can do whatever you want with it. Okay? So, as usual, it gets stuck on a live demo, but we can we can run this manually.
Oh my god.
What the I'm setting a variable. It's doing this. No, it's not going to work. What am I missing? Still stuck? Oh, okay. It's done. Okay, so here we got we got Let's see. How many tokens? How many tokens did you spend on getting the these? So, even if you're not be able to building pipelines for any you know, big companies and you don't need to get for personal use, I always have the MCP connected. I don't ask the you know, LLM to go do I always ask build a script that can later on be used by it. It's using its own script to save the tokens. \>\> Literally like we're talking about 1,000 tokens where if you know if if it needs to go through the JSON, it's like 10,000 tokens. We're talking about like literally pennies compared to what it would be to actually scrape it. Any more questions? How about this? Anybody has problems with actually getting data? No, no question. Everybody is like I'm not sharing my problems. I have no problems. And they go home and like oh I do have a problem. But in any case, listen, we have a booth. If you don't if you want to talk in private and you have questions about data or access of LLM,
feel free. I'm always willing to help. Um we can connect on LinkedIn by the way, guys, if you want. Let me just summarize a few things what we did here.
So um self-healing pipeline. When you have when your agent actually has access to a blocked website and you're working I mean, the MCP is mostly useful in about 20% of the of the domains, right? The ones that have Akamai, Data Dome, Cloudflare, those heavy protected domains. They're the usually the most juiciest one, right? Like real estate or uh big e-commerce places. Uh with our MCP, it has access to it. If it has access to it, it can build a scraper. If it can build a scraper, it can maintain the scraper. This is basically the whole thing that I wanted to show you. That in Bright Data, we have all the tools that your agent might need to explore, to build and maintain uh a pipeline, right? So, if you want to set up a listener for an apartment and you're looking to move into a cheaper place or maybe you want to book a table in a restaurant where it's always packed. And literally I I I have a listener right now. I'm waiting for 2 months already. Everybody is like booking right away. So, as soon as the spot opens, it will automatically book a spot for me. It can be very useful in just kind of like even even the small things, personal things, right? I'm not talking about just enterprise scale where I need to download a million records. Okay? But having access to all the websites is this is important. \>\> So, apart from scraping can you also do actions on the website? \>\> Yes, of course. Fill up a form, submit it as well. Yes. The only thing you can't do is log in. Right? So, if you have let's say you
need to perform a search, right? You cannot generate the URL, you need to click buttons. Let's say flights. You want to check flights availability, Skyscanner or something like that, right? The URL is usually a hash and you
can't do anything with it. So, yes, LLM can spool a browser, remote browser, even if it's a geo-restricted site. It can be like, "Okay, I want IP from the United States." So, it's open a browser with the United States IP and then it goes and clicks things, fills it up, and so on. The beauty of it is that our browser will mimic real human behavior. So, when your agent clicks, it's not a teleportation. There's a mouse pre-recorded like a real human being moving it. When it types, it will type a little slower, speed up, like maybe even mistake, and so on. So, we have pre-recorded typing, we have pre-recorded mouse movements. So, if the website has a tracker that's constantly sending to the server what the user is doing, it will look like a real human being. Doesn't matter what your agent, even if it's a low like for browsing agents, I'm never used the top models, right? For example, if with Claude Haiku model is more than enough for browsing. If it's being masked that it's a real human, it works just fine. And so, that's pretty much it. If you guys want to connect on LinkedIn, feel free. I'm always willing to help. If you have any questions regarding how to get data, if you have problems with accessing anything, feel free to message me. I feel like there's been so much more than 15 minutes. I love it. It's the second time I'm doing a speech and I got like half an hour instead of 15 minutes. It's amazing. Um Now's the time if you have any questions, discussions. No, nothing? Great. Okay, guys, I guess I will conclude this session in this Feel free to come to the booth on the third floor if you have more questions and um let's keep the public data public. Thank you.
