scrapfly-browser

Compare original and translation side by side

🇺🇸

Original

English

🇨🇳

Translation

Chinese

Scrapfly Cloud Browser

Use the Scrapfly Cloud Browser API with Python Playwright to automate remote cloud browsers with built-in proxy rotation, anti-bot fingerprinting, and geo-targeting.

使用 Scrapfly Cloud Browser API 与 Python Playwright 自动化远程云端浏览器，提供代理轮换、浏览器指纹管理和地理定向功能，以应对反机器人防护。

When to use

适用场景

Automating browser interactions on remote cloud browsers (login flows, form filling, navigation)
Scraping JavaScript-heavy sites that require full browser automation
Bypassing anti-bot protections with managed browser fingerprinting
Running Playwright scripts through geo-targeted proxies
Maintaining persistent browser sessions across multiple connections
Downloading files through browser interactions

在远程云端浏览器上执行交互自动化（登录流程、表单填写、页面导航）
抓取需要完整浏览器自动化的 JavaScript 密集型站点
借助托管的浏览器指纹模拟绕过反机器人防护
通过地理定向代理运行 Playwright 脚本
在多次连接之间维持持久浏览器会话
通过浏览器交互下载文件

Setup

初始化

bash

pip install playwright

The API key must be provided via environment variable

SCRAPFLY_API_KEY

or passed directly in the connection URL.

bash

pip install playwright

API 密钥必须通过环境变量

SCRAPFLY_API_KEY

提供，或直接传入连接 URL。

API Reference

API 参考

Connection: WebSocket CDP (Chrome DevTools Protocol)

wss://browser.scrapfly.io?api_key=YOUR_API_KEY

连接： WebSocket CDP（Chrome DevTools Protocol）

wss://browser.scrapfly.io?api_key=YOUR_API_KEY

Connection Parameters

连接参数

Parameter	Type	Default	Description
`api_key`	str	required	Scrapfly API key
`proxy_pool`	str	`"datacenter"`	Proxy type: `"datacenter"` or `"residential"` . The Web Scraping API words `basic` , `unblocker` , `unblocker_ultra` and `auto` do not apply to the Cloud Browser, which refuses them with `ERR::BROWSER::CONFIG_ERROR`
`os`	str	random	OS fingerprint: `"linux"` , `"windows"` , or `"macos"`
`country`	str	None	Proxy country (ISO 3166-1 alpha-2, e.g. `"us"` , `"de"` )
`session`	str	None	Session ID for browser state persistence
`auto_close`	str	`"true"`	Close session on disconnect. Set `"false"` to keep alive
`timeout`	int	900	Max session duration in seconds (900-1800)
`debug`	str	`"false"`	Enable session video recording
`downloads_stream`	str	None	`"1"` returns large downloads as IO stream handles instead of inline base64

参数	类型	默认值	说明
`api_key`	str	必填	Scrapfly API 密钥
`proxy_pool`	str	`"datacenter"`	代理类型： `"datacenter"` 或 `"residential"` 。Web Scraping API 中的 `basic` 、 `unblocker` 、 `unblocker_ultra` 和 `auto` 不适用于 Cloud Browser；Cloud Browser 会拒绝这些值并返回 `ERR::BROWSER::CONFIG_ERROR`
`os`	str	随机	操作系统指纹： `"linux"` 、 `"windows"` 或 `"macos"`
`country`	str	None	代理所在国家/地区（ISO 3166-1 alpha-2，例如 `"us"` 、 `"de"` ）
`session`	str	None	用于持久保存浏览器状态的会话 ID
`auto_close`	str	`"true"`	断开连接时关闭会话。设置为 `"false"` 可使会话保持存活
`timeout`	int	900	会话最长持续时间（秒）（900-1800）
`debug`	str	`"false"`	启用会话视频录制
`downloads_stream`	str	None	`"1"` 将大文件作为 IO 流句柄返回，而不是以内联 base64 返回

Building the Connection URL

构建连接 URL

python

import os
from urllib.parse import urlencode

params = {
    "api_key": os.environ["SCRAPFLY_API_KEY"],
    "proxy_pool": "datacenter",
    "os": "linux",
}
BROWSER_WS = f"wss://browser.scrapfly.io?{urlencode(params)}"

python

import os
from urllib.parse import urlencode

params = {
    "api_key": os.environ["SCRAPFLY_API_KEY"],
    "proxy_pool": "datacenter",
    "os": "linux",
}
BROWSER_WS = f"wss://browser.scrapfly.io?{urlencode(params)}"

Examples

示例

Basic page navigation

基础页面导航

python

from playwright.sync_api import sync_playwright
import os

API_KEY = os.environ["SCRAPFLY_API_KEY"]
BROWSER_WS = f"wss://browser.scrapfly.io?api_key={API_KEY}&proxy_pool=datacenter&os=linux"

with sync_playwright() as p:
    browser = p.chromium.connect_over_cdp(BROWSER_WS)
    try:
        context = browser.contexts[0]
        page = context.pages[0] if context.pages else context.new_page()

        page.goto("https://web-scraping.dev")
        print("Title:", page.title())
        print("Content:", page.content()[:500])
    finally:
        browser.close()

python

from playwright.sync_api import sync_playwright
import os

API_KEY = os.environ["SCRAPFLY_API_KEY"]
BROWSER_WS = f"wss://browser.scrapfly.io?api_key={API_KEY}&proxy_pool=datacenter&os=linux"

with sync_playwright() as p:
    browser = p.chromium.connect_over_cdp(BROWSER_WS)
    try:
        context = browser.contexts[0]
        page = context.pages[0] if context.pages else context.new_page()

        page.goto("https://web-scraping.dev")
        print("Title:", page.title())
        print("Content:", page.content()[:500])
    finally:
        browser.close()

Async Playwright

异步 Playwright

python

from playwright.async_api import async_playwright
import asyncio
import os

API_KEY = os.environ["SCRAPFLY_API_KEY"]
BROWSER_WS = f"wss://browser.scrapfly.io?api_key={API_KEY}&proxy_pool=datacenter&os=linux"

async def main():
    async with async_playwright() as p:
        browser = await p.chromium.connect_over_cdp(BROWSER_WS)
        try:
            context = browser.contexts[0]
            page = context.pages[0] if context.pages else await context.new_page()

            await page.goto("https://web-scraping.dev")
            print("Title:", await page.title())
        finally:
            await browser.close()

asyncio.run(main())

python

from playwright.async_api import async_playwright
import asyncio
import os

API_KEY = os.environ["SCRAPFLY_API_KEY"]
BROWSER_WS = f"wss://browser.scrapfly.io?api_key={API_KEY}&proxy_pool=datacenter&os=linux"

async def main():
    async with async_playwright() as p:
        browser = await p.chromium.connect_over_cdp(BROWSER_WS)
        try:
            context = browser.contexts[0]
            page = context.pages[0] if context.pages else await context.new_page()

            await page.goto("https://web-scraping.dev")
            print("Title:", await page.title())
        finally:
            await browser.close()

asyncio.run(main())

Login flow with form filling

登录流程与表单填写

python

with sync_playwright() as p:
    browser = p.chromium.connect_over_cdp(BROWSER_WS)
    try:
        context = browser.contexts[0]
        page = context.pages[0] if context.pages else context.new_page()

        page.goto("https://web-scraping.dev/login")
        page.fill("input[name=username]", "user123")
        page.fill("input[name=password]", "password")
        page.click("button[type=submit]")
        page.wait_for_selector("div#secret-message")

        print("Logged in:", page.url)
    finally:
        browser.close()

python

with sync_playwright() as p:
    browser = p.chromium.connect_over_cdp(BROWSER_WS)
    try:
        context = browser.contexts[0]
        page = context.pages[0] if context.pages else context.new_page()

        page.goto("https://web-scraping.dev/login")
        page.fill("input[name=username]", "user123")
        page.fill("input[name=password]", "password")
        page.click("button[type=submit]")
        page.wait_for_selector("div#secret-message")

        print("Logged in:", page.url)
    finally:
        browser.close()

Take a screenshot

截取屏幕截图

python

with sync_playwright() as p:
    browser = p.chromium.connect_over_cdp(BROWSER_WS)
    try:
        context = browser.contexts[0]
        page = context.pages[0] if context.pages else context.new_page()

        page.goto("https://web-scraping.dev")
        page.screenshot(path="screenshot.png", full_page=True)
    finally:
        browser.close()

python

with sync_playwright() as p:
    browser = p.chromium.connect_over_cdp(BROWSER_WS)
    try:
        context = browser.contexts[0]
        page = context.pages[0] if context.pages else context.new_page()

        page.goto("https://web-scraping.dev")
        page.screenshot(path="screenshot.png", full_page=True)
    finally:
        browser.close()

Scrape data with selectors

使用选择器抓取数据

python

with sync_playwright() as p:
    browser = p.chromium.connect_over_cdp(BROWSER_WS)
    try:
        context = browser.contexts[0]
        page = context.pages[0] if context.pages else context.new_page()

        page.goto("https://web-scraping.dev/products")
        page.wait_for_selector("div.product")

        products = page.query_selector_all("div.product")
        for product in products:
            name = product.query_selector("a").inner_text()
            price = product.query_selector(".price").inner_text()
            print(f"{name}: {price}")
    finally:
        browser.close()

python

with sync_playwright() as p:
    browser = p.chromium.connect_over_cdp(BROWSER_WS)
    try:
        context = browser.contexts[0]
        page = context.pages[0] if context.pages else context.new_page()

        page.goto("https://web-scraping.dev/products")
        page.wait_for_selector("div.product")

        products = page.query_selector_all("div.product")
        for product in products:
            name = product.query_selector("a").inner_text()
            price = product.query_selector(".price").inner_text()
            print(f"{name}: {price}")
    finally:
        browser.close()

Geo-targeted browsing with residential proxy

使用住宅代理进行地理定向浏览

python

BROWSER_WS = (
    f"wss://browser.scrapfly.io?"
    f"api_key={API_KEY}"
    f"&proxy_pool=residential"
    f"&country=de"
    f"&os=windows"
)

with sync_playwright() as p:
    browser = p.chromium.connect_over_cdp(BROWSER_WS)
    try:
        context = browser.contexts[0]
        page = context.pages[0] if context.pages else context.new_page()

        page.goto("https://web-scraping.dev")
        print("Page content from DE proxy:", page.title())
    finally:
        browser.close()

python

BROWSER_WS = (
    f"wss://browser.scrapfly.io?"
    f"api_key={API_KEY}"
    f"&proxy_pool=residential"
    f"&country=de"
    f"&os=windows"
)

with sync_playwright() as p:
    browser = p.chromium.connect_over_cdp(BROWSER_WS)
    try:
        context = browser.contexts[0]
        page = context.pages[0] if context.pages else context.new_page()

        page.goto("https://web-scraping.dev")
        print("Page content from DE proxy:", page.title())
    finally:
        browser.close()

Persistent sessions (multi-step workflows)

持久会话（多步骤工作流）

Uses the same login credentials and selectors as the Login flow with form filling example.

python

LOGIN_URL = "https://web-scraping.dev/login"
USERNAME = "user123"
PASSWORD = "password"

使用与登录流程与表单填写示例相同的登录凭据和选择器。

python

LOGIN_URL = "https://web-scraping.dev/login"
USERNAME = "user123"
PASSWORD = "password"

Step 1: Login and keep session alive

BROWSER_WS = ( f"wss://browser.scrapfly.io?" f"api_key={API_KEY}" f"&session=my-session" f"&auto_close=false" f"&proxy_pool=datacenter" )

with sync_playwright() as p: browser = p.chromium.connect_over_cdp(BROWSER_WS) try: context = browser.contexts[0] page = context.pages[0] if context.pages else context.new_page()

    page.goto(LOGIN_URL)
    page.fill("input[name=username]", USERNAME)
    page.fill("input[name=password]", PASSWORD)
    page.click("button[type=submit]")
    page.wait_for_selector("div#secret-message")
    print("Login complete, session preserved")
finally:
    browser.close()  # Disconnects but session stays alive

BROWSER_WS = ( f"wss://browser.scrapfly.io?" f"api_key={API_KEY}" f"&session=my-session" f"&auto_close=false" f"&proxy_pool=datacenter" )

with sync_playwright() as p: browser = p.chromium.connect_over_cdp(BROWSER_WS) try: context = browser.contexts[0] page = context.pages[0] if context.pages else context.new_page()

    page.goto(LOGIN_URL)
    page.fill("input[name=username]", USERNAME)
    page.fill("input[name=password]", PASSWORD)
    page.click("button[type=submit]")
    page.wait_for_selector("div#secret-message")
    print("Login complete, session preserved")
finally:
    browser.close()  # Disconnects but session stays alive

Step 2: Reconnect to same session (still authenticated with same credentials)

BROWSER_WS = ( f"wss://browser.scrapfly.io?" f"api_key={API_KEY}" f"&session=my-session" f"&proxy_pool=datacenter" )

with sync_playwright() as p: browser = p.chromium.connect_over_cdp(BROWSER_WS) try: context = browser.contexts[0] page = context.pages[0] if context.pages else context.new_page()

    page.goto(LOGIN_URL)
    page.wait_for_selector("div#secret-message")
    print("Accessed protected page (still logged in):", page.title())
finally:
    browser.close()

undefined

BROWSER_WS = ( f"wss://browser.scrapfly.io?" f"api_key={API_KEY}" f"&session=my-session" f"&proxy_pool=datacenter" )

with sync_playwright() as p: browser = p.chromium.connect_over_cdp(BROWSER_WS) try: context = browser.contexts[0] page = context.pages[0] if context.pages else context.new_page()

    page.goto(LOGIN_URL)
    page.wait_for_selector("div#secret-message")
    print("Accessed protected page (still logged in):", page.title())
finally:
    browser.close()

undefined

Infinite scroll scraping

无限滚动抓取

python

with sync_playwright() as p:
    browser = p.chromium.connect_over_cdp(BROWSER_WS)
    try:
        context = browser.contexts[0]
        page = context.pages[0] if context.pages else context.new_page()

        page.goto("https://web-scraping.dev/testimonials")
        page.wait_for_selector("span.rating")

        all_items = []
        for _ in range(10):  # Scroll 10 times
            items = page.query_selector_all("span.rating")
            all_items = items
            page.evaluate("window.scrollTo(0, document.body.scrollHeight)")
            page.wait_for_timeout(2000)

        print(f"Collected {len(all_items)} reviews")
    finally:
        browser.close()

python

with sync_playwright() as p:
    browser = p.chromium.connect_over_cdp(BROWSER_WS)
    try:
        context = browser.contexts[0]
        page = context.pages[0] if context.pages else context.new_page()

        page.goto("https://web-scraping.dev/testimonials")
        page.wait_for_selector("span.rating")

        all_items = []
        for _ in range(10):  # Scroll 10 times
            items = page.query_selector_all("span.rating")
            all_items = items
            page.evaluate("window.scrollTo(0, document.body.scrollHeight)")
            page.wait_for_timeout(2000)

        print(f"Collected {len(all_items)} reviews")
    finally:
        browser.close()

Execute JavaScript in page

在页面中执行 JavaScript

python

with sync_playwright() as p:
    browser = p.chromium.connect_over_cdp(BROWSER_WS)
    try:
        context = browser.contexts[0]
        page = context.pages[0] if context.pages else context.new_page()

        page.goto("https://web-scraping.dev")

        # Extract data using JavaScript
        data = page.evaluate("""
            () => {
                return {
                    title: document.title,
                    links: Array.from(document.querySelectorAll('a')).map(a => ({
                        text: a.innerText,
                        href: a.href,
                    })),
                };
            }
        """)
        print(f"Found {len(data['links'])} links")
    finally:
        browser.close()

python

with sync_playwright() as p:
    browser = p.chromium.connect_over_cdp(BROWSER_WS)
    try:
        context = browser.contexts[0]
        page = context.pages[0] if context.pages else context.new_page()

        page.goto("https://web-scraping.dev")

        # Extract data using JavaScript
        data = page.evaluate("""
            () => {
                return {
                    title: document.title,
                    links: Array.from(document.querySelectorAll('a')).map(a => ({
                        text: a.innerText,
                        href: a.href,
                    })),
                };
            }
        """)
        print(f"Found {len(data['links'])} links")
    finally:
        browser.close()

Multi-page navigation

多页面导航

python

with sync_playwright() as p:
    browser = p.chromium.connect_over_cdp(BROWSER_WS)
    try:
        context = browser.contexts[0]
        page = context.pages[0] if context.pages else context.new_page()

        urls = [
            "https://web-scraping.dev/products",
            "https://web-scraping.dev/products?page=2",
            "https://web-scraping.dev/products?page=3",
        ]

        results = []
        for url in urls:
            page.goto(url)
            results.append({"url": url, "content": content})

        print(f"Scraped {len(results)} pages")
    finally:
        browser.close()

python

with sync_playwright() as p:
    browser = p.chromium.connect_over_cdp(BROWSER_WS)
    try:
        context = browser.contexts[0]
        page = context.pages[0] if context.pages else context.new_page()

        urls = [
            "https://web-scraping.dev/products",
            "https://web-scraping.dev/products?page=2",
            "https://web-scraping.dev/products?page=3",
        ]

        results = []
        for url in urls:
            page.goto(url)
            results.append({"url": url, "content": content})

        print(f"Scraped {len(results)} pages")
    finally:
        browser.close()

File Downloads

文件下载

A download triggered in the page (a click behind a login, an anti-bot flow) is captured by the session and read back over CDP with the

ScrapiumBrowser.*

commands. They are browser-level: send them on a CDP session.

Command	Params	Returns
`ScrapiumBrowser.hasDownloads`	none	`{"result": true\|false}`
`ScrapiumBrowser.getDownloadsMetadatas`	none	`{"metadata": {"<filename>": <bytes>}}` — sizes only, no content, no state
`ScrapiumBrowser.getDownloadMetadatas`	`filename`	`{"size": <bytes>}` for one file
`ScrapiumBrowser.getDownload`	`filename` (required), `delete` (optional)	`{"data": "<base64>"}` — one file, content carried once
`ScrapiumBrowser.getDownloads`	`delete` (optional)	every file's content, carried twice: `files` (by name) and `files_by_guid`

页面中触发的下载（例如登录后的点击、反机器人验证流程）会被会话捕获，并通过 CDP 使用

ScrapiumBrowser.*

命令读回。这些命令属于浏览器级别：应在 CDP session 上发送。

命令	参数	返回值
`ScrapiumBrowser.hasDownloads`	无	`{"result": true\|false}`
`ScrapiumBrowser.getDownloadsMetadatas`	无	`{"metadata": {"<filename>": <bytes>}}` — 仅包含大小，不含内容和状态
`ScrapiumBrowser.getDownloadMetadatas`	`filename`	单个文件的 `{"size": <bytes>}`
`ScrapiumBrowser.getDownload`	`filename` （必填）、 `delete` （可选）	`{"data": "<base64>"}` — 单个文件，内容仅传输一次
`ScrapiumBrowser.getDownloads`	`delete` （可选）	所有文件的内容，内容会传输两次：分别位于 `files` （按名称索引）和 `files_by_guid` 中

Wait before you read

读取下载内容前请等待

Nothing queues the read behind the download. Listen for

Browser.downloadProgress

with

state == "completed"

, or poll

getDownloadsMetadatas

until the file is there and its size has stopped moving. Calling

getDownloads

straight after the click returns

{}

and your lookup raises

KeyError

python

import base64
import time

def wait_for_download(cdp, filename, timeout=60):
    """Poll until the file exists and its size is stable across two reads."""
    deadline = time.monotonic() + timeout
    last = -1
    while time.monotonic() < deadline:
        size = cdp.send("ScrapiumBrowser.getDownloadsMetadatas")["metadata"].get(filename)
        if size is not None and size == last:
            return size
        last = size if size is not None else -1
        time.sleep(0.5)
    raise TimeoutError(f"{filename} did not finish downloading")


with sync_playwright() as p:
    browser = p.chromium.connect_over_cdp(BROWSER_WS)
    try:
        context = browser.contexts[0]
        page = context.pages[0] if context.pages else context.new_page()
        cdp = context.new_cdp_session(page)

        page.goto("https://web-scraping.dev/file-download")
        page.click("#download-btn")

        size = wait_for_download(cdp, "download-sample.pdf")

        # One file by name: content is carried once, so the reply is ~1.34x the file.
        result = cdp.send("ScrapiumBrowser.getDownload", {"filename": "download-sample.pdf"})
        open("download-sample.pdf", "wb").write(base64.b64decode(result["data"]))
    finally:
        browser.close()

Downloaded files are cleaned up when the session ends. Pass

delete: True

getDownload

getDownloads

to reclaim disk during a long-running session.

读取操作不会自动排在下载任务之后。请监听

Browser.downloadProgress

且其

state == "completed"

，或者轮询

getDownloadsMetadatas

，直到文件出现且大小不再变化。点击后立即调用

getDownloads

会返回

{}

，随后执行查找将引发

KeyError

。

python

import base64
import time

def wait_for_download(cdp, filename, timeout=60):
    """Poll until the file exists and its size is stable across two reads."""
    deadline = time.monotonic() + timeout
    last = -1
    while time.monotonic() < deadline:
        size = cdp.send("ScrapiumBrowser.getDownloadsMetadatas")["metadata"].get(filename)
        if size is not None and size == last:
            return size
        last = size if size is not None else -1
        time.sleep(0.5)
    raise TimeoutError(f"{filename} did not finish downloading")


with sync_playwright() as p:
    browser = p.chromium.connect_over_cdp(BROWSER_WS)
    try:
        context = browser.contexts[0]
        page = context.pages[0] if context.pages else context.new_page()
        cdp = context.new_cdp_session(page)

        page.goto("https://web-scraping.dev/file-download")
        page.click("#download-btn")

        size = wait_for_download(cdp, "download-sample.pdf")

        # One file by name: content is carried once, so the reply is ~1.34x the file.
        result = cdp.send("ScrapiumBrowser.getDownload", {"filename": "download-sample.pdf"})
        open("download-sample.pdf", "wb").write(base64.b64decode(result["data"]))
    finally:
        browser.close()

会话结束时，已下载文件会被清理。对于长时间运行的会话，向

getDownload

getDownloads

传入

delete: True

可回收磁盘空间。

Large files: size the reply before you ask for it

大文件：请求前先评估响应大小

getDownloads

answers with the whole payload inside one CDP message, and a WebSocket message above the client's limit is a protocol error, not a slow read: the client closes with code

and the session dies with it, surfacing as

Target closed

Session closed

```
getDownloads
```
reply ≈ 2.7x the file — base64 costs a third and the content is carried under both
```
files
```
and
```
files_by_guid
```
.
```
getDownload
```
reply ≈ 1.34x the file. Prefer it when you know the name.
Server-side cap: 500 MB per file. An oversized file is never encoded, and the call it appears in fails:
```
getDownloads
```
answers with
```
download too large: "export.csv" (620000000 bytes) exceeds the 524288000 byte limit per file
```
and returns nothing, including the small files that were fine.
```
getDownloadsMetadatas
```
still lists the oversized file, and
```
getDownload
```
still serves the others by name — that pair is the recovery path.
Call
```
getDownloadsMetadatas
```
first and branch on the real size. Past a few tens of megabytes, stream instead of inlining.

getDownloads

会在一个 CDP 消息中返回完整负载。WebSocket 消息一旦超过客户端限制，就会触发协议错误，而不是仅仅导致读取变慢：客户端会以代码

关闭连接，会话也会随之终止，表现为

Target closed

Session closed

。

```
getDownloads
```
响应约为文件大小的 2.7 倍——base64 会增加约三分之一的开销，并且内容会同时通过
```
files
```
和
```
files_by_guid
```
传输。
```
getDownload
```
响应约为文件大小的 1.34 倍。已知文件名时应优先使用。
服务端上限：每个文件 500 MB。 超出上限的文件不会被编码，且包含该文件的调用会失败：
```
getDownloads
```
会返回
```
download too large: "export.csv" (620000000 bytes) exceeds the 524288000 byte limit per file
```
，并且不返回任何内容，包括原本正常的小文件。
```
getDownloadsMetadatas
```
仍会列出超大文件，而
```
getDownload
```
仍可按名称返回其他文件——将二者结合使用即可恢复。
应先调用
```
getDownloadsMetadatas
```
，再根据实际大小选择处理方式。超过几十 MB 后，应改用流式传输而不是内联返回。

Stream large downloads with

downloads_stream=1

使用

downloads_stream=1

流式传输大文件

Add

downloads_stream=1

to the Cloud Browser WebSocket URL. It is a connection option: nothing later in the session can turn it on.

getDownloads

then answers with a smaller shape — each file's content once under

files_by_guid

, with

guid_names

mapping GUID to filename, and anything above ~8 MB of base64 handed back under

streams

as an IO handle instead of inline content:

json

{
  "files": {},
  "files_by_guid": {"9F2C1A4E-...": "JVBERi0xLjQKJe..."},
  "guid_names": {"9F2C1A4E-...": "invoice.pdf", "B71D0E88-...": "export.csv"},
  "streams": {
    "B71D0E88-...": {"stream": "scrapfly-download:1", "size": 91250688, "name": "export.csv"}
  }
}

stream

value is an ordinary IO handle, drained with

IO.read

IO.close

python

import base64
from urllib.parse import urlencode
from playwright.sync_api import sync_playwright

params = {"api_key": API_KEY, "proxy_pool": "datacenter", "downloads_stream": "1"}
BROWSER_WS = f"wss://browser.scrapfly.io?{urlencode(params)}"

CHUNK = 8 * 1024 * 1024  # base64 characters, not raw bytes (32 MB is the cap)


def drain(cdp, handle, destination):
    """Read a scrapfly-download: handle to eof. Sequential only, no offset."""
    encoded = []
    try:
        while True:
            read = cdp.send("IO.read", {"handle": handle, "size": CHUNK})
            if read.get("data"):
                encoded.append(read["data"])
            if read.get("eof"):
                break
    finally:
        try:
            cdp.send("IO.close", {"handle": handle})
        except Exception:
            pass

    # Chunks are cut on base64 group boundaries, so they concatenate as text
    # and decode once.
    with open(destination, "wb") as out:
        out.write(base64.b64decode("".join(encoded)))


with sync_playwright() as p:
    browser = p.chromium.connect_over_cdp(BROWSER_WS)
    try:
        context = browser.contexts[0]
        page = context.pages[0] if context.pages else context.new_page()
        cdp = context.new_cdp_session(page)

        page.goto("https://web-scraping.dev/file-download")
        page.click("#download-btn")
        wait_for_download(cdp, "download-sample.pdf")

        result = cdp.send("ScrapiumBrowser.getDownloads")

        # Small files still arrive inline, large ones as handles. Handle both.
        for guid, b64 in result.get("files_by_guid", {}).items():
            open(result["guid_names"][guid], "wb").write(base64.b64decode(b64))
        for guid, stream in result.get("streams", {}).items():
            drain(cdp, stream["stream"], stream["name"])
    finally:
        browser.close()

Rules for download handles (they differ from Chromium's own IO streams):

Reads are sequential. Passing
```
offset
```
is an error: bytes are released as they are read.
size
counts base64 characters here, not raw bytes. Chunks are capped at 32 MB and default to 4 MB when no
```
size
```
is sent.
Loop until
eof
is true. An empty
```
data
```
without
```
eof
```
is not the end.
Always
IO.close
, including on the error path. An abandoned handle is reclaimed after 120 seconds, and reaching
```
eof
```
releases it, but closing is the contract.
Only
```
scrapfly-download:<n>
```
handles are served by the session.
```
blob:
```
handles,
```
Page.printToPDF
```
streams and every other IO handle reach Chromium untouched.

在 Cloud Browser WebSocket URL 中添加

downloads_stream=1

。这是一个连接选项：会话开始后无法再启用。启用后，

getDownloads

会返回更精简的结构——每个文件的内容在

files_by_guid

下仅出现一次，

guid_names

将 GUID 映射到文件名，而超过约 8 MB 的 base64 内容会通过

streams

以 IO 句柄形式返回，而不是作为内联内容返回：

json

{
  "files": {},
  "files_by_guid": {"9F2C1A4E-...": "JVBERi0xLjQKJe..."},
  "guid_names": {"9F2C1A4E-...": "invoice.pdf", "B71D0E88-...": "export.csv"},
  "streams": {
    "B71D0E88-...": {"stream": "scrapfly-download:1", "size": 91250688, "name": "export.csv"}
  }
}

stream

值是普通 IO 句柄，可使用

IO.read

IO.close

将其读取至结束：

python

import base64
from urllib.parse import urlencode
from playwright.sync_api import sync_playwright

params = {"api_key": API_KEY, "proxy_pool": "datacenter", "downloads_stream": "1"}
BROWSER_WS = f"wss://browser.scrapfly.io?{urlencode(params)}"

CHUNK = 8 * 1024 * 1024  # base64 characters, not raw bytes (32 MB is the cap)


def drain(cdp, handle, destination):
    """Read a scrapfly-download: handle to eof. Sequential only, no offset."""
    encoded = []
    try:
        while True:
            read = cdp.send("IO.read", {"handle": handle, "size": CHUNK})
            if read.get("data"):
                encoded.append(read["data"])
            if read.get("eof"):
                break
    finally:
        try:
            cdp.send("IO.close", {"handle": handle})
        except Exception:
            pass

    # Chunks are cut on base64 group boundaries, so they concatenate as text
    # and decode once.
    with open(destination, "wb") as out:
        out.write(base64.b64decode("".join(encoded)))


with sync_playwright() as p:
    browser = p.chromium.connect_over_cdp(BROWSER_WS)
    try:
        context = browser.contexts[0]
        page = context.pages[0] if context.pages else context.new_page()
        cdp = context.new_cdp_session(page)

        page.goto("https://web-scraping.dev/file-download")
        page.click("#download-btn")
        wait_for_download(cdp, "download-sample.pdf")

        result = cdp.send("ScrapiumBrowser.getDownloads")

        # Small files still arrive inline, large ones as handles. Handle both.
        for guid, b64 in result.get("files_by_guid", {}).items():
            open(result["guid_names"][guid], "wb").write(base64.b64decode(b64))
        for guid, stream in result.get("streams", {}).items():
            drain(cdp, stream["stream"], stream["name"])
    finally:
        browser.close()

下载句柄的规则（它们与 Chromium 自带的 IO 流不同）：

读取必须按顺序进行。 传入
```
offset
```
会报错：字节会随读取被释放。
size
在这里统计的是 base64 字符数，而不是原始字节数。分块最大为 32 MB；未发送
```
size
```
时默认为 4 MB。
持续循环，直到
eof
为 true。
```
data
```
为空但
```
eof
```
不为 true 并不表示结束。
务必调用
IO.close
，错误路径中也不例外。放弃的句柄会在 120 秒后被回收；读取到
```
eof
```
也会释放句柄，但显式关闭仍是必须遵循的约定。
会话只会提供
```
scrapfly-download:<n>
```
句柄。
```
blob:
```
句柄、
```
Page.printToPDF
```
流及其他所有 IO 句柄不会被处理，而会原样传递给 Chromium。

No download at all: stream a URL out of the page

完全不触发下载：从页面流式读取 URL

When the URL is known and the browser download machinery is not needed, fetch it in the page and stream the Blob. Nothing large ever crosses the wire, and the 500 MB cap does not apply on this path.

python

CHUNK = 10 * 1024 * 1024  # raw bytes here; the reply inflates ~1.34x

evaluated = cdp.send("Runtime.evaluate", {
    "expression": f"fetch({url!r}).then(r => r.blob())",
    "awaitPromise": True,
})
resolved = cdp.send("IO.resolveBlob", {"objectId": evaluated["result"]["objectId"]})
handle = "blob:" + resolved["uuid"]

当 URL 已知且不需要浏览器的下载机制时，可以在页面中 fetch 该 URL 并流式传输 Blob。这样不会有大数据通过传输链路，且此路径不受 500 MB 上限约束。

python

CHUNK = 10 * 1024 * 1024  # raw bytes here; the reply inflates ~1.34x

evaluated = cdp.send("Runtime.evaluate", {
    "expression": f"fetch({url!r}).then(r => r.blob())",
    "awaitPromise": True,
})
resolved = cdp.send("IO.resolveBlob", {"objectId": evaluated["result"]["objectId"]})
handle = "blob:" + resolved["uuid"]

Drain with IO.read / IO.close, decoding each chunk when base64Encoded is true.


`Page.printToPDF` takes `transferMode: "ReturnAsStream"` and returns a handle the
same way.

**Do not use `Network.getResponseBody` for a large body.** Chromium evicts big
response bodies from the inspector cache before you can ask
(`Request content was evicted from inspector cache`), and for a text content type
it returns the body already decoded, which silently corrupts binary payloads.


`Page.printToPDF` 使用 `transferMode: "ReturnAsStream"` 时，也会以同样的方式返回一个句柄。

**不要对大型响应正文使用 `Network.getResponseBody`。** Chromium 会在你请求之前将大型响应正文从 Inspector 缓存中淘汰（`Request content was evicted from inspector cache`）；当内容类型为文本时，返回的正文已经被解码，从而悄悄破坏二进制负载。

Anti-Bot Features

反机器人防护功能

The Cloud Browser automatically manages browser fingerprinting:

TLS/JA3 fingerprints matching real browsers
User-Agent, device metrics, timezone, locale (based on
```
country
```
)
WebGL and Canvas fingerprint emulation
HTTP/2 fingerprints

Cloud Browser 会自动管理浏览器指纹：

与真实浏览器匹配的 TLS/JA3 指纹
User-Agent、device metrics、timezone、locale（基于
```
country
```
）
WebGL 和 Canvas 指纹模拟
HTTP/2 指纹

Important Notes

重要说明

Always use
connect_over_cdp()
(not
```
connect()
```
) to connect to the Cloud Browser
Always use
browser.contexts[0]
to get the pre-configured context. Do NOT create new contexts - the Cloud Browser provides a context with proper fingerprinting
Always close the browser in a
try/finally
block to stop billing. Forgetting to close means continuous charges
Do NOT set fingerprint overrides (user agent, viewport, timezone, etc.) - they are managed automatically
Sessions expire after 1 hour of inactivity
Maximum session duration is 30 minutes (1800 seconds)
Maximum file download size is 500 MB per file, and one oversized file fails the whole
```
getDownloads
```
call. Past a few tens of megabytes use
```
downloads_stream=1
```
and drain the IO handle, see File Downloads
Start with
```
proxy_pool=datacenter
```
, upgrade to
```
residential
```
only when needed for anti-bot bypass
Use
```
debug=true
```
for session video recording when troubleshooting

务必使用
connect_over_cdp()
（而不是
```
connect()
```
）连接 Cloud Browser
务必使用
browser.contexts[0]
获取预配置的 context。不要创建新 context——Cloud Browser 已提供一个具备正确指纹模拟的 context
务必在
try/finally
块中关闭 browser，以停止计费。忘记关闭将持续产生费用
不要设置指纹覆盖项（User-Agent、viewport、timezone 等）——这些均由系统自动管理
会话在 1 小时无活动后过期
会话最长持续时间为 30 分钟（1800 秒）
每个文件的最大下载大小为 500 MB；一个超大文件会导致整个
```
getDownloads
```
调用失败。超过几十 MB 时，请使用
```
downloads_stream=1
```
并读取 IO 句柄，参见文件下载
初始使用
```
proxy_pool=datacenter
```
，仅在需要绕过反机器人防护时升级为
```
residential
```
排查问题时，使用
```
debug=true
```
录制会话视频