browser-harness - self-healing LLM browser automation

Keyword:

browser-harness

·

self-healing browser

·

llm browser automation

·

cdp agent

Direct WebSocket connection between an LLM agent and Chrome via Chrome DevTools Protocol. The agent can inspect the page, write helper code, reuse domain skills, and verify the task without an extra browser abstraction layer.

agent-browser

When to use this skill

• The user needs an LLM agent to complete a multi-step browser workflow: login, navigation, form fill, data extraction, download, or verification.
• The workflow needs a clean browser profile or repeatable CDP verification instead of the user's already-open browser state.
• The user is running from Codex CLI or Antigravity and needs a local browser harness the agent can operate with shell/Python commands.
• Claude reports image/screenshot/tool errors when browser screenshots are written, resized, or re-opened.
• The target DOM changes and the agent should add or repair helpers in

  agent-workspace/agent_helpers.py

  .
• The task benefits from site-specific domain skills in

  agent-workspace/domain-skills/

  .
• Browser Use Cloud is justified for concurrent browsers, proxies, or captcha solving on allowed targets.

Do not use this skill when

• The task is simple HTML extraction without browser state or JS interaction -> route to

  scrapling

  .
• The task is exact human UI annotation or pointing at a rendered issue -> route to

  agentation

  .
• The task must reuse the user's already-open authenticated Chrome profile -> route to

  playwriter

  .
• The task is React component source capture -> route to

  react-grab

  .
• The task is ordinary Playwright/Puppeteer script authoring without agent autonomy -> use that stack directly.

Instructions

Step 1: Choose the execution packet

• local-cdp: local Chrome/Chromium with

  --remote-debugging-port=9222

  .
• codex-cdp: Codex CLI controls the same local checkout and CDP endpoint.
• antigravity-cdp: Antigravity (

  agy

  ) uses the same workspace and Chrome debugging endpoint.
• claude-vision-safe: screenshot capture must use the safe image pipeline below.
• domain-skill: add or repair a site-specific helper in

  agent-workspace/domain-skills/

  .
• cloud-browser: Browser Use Cloud is needed and allowed.

Step 2: Install browser-harness

git clone https://github.com/browser-use/browser-harness.git
cd browser-harness
python3 -m venv .venv
source .venv/bin/activate
pip install -e .

Set up https://github.com/browser-use/browser-harness for me

• Python 3.10+
• Chrome or Chromium

• http://localhost:9222/json

  reachable from the agent runtime

Step 3: Enable Chrome remote debugging

# macOS
/Applications/Google\ Chrome.app/Contents/MacOS/Google\ Chrome \
  --remote-debugging-port=9222 --user-data-dir=/tmp/chrome-debug

# Linux
google-chrome --remote-debugging-port=9222 --user-data-dir=/tmp/chrome-debug

# Windows PowerShell
& "C:\Program Files\Google\Chrome\Application\chrome.exe" `
  --remote-debugging-port=9222 --user-data-dir="$env:TEMP\chrome-debug"

curl -s http://localhost:9222/json

Step 4: Platform-specific notes

.venv

agy

agy

localhost:9222

cd ~/browser-harness
source .venv/bin/activate
python -c "import browser_harness; print('browser-harness OK')"
curl -s http://localhost:9222/json

Step 5: Apply the Claude-safe screenshot patch

src/browser_harness/helpers.py

diff --git a/src/browser_harness/helpers.py b/src/browser_harness/helpers.py
--- a/src/browser_harness/helpers.py
+++ b/src/browser_harness/helpers.py
@@
-import base64, importlib.util, json, math, os, sys, time, urllib.request
+import base64, importlib.util, io, json, math, os, sys, time, urllib.request
@@
-            img = Image.open(path)
+            with Image.open(path) as src:
+                img = src.copy()
@@
-    open(path, "wb").write(base64.b64decode(r["data"]))
+    data = base64.b64decode(r["data"])
     if max_dim:
         from PIL import Image
-        img = Image.open(path)
+        img = Image.open(io.BytesIO(data))
         if max(img.size) > max_dim:
             img.thumbnail((max_dim, max_dim))
-            img.save(path)
+            buf = io.BytesIO()
+            img.save(buf, format="PNG")
+            data = buf.getvalue()
+    with open(path, "wb") as f:
+        f.write(data)

• Image.open(path)

  keeps a lazy file handle unless copied or closed.
• Claude image/tool pipelines are more likely to fail when a PNG is opened, rewritten, then reopened by the agent in quick succession.
• In-memory resize via

  io.BytesIO

  avoids the write-read-write cycle.
• Writing once with

  with open(path, "wb")

  produces a stable file for Claude vision upload.

path = capture_screenshot(max_dim=1800)

max_dim=1800

Step 6: Run browser tasks

Open the local app, complete the signup form, and verify that the dashboard appears.
Navigate to GitHub, open the first open issue, and summarize the acceptance criteria.
Fill in the contact form at example.com and confirm the success message.

1. Connect to Chrome via CDP.
2. Inspect tabs and page state.
3. Reuse existing helpers in

   agent-workspace/agent_helpers.py

   .
4. Add missing helpers in

   agent-workspace/agent_helpers.py

   or

   agent-workspace/domain-skills/

   .
5. Verify completion with text, URL, DOM state, screenshot, or downloaded artifact evidence.

Step 7: Extend with domain skills

agent-workspace/domain-skills/
├── github.py
├── linkedin.py
└── your-site.py

def login(page, username: str, password: str):
    """Log into mysite.com."""
    page.goto("https://mysite.com/login")
    page.fill("#username", username)
    page.fill("#password", password)
    page.click("button[type=submit]")
    page.wait_for_url("**/dashboard")

Step 8: Browser Use Cloud escalation

from browser_harness import BrowserUseCloud

client = BrowserUseCloud(api_key="YOUR_API_KEY")
result = client.run("Extract the dashboard data and return a CSV summary")
print(result)

Best practices

1. Start with

   local-cdp

   ; escalate only when the local CDP endpoint cannot satisfy the job.
2. Keep core package edits minimal. Put ordinary workflow logic in

   agent_helpers.py

   or domain skills.
3. Apply the Claude-safe screenshot patch before image-heavy Claude Code runs.
4. For Codex and Antigravity, prefer explicit shell/Python commands over Claude-only plugin instructions.
5. Treat every browser task as incomplete until the agent records final evidence.
6. Use

   scrapling

   for stateless scraping and

   playwriter

   for already-open authenticated browser reuse.
7. Do not bypass site terms, robots, rate limits, or authorization boundaries.

Quick verification

cd ~/browser-harness
source .venv/bin/activate
python -c "import browser_harness; print('browser-harness OK')"
curl -s http://localhost:9222/json

References

• browser-use/browser-harness GitHub
• scrapling — stateless HTML/JS scraping without agent-owned browser state
• playwriter — running-browser reuse when existing login/session state matters
• agentation — rendered-UI feedback and human annotation packets