Expose site actions
Built WebMCP tool suites that surfaced site actions as navigator.modelContext tools callable by LLM agents.
HUAWEI · WEB TECHNOLOGY LAB · 2026
WebMCP & Browser Agent Evaluation
I built the tooling and evaluation infrastructure used to study how LLM agents act across real web environments—and to separate model behavior from failures in the surrounding system.
- 9+ sites
- WebMCP tool suites
- 5 modes
- WebArena study design
- 111 / 36
- dispatch failures / tasks
- 38% → 18%
- element-ID error rate
THE QUESTION
Can browser-native tools make agent evaluation more reliable?
Browser agents can fail because of the model, the page, the tool catalog, or the evaluation harness. The study needed infrastructure that could expose site actions consistently, compare five execution modes, and identify where a failure actually entered the stack.
MY ROLE
Build the system around the experiment.
Make runs reproducible
Engineered a Python evaluation harness with a site-injection proxy and automated tool verification across Docker-hosted sites.
Support comparison
Built and debugged browser-agent infrastructure for a WebArena A/B study spanning five modes and multiple models.
Trace failures end to end
Followed errors across tool registration, dispatch, page state, and agent execution instead of treating every failed task as a model failure.
EVALUATION PIPELINE
One traceable path from site action to study result.
Verification sits inside the pipeline, so a missing or misrouted tool call is visible before it is mistaken for agent behavior.
- 01
Tool suites
Site actions become browser-native tools.
- 02
Injection + verification
A proxy injects tools and checks availability.
- 03
Agent runner
BrowserGym executes WebArena study modes.
- 04
Failure analysis
Runs are classified before results are compared.
FAILURE ANALYSIS
The model was not always the problem.
Dispatch defect
111 failures across 36 tasks
A dispatch bug caused repeated failures across the study. Tracing the call path made that infrastructure fault visible instead of letting it distort the model comparison.
Element alignment
38% → 18% error rate
After the failure path was isolated and the infrastructure was corrected, the observed element-ID error rate fell by 20 percentage points.
Metric scope: the before-and-after rates come from internal WebArena study runs. Confidential study data, model-level results, and implementation details are intentionally omitted.
TAKEAWAY
Evaluation quality is a systems problem.
Giving an agent better tools is only useful when tool registration, dispatch, page state, and scoring stay aligned. The most important work was building enough observability to tell those layers apart.