Expose site actions
Built WebMCP tool suites that surfaced site actions as navigator.modelContext tools callable by LLM agents.
HUAWEI · WEB TECHNOLOGY LAB · 2026
WebMCP & Browser Agent Evaluation
I built the tooling and evaluation infrastructure used to study how LLM agents act across real web environments—and to separate model behavior from failures in the surrounding system.
- 9+ sites
- WebMCP tool suites
- 5 modes
- WebArena study design
- 10%
- avg. success rate increase
- 40%
- token reduction vs. baseline
THE QUESTION
Can browser-native tools make agent evaluation more reliable?
Browser agents can fail because of the model, the page, the tool catalog, or the evaluation harness. The study needed infrastructure that could expose site actions consistently, compare five execution modes, and identify where a failure actually entered the stack.
MY ROLE
Build the system around the experiment.
Make runs reproducible
Engineered a Python evaluation harness with a site-injection proxy and automated tool verification across Docker-hosted sites.
Support comparison
Built and debugged browser-agent infrastructure for a WebArena A/B study spanning five modes and multiple models.
Trace failures end to end
Followed errors across tool registration, dispatch, page state, and agent execution instead of treating every failed task as a model failure.
EVALUATION PIPELINE
One traceable path from site action to study result.
Verification sits inside the pipeline, so a missing or misrouted tool call is visible before it is mistaken for agent behavior.
- 01
Tool suites
Site actions become browser-native tools.
- 02
Injection + verification
A proxy injects tools and checks availability.
- 03
Agent runner
BrowserGym executes WebArena study modes.
- 04
Failure analysis
Runs are classified before results are compared.
OUTCOME
Cleaner evaluation, better agent efficiency.
Task success
10% average success rate increase
Across benchmark runs, the improved tool and evaluation path raised average task success by 10% compared with baseline.
Token efficiency
40% token reduction
The same workflow used 40% fewer tokens than baseline, reducing unnecessary reasoning and recovery work during evaluated tasks.
Metric scope: the percentages compare the WebMCP-enabled evaluation setup with the study baseline. Confidential study data, model-level results, and implementation details are intentionally omitted.
TAKEAWAY
Evaluation quality is a systems problem.
Giving an agent better tools is only useful when tool registration, dispatch, page state, and scoring stay aligned. The most important work was building enough observability to tell those layers apart.