![\"Bot](\"https://research.poh.org/results.png\")
Milena Rmus, Mayank Agrawal, and Mathew Hardy | Proof of Human Research | October 8, 2026
\nWe revisit our bot detection benchmark with stealth browsers and modern AI agents, and find detection rates\nranging from 9% to 99%.
\nA year ago, we evaluated five leading bot detection\nsystems (Proof of Human, Google reCAPTCHA v3, hCaptcha, Fingerprint Pro, and Cloudflare Turnstile) against\nautomated browsers programmed to complete a set of predefined tasks. Detection rates varied substantially,\nranging from 33% to 87%. Systems that incorporated behavioral signals alongside device-level signals generally\noutperformed those relying on device information alone.
\nSince then, browser automation has advanced considerably. Modern AI agents can interpret webpages, plan actions,\nand interact with interfaces through increasingly capable browser environments. We therefore revisited our\nbenchmark with an expanded set of adversaries, including conventional browser automation, stealth browsers, and\nAI-powered browsing agents, and tested a similar set of detection systems: Proof of Human, Google reCAPTCHA v3,\nhCaptcha, Cloudflare Turnstile, and Cloudflare Bot Management. The results reveal similarly wide variation in\ndetection performance.
\nIn this post, we examine how each system performed, where it failed, and what these results suggest about the\nchallenges of distinguishing humans from increasingly capable AI agents.
\nOur task used a five-question\nsurvey combining multiple-choice items with open-ended responses. Each of\nthe five adversaries (i.e., AI agents) attempted the survey 20 times, yielding up to 100 sessions per detection system. Some\ndetection systems failed or crashed before session completion, leaving between 89 and 100 evaluable sessions for\neach system.
\nThe table below summarizes the detection systems included in the benchmark and the signals each uses to identify\nautomated activity.
\n| Detection system | Description |
|---|---|
| Proof of Human | Uses a machine-learning model to classify sessions as human or automated based on both behavioral and device signals, including typing, mouse movements, clicks, and scrolling. |
| Google reCAPTCHA v3 | Assigns each session a score from 0 to 1 indicating the likelihood that the user is human. It operates without a visible challenge and considers browser-integrity and Google-side reputation signals; the site determines the threshold for accepting a session. |
| hCaptcha | Presents an “I am human” checkbox and evaluates the browser's risk profile. Depending on the result, it either verifies the session immediately or presents an image-based challenge that must be completed. |
| Cloudflare Turnstile | Performs browser-integrity checks in the background and issues a verification token when a session passes. It generally operates without a puzzle but may request user interaction when signals are ambiguous. |
| Cloudflare Bot Management | Evaluates requests using heuristics and machine-learning models trained on traffic across Cloudflare's network. Although the full system assigns scores from 1 (automated) to 99 (human), the Pro-tier implementation returns broader classifications such as likely_human, likely_automated, automated, or verified_bot. |
Our set of adversaries covers increasingly sophisticated forms of automation: conventional scripted browsers,\nstealth browsers designed to conceal automation signals, and AI agents capable of navigating webpages and\ncompleting tasks autonomously.
\n| Adversary | Description |
|---|---|
| Chromium | A standard Chromium browser automated through Playwright and the Chrome DevTools Protocol, without stealth modifications. It retains common indicators of automation, such as navigator.webdriver and CDP-related artifacts, and serves as the baseline “obvious bot.” |
| CloakBrowser | A stealth-focused Chromium fork that modifies browser fingerprints, including GPU, canvas, WebGL, audio, fonts, user-agent, and TLS signals, at the source-code level. We ran it through a residential proxy with its optional humanization features for mouse and keyboard input enabled. |
| Browser Use | An open-source AI-agent framework that uses an LLM (Claude Opus in our setup) and visual page information to plan actions and operate an unmodified browser from cloud infrastructure. |
| Perplexity Comet | Uses Perplexity's AI-powered browser, which uses an LLM and visual information to navigate webpages and complete tasks. |
| Grok | xAI's browser-use agent, which uses an LLM and visual information to navigate and complete web tasks. |
We found that detection performance varied substantially across systems. Aggregated across all adversaries, Proof of Human\nidentified 99% of automated sessions, followed by reCAPTCHA v3 at 69.4%, hCaptcha at 44.3%, Cloudflare Turnstile\nat 42.1%, and Cloudflare Bot Management at 9.1%.
\nFigure 1: Bot benchmarking results. Each bar shows the\npercentage of automated sessions identified by the given detection system, aggregated across all five\nadversaries.
\nPerformance differed considerably by adversary. The clearest split was between scripted browsers and AI\nagents. hCaptcha and Cloudflare Turnstile caught nearly every scripted Chromium and CloakBrowser session but few of the\nAI-agent sessions. reCAPTCHA v3 showed the opposite pattern, identifying most agent sessions but none\nfrom CloakBrowser. Cloudflare Bot Management caught few sessions from any adversary, while Proof of Human\ndetected at least 95% of sessions overall.
\n![\"Detection](\"https://research.poh.org/heatmap.png\")
Figure 2: Percentage of sessions identified as automated for\neach combination of detection system and adversary. Green cells indicate adversaries that were caught; red\ncells indicate adversaries that fooled the detection system.
\nAs most detection systems are black-box, we cannot directly determine which signals produced each result.\nHowever, the systems' observed behavior, together with publicly available documentation, suggests several likely sources for this failure:
\nreCAPTCHA v3 detected most sessions from standard Chromium and the cloud-hosted agents but
did not detect CloakBrowser. CloakBrowser was the only adversary that combined a source-level modified browser\nfingerprint with a residential IP address. Our results cannot determine whether its browser fingerprint, residential IP,\nor a combination of the two was primarily responsible for its success.
hCaptcha gave challenges to both Chromium and CloakBrowser, neither of which could reliably complete
the image-based task. By contrast, the three vision-capable agents—Browser Use, Grok, and Perplexity\nComet—were generally able to interpret and solve the image grid, allowing most of their sessions to\npass.
Cloudflare Turnstile detected both scripted-browser adversaries: Chromium and CloakBrowser
failed to receive verification tokens, even after the full 30-second waiting period. However, it did not\ndetect any of the three AI agents, and did not even issue challenges during these sessions.
Cloudflare Bot Management detected relatively few sessions overall. This pattern is consistent with the stealth browser and cloud agents presenting clean
browser-level signals, although the higher detection rate for Grok may reflect infrastructure-level signals Cloudflare uses, such as IP or TLS reputation.
Several limitations should be considered when interpreting these results:
\nWe evaluated the product tiers available to us rather than enterprise versions, which may incorporate
additional signals or more advanced detection capabilities.
We measured detection rates for automated sessions but did not evaluate false-positive rates among human
respondents.
Some systems, including reCAPTCHA, may become better calibrated as they observe more traffic and establish a
reputation for a website. Detection rates could therefore differ in a longer-running deployment.
Our results suggest that bot detection is entering a new phase. Traditional automation signals remain effective\nagainst scripted browsers, but they are less reliable against AI agents that genuine browser\nenvironments intelligently. Interactive challenges may offer limited protection when vision-capable agents can interpret\nand complete them. And as browser-level indicators become easier to avoid, the way a user behaves throughout a\nsession becomes increasingly important for distinguishing human activity from automation.
\nFuture benchmarks should test larger volumes of traffic, include human participants to measure false-positive\nrates, and evaluate more complex tasks and emerging agents. At Proof of Human, we will continue tracking these\nchanges and developing methods in this evolving threat landscape. To test Proof of Human on\nyour own application, visit poh.org.
\nMilena Rmus, Mayank Agrawal, and Mathew Hardy work at Proof of Human, PBC, where they are building Proof of\nHuman, an invisible authentication system for the web. Previously, they completed PhDs in cognitive science at\nthe University of California, Berkeley (Milena) and Princeton University (Mayank and Matt).
","headings":[{"level":1,"text":"Benchmarking Bot Detection Systems Against Modern AI Agents in 2026","id":"benchmarking-bot-detection-systems-against-modern-ai-agents-in-2"},{"level":2,"text":"Methodology","id":"methodology"},{"level":2,"text":"Results","id":"results"},{"level":2,"text":"Limitations","id":"limitations"},{"level":2,"text":"Conclusion","id":"conclusion"},{"level":3,"text":"About the Authors","id":"about-the-authors"}]}}