Cloudflare challenges automated visitors

Cloudflare can answer a request it suspects is automated with a challenge page instead of the page itself: a check a browser can pass and most software can't. Every challenge page carries the response header cf-mitigated: challenge, which is how we know this was one.

We identify ourselves honestly, so we were challenged as any automated visitor would be. That includes AI agents fetching the page because a person asked them to, and smaller search and AI crawlers. The large ones may still get through: Cloudflare lets in bots it can verify, by a signature, a published list of addresses or reverse DNS, and flags a request that only claims to be one of them as a fake.

So this says the page is closed to software Cloudflare doesn't know, not that Google or ChatGPT search can't read it. Whether that is what the site wants is a choice, which is why this is a notice.

Sources: Detecting a challenge page response, Verified bots, Fake bot detection.

In Cloudflare, look at the site's bot settings under Security: Bot Fight Mode, a challenge in a custom rule, or the controls that block AI crawlers. Decide which automated visitors the page is for. A page meant to be found and read by AI assistants shouldn't challenge every visitor that isn't a browser.