Skip to main content

Scanner errors

When Marker.io can't scan your website, it shows the reason in the scan error modal. Here's what each error means and how to fix it.

Which error are you seeing?

  • Scanner blocked by anti-bot protection

  • Scanner couldn't authenticate on your website

  • Scanner rate-limited by your website

Scanner blocked by anti-bot protection

Marker.io's scanner couldn't reach your website. This is common on sites protected by Cloudflare, Akamai, AWS WAF, or similar services.

Our scanner identifies itself with this user agent:

MarkerIO-Scanner/1.0 (+https://app.marker.io/scanner-bot)

To let it through, allow this user agent in your bot-protection settings.

Cloudflare

Allowlist the scanner with a Skip rule:

  1. Open the Cloudflare dashboard and select your domain.

  2. Go to Security → Security rules (older dashboards: Security → WAF → Custom rules) and click Create rule → Custom rules.

  3. Under When incoming requests match, use the expression (http.user_agent contains "MarkerIO-Scanner").

  4. Under Then take action, choose Skip and tick All remaining custom rules, All managed rules, All rate limiting rules, and All Super Bot Fight Mode rules (if you use it).

  5. Deploy the rule and retry the scan.

Bot Fight Mode on the Free plan can't be skipped from a custom rule. If your site uses it, turn it off under Security → Bots while the scan runs, or upgrade to a plan with Super Bot Fight Mode.

Scanner couldn't authenticate on your website

Your website doesn't give the scanner access. Open Project → Settings → Monitor and use the Site access settings, which offer four ways to grant it. Pick whichever matches how your site is protected, or combine several.

Any credentials you enter here are encrypted, stored securely, and used only by the scanner to reach your site.

Username and password (Auth prompt)

For sites that show a username and password popup before the page loads. This is often called HTTP Basic Auth and is common on staging environments. In the Auth prompt row, enter the username and password the scanner should use.

Session cookies

For sites where access is tied to a session cookie. Ask a developer for the relevant cookies, then paste them in the Cookies row. The scanner sends them with every request. This works best with long-lived sessions. Short-lived cookies need refreshing periodically.

Custom request headers

For sites that gate access with a token, like an API key or a Bearer token. Add the header your site expects in the Custom headers row and the scanner includes it on every request.

Custom user agent

For sites that only accept a specific kind of client, or that filter on user agent. Use the User Agent row to set the identifier the scanner sends, for example a standard Chrome user agent string. The scanner uses it instead of its default MarkerIO-Scanner identifier.

Scanner rate-limited by your website

Your website is throttling Marker.io's scanner. This usually means your site, or the service in front of it like Cloudflare or AWS WAF, treats the scanner like a regular visitor and starts rejecting requests once it sees too many in a short period.

How to fix it

Allowlist the scanner in your rate-limit rules so its traffic isn't counted against the same quota as your visitors. Ask whoever manages your site's protection to exclude requests with this user agent from rate limiting:

MarkerIO-Scanner/1.0 (+https://app.marker.io/scanner-bot)

This works in Cloudflare, AWS WAF, and most CDNs and WAFs.

If allowlisting isn't an option

Ask the scanner to slow down by adding these two lines to your site's robots.txt:

  • User-agent: MarkerIO-Scanner

  • Crawl-delay: 5

The scanner reads this before each scan and waits that many seconds between requests. Use this only as a fallback. On larger sites it noticeably increases scan time, and allowlisting gives a much better experience.

Need Help?

If you have any questions, comments, or corrections, chat with us at the bottom right of our web pages.

Did this answer your question?