Verifies the signed Ed25519 key directory behind Web Bot Auth
The auditor ports the checks of Cloudflare's http-signature-directory validator:
Locate — builds <scheme>://<host>/.well-known/http-message-signatures-directory from the audited URL (its path is ignored) and fetches it with Accept: application/http-message-signatures-directory+json, following redirects.
Blocked or missing — HTTP 403, 429 or a Cloudflare challenge ends the run as a blocked bot (590). Any other non-2xx status fails (400): a bot owner without a reachable directory cannot be verified.
Keys — parses the body as a JWKS. Each key must be an OKP key on curve Ed25519 with a 32-byte base64url x. The RFC 7638 thumbprint of each key is the keyid verifiers expect.
Signatures — reads Signature-Input and Signature. For every key there must be an entry with tag="http-message-signatures-directory", keyid equal to the thumbprint, created, expires and the ("@authority";req) component. The entry must not be expired, and the Ed25519 signature must verify over the RFC 9421 signature base built from the final response URL.
Hygiene — warns when the directory is served over http, with the wrong Content-Type, or with a Cache-Control lifetime longer than the signature's remaining validity (a cached copy would be served with an expired signature).
Aggregate — ERROR wins, then FAIL, then WARNING; if every rule passes, the audit is SUCCESS.
Specification pin — every run emits 37A-WebBotAuthAuditor-100 (INFORMATION) naming draft-meunier-http-message-signatures-directory-03 and RFC 9421. That check does not change overall status.
Every row maps to one checkRule* method and one CheckCode. 37A-WebBotAuthAuditor-100 is emitted on every run, including fetch errors. Key rules (403–409) run once per published key, so one audit can carry several checks with the same code. 401 skips every later FAIL rule; 402 skips the key rules; a key that fails 403 or 404 skips 406–409; 405 skips 406–409 for all keys; 406 skips 407–409 for that key; 407 skips 408 and 409. CheckCode suffixes use status ranges: 100 information, 200 success, 300–399 warning, 400–499 fail, 500–599 error.
| CheckCode | Status | When it fires | Recommendation | Why it matters |
|---|---|---|---|---|
37A-WebBotAuthAuditor-100 | INFORMATION | Every run. Records that this auditor implements draft-meunier-http-message-signatures-directory-03 and RFC 9421. | — (recommendation is null; nothing for the site to change) | — |
37A-WebBotAuthAuditor-300 | WARNING | The directory was fetched over http. | Serve {directoryUrl} over HTTPS and point Signature-Agent at the https origin. | Over plain HTTP the directory can be swapped in transit, and Cloudflare only accepts https Signature-Agent origins, so the bot cannot be verified. |
37A-WebBotAuthAuditor-400 | FAIL | The directory answered a status other than 2xx (after the blocked check). | Publish the JWKS at {directoryUrl} so it answers HTTP 200 with media type application/http-message-signatures-directory+json. | Without a reachable key directory, origins such as Cloudflare cannot verify your bot's signatures, so it is treated as an unknown crawler and may be blocked. |
37A-WebBotAuthAuditor-301 | WARNING | Content-Type is absent or its media type is not application/http-message-signatures-directory+json. | Set Content-Type: application/http-message-signatures-directory+json on {directoryUrl}. | Verifiers that check the media type skip the directory, so keys may be ignored even though the signature is right. |
37A-WebBotAuthAuditor-401 | FAIL | The body is not a JSON object with a keys array. | Return a JSON object with a "keys" array of JWKs at {directoryUrl}, for example {"keys":[{"kty":"OKP","crv":"Ed25519","x":"…"}]}. | A directory that cannot be parsed hands verifiers no keys, so every signed request from your bot fails verification. |
37A-WebBotAuthAuditor-402 | FAIL | The keys array is empty. | Add the Ed25519 public key your bot signs with to the "keys" array at {directoryUrl}. | With no keys published there is nothing to verify your bot's signatures against, so verification always fails. |
37A-WebBotAuthAuditor-403 | FAIL | A key is not kty=OKP on curve Ed25519. | Replace key {index} ({kid or thumbprint}) with an Ed25519 OKP key ({"kty":"OKP","crv":"Ed25519","x":"…"}). | Cloudflare Web Bot Auth only verifies Ed25519 signatures, so requests signed with this key are rejected. |
37A-WebBotAuthAuditor-404 | FAIL | An Ed25519 key's x is missing, not base64url, or does not decode to 32 bytes. | Export the public key of key {index} again and publish its raw 32 bytes base64url-encoded (no padding) in "x". | A malformed public key cannot be imported by verifiers, so your bot's signatures can never be checked. |
37A-WebBotAuthAuditor-405 | FAIL | The response has no Signature-Input or no Signature header. | Sign the directory response: add Signature-Input: sig1=("@authority";req);alg="ed25519";keyid="{thumbprint}";created=…;expires=…;tag="http-message-signatures-directory" and the matching Signature: sig1=:…: header. | Verifiers only trust keys whose directory response is signed by them, so an unsigned directory leaves your bot unverified. |
37A-WebBotAuthAuditor-406 | FAIL | No Signature-Input entry has tag="http-message-signatures-directory", keyid equal to the key's thumbprint, created, expires and an @authority component. | Add a Signature-Input entry for key {thumbprint} with tag "http-message-signatures-directory", keyid {thumbprint}, created, expires and the "@authority";req component, plus its Signature entry. | Verifiers ignore keys that do not have their own valid directory signature, so this key cannot vouch for your bot. |
37A-WebBotAuthAuditor-407 | FAIL | The matching entry covers @authority without the req parameter. | Change the covered component in {label} from "@authority" to "@authority";req and sign again. | Cloudflare's verifier resolves @authority from the request, so a directory signed without req fails verification and your bot stays unverified. |
37A-WebBotAuthAuditor-408 | FAIL | The entry's expires is in the past. | Sign the directory response on every request, or refresh the cached copy before {expires}, so that expires of {label} is in the future. | An expired directory signature is rejected like a missing one, so your bot loses verification until the directory is re-signed. |
37A-WebBotAuthAuditor-409 | FAIL | The entry's Signature is missing, cannot be decoded, or does not verify with the key under Ed25519. | Sign exactly the components listed in Signature-Input {label} (RFC 9421 signature base, Ed25519) with the private key of {thumbprint} and publish the result in the Signature header. | A signature that does not verify means the published key and the signing key differ, so verifiers treat your bot as an impostor. |
37A-WebBotAuthAuditor-302 | WARNING | Cache-Control max-age or s-maxage is longer than the matching signature's remaining validity (skipped when the signature is already expired). | Lower max-age below {remaining} s or extend expires of {label} beyond the cache lifetime of {maxAge} s. | A cache that outlives the signature serves an expired directory, and verifiers then reject your bot until the cache refreshes. |
37A-WebBotAuthAuditor-500 | ERROR | The URL is invalid or the directory could not be fetched (DNS, TLS, timeout). | Make sure {directoryUrl} resolves and answers over TLS, then run the audit again. Last error was {error}. | — |
37A-WebBotAuthAuditor-200 | SUCCESS | All rules passed. Message is The key directory at "{directoryUrl}" serves {n} Ed25519 key(s) ({thumbprints}) and its signature(s) verified; "{label}" expires at {expires}. | — | — |
37A-WebBotAuthAuditor-590 | ERROR | The key directory answered HTTP 403, HTTP 429 or a Cloudflare challenge, so 37AuditsBot was blocked before it could read the directory. | Allow 37AuditsBot in your firewall or bot protection: allowlist its user agent, or verify its Web Bot Auth signature. See https://www.37audits.com/bot. | While 37AuditsBot is blocked this audit cannot look at the site, so problems on it go unreported. |
| Status | Description | Test logic |
|---|---|---|
| SUCCESS | The directory serves Ed25519 keys and every key has a valid, unexpired directory signature | HTTP 2xx, JWKS of Ed25519 keys, one matching Signature-Input/Signature entry per key that verifies (200). |
| WARNING | The directory verifies but its transport, media type or cache lifetime is off | Fetched over http (300); wrong or missing Content-Type (301); Cache-Control outlives the signature (302). |
| FAIL | The directory is missing, malformed, unsigned, or its signature is wrong | Non-2xx status (400); body is not a JWKS (401); no keys (402); non-Ed25519 key (403); invalid x (404); no signature headers (405); no entry for a key (406); @authority without req (407); expired (408); verification failed (409). |
| ERROR | The auditor could not fetch the directory | Invalid URL, DNS, TLS or timeout (500); 37AuditsBot blocked (590). |
| INFORMATION | Specification pin | Always emitted (100); does not change overall status. |
Cloudflare and other origins verify your crawler by fetching this directory and checking the signature on it. A missing, unsigned or expired directory makes every signed request from your bot look unverified, and bot protection treats it like any unknown crawler.
Rotate keys by publishing the old and the new key together, each with its own directory signature, until the bot has switched.
The directory response is signed with a created/expires window. A CDN, framework cache or static export that serves the same response past expires hands verifiers an expired signature. Sign on every request, or keep the cache lifetime below the validity window.
The bot's own signed requests (tag="web-bot-auth", the Signature-Agent header) leave the bot, not the site, so they cannot be audited from the outside. Use Cloudflare's crawltest.com to test those.
The content-digest component the draft recommends is not required; Cloudflare's validator does not require it either.
Registration with Cloudflare Verified Bots is a separate submission and is not verified.
This documentation is built and hosted on Mintlify, a developer documentation platform.
Add this auditor to your monitoring setup and start identifying issues on your websites today.