Scanner etiquette
Doorman walks a website the way a buyer's AI agent does, and it says who it is while it does so. This page is the whole of its behaviour: what it sends, what it reads, how fast, what it stores, and how to make it stop.
Who is knocking
Every request carries this User-Agent, with this page as its contact address:
DoormanScanner/1.0 (+https://wicketry.com/doorman/etiquette; an agent-readiness check; reads public pages only)
What it reads
Public pages only, by GET. First /robots.txt; then the home page (following redirects, and recording each hop); the plain-http address once, to see whether it redirects; the sitemap; /llms.txt and the pages it points to; up to three product pages and the pricing page the site itself points to; and these addresses an agent guesses:
/.well-known/agent.json— the agent card (who you are, for agents; JSON)/.well-known/api-catalog— the API catalogue (RFC 9727, a linkset of service-desc/service-doc)/agents.md— agents.md (how an agent should use the site)/.well-known/security.txt— security.txt (RFC 9116)/security.txt— security.txt at the root (the older place)/pricing— a pricing page at the obvious path/legal— legal at the obvious path/terms— terms at the obvious path/privacy— privacy at the obvious path/support— support at the obvious path/contact— contact at the obvious path/status— a status page/changelog— a changelog/docs— documentation/for-agents— a page written for agents/api/openapi.json— an OpenAPI description/.well-known/openapi.json— OpenAPI at the well-known path/data-for-agents— downloadable data for agents
How it behaves
- At most 40 requests per scan, at least 0.5 seconds apart.
- One scan per site per day. A second request the same day returns the morning's result.
- No JavaScript is executed, no cookies are kept, no form is submitted, nothing is posted. Most agents do not run scripts either, and the report says so.
- Responses are read up to 400 kB; nothing larger is fetched further.
robots.txt, and how to opt out
The scanner reads /robots.txt before anything else and honours the * group and any group naming DoormanScanner. A path either group disallows is not fetched, and the scan's own log lists the paths it skipped. To keep the scanner off a site entirely:
User-agent: DoormanScanner Disallow: /
A site with that line — or with Disallow: / for everyone — is never graded: it is listed as unscanned, with no grade and no badge, and the scanner leaves after reading robots.txt. The same happens to a site that blocks the scanner or does not answer. Alternatively, write to [email protected]: the row and the badge are removed at the next build, within a day, and the site is not scanned again.
What it stores
The scan result: the addresses requested with their status codes and timing, the response headers the checks name, the structured data it could parse, and the short quotations the report shows. No personal data is sought or kept; a submission's reply address is kept only to answer it.
What a grade is, and is not
A grade is weighted points earned ÷ points available on the day of the scan. It describes how legible a site was to an agent that day and nothing more: it is not a certificate, not a compliance statement and not a ranking, and every finding shows the request it rests on. Findings say what an agent got and what would change it; whether a site should change anything is the site's own decision.
Questions, corrections or an opt-out: [email protected]. A person reads it.