How do I identify a real Googlebot?

Not by its user agent, which any client can send. Check the address: it has to be in one of the IP lists Google publishes, or have a reverse DNS name on Google's crawler domain that resolves back to the same address. The same check works for every crawler whose operator publishes a list or a reverse DNS rule.

How do I let real crawlers in and keep fake ones out?

Decide by the result of the check, never by the user agent. Use each operator's current list rather than a copy made once, because operators add and retire ranges. Require the forward lookup after reverse DNS, and do not treat every name on an operator's cloud domain as its crawler: customers rent machines there too.

Why can some crawlers not be verified?

Their operators publish no IP list, no reverse DNS rule and no signing keys, so nothing outside the request can confirm them, and the name in the log is only a claim. Some operators publish only the network their crawlers run in, and then the check can say no more than that the address belongs to it.

Why does a real crawler sometimes show COULD NOT CHECK?

The operator's list could not be loaded, or DNS did not answer in time, so there was nothing to decide by. Run the check again in a minute. An answer is given only when the operator's own source confirms or rules out the address.

Crawler Verifier

Verify Googlebot, Bingbot, GPTBot, ClaudeBot and other crawlers: check an IP address or a whole log line against what each operator publishes.

Input

Paste an IP address from your access log, with its User-Agent if you have it, or paste the whole log line. Nothing is sent to that address: the check reads the lists the operators publish and asks DNS.

Terminal

Console ready. Execute a command to see output...

About Crawler Verifier

What crawler verification is

A crawler names itself in the User-Agent header, and any client can copy that name. What a client cannot copy is the address its connection comes from. Operators publish how to recognise their own addresses: a list of the IP ranges their crawlers use, a reverse DNS name on their own domain, or both. Verifying Googlebot, GPTBot, ClaudeBot or any other crawler means checking the address against what its operator publishes, and ignoring the name it gives.

This page checks one address, or one line from an access log, against those sources. The methods are compared side by side in AI Crawler Verification: Is That Really ChatGPT in Your Logs?. To check a request that an agent signed, use the Agent Verifier.

What is checked

The operator's IP list. Whether the address is inside the current list of the crawler it claims to be, which range matched, and when the operator last updated the list. A list that has not changed for a long time is marked as old, and reverse DNS decides instead.
Reverse DNS with a forward check. Where the operator documents this method, the address must have a name on the operator's domain, and that name must resolve back to the same address. A name that only looks right, or does not resolve back, does not count.
The network. Who announces the address, and whether it is a machine rented from a cloud provider rather than the operator's own crawler. For an operator that publishes no list, the network is as far as the check can go.
Signing keys. Whether the operator also publishes keys to sign its requests with Web Bot Auth, so that a signed request from it can be checked cryptographically.

Which crawlers are covered

Search engines. Google, Bing, Apple, DuckDuckGo, Yandex, Baidu.
AI companies. OpenAI, Anthropic, Perplexity.
SEO crawlers. Ahrefs.
Social networks (network only). Meta.

Any other bot name is still read. When its operator publishes nothing that could confirm it, the result says that the name is only a claim.

How to read the result

The first line is the verdict. A green line means the operator's own source confirms the address. A red line means the address is outside every source the operator publishes. An orange line means the operator offers no way to confirm its crawler, or only its network. COULD NOT CHECK means a list or DNS did not answer in time.

The lines under the verdict are the evidence: the matched range and the date of its list, the reverse DNS name and whether it resolves back, the network, and the operator's signing keys where they exist.

How to use Crawler Verifier

Paste the address or the log line
Copy the client IP address from your access log into the first field, IPv4 or IPv6. Or paste the whole log line, plain text or JSON, and the address and the User-Agent are read from it. The result shows what was read, so if your log records a proxy or a load balancer instead of the client, paste the client address by hand.
Add the User-Agent
Optional. The User-Agent is the name the client gave itself, and with it the check answers whether the client is that crawler: a false name shows as a red line. Without it, the address is looked up in every published list and in reverse DNS, and the result names the crawler it belongs to, or says it belongs to none.
Run the check
Press Check. The verdict appears in the terminal panel, with the evidence lines under it.
Act on the verdict
Treat the client as the crawler it names only when the first line is green. Any other verdict means the address does not prove the name. After COULD NOT CHECK, run the check again in a minute.

Frequently Asked Questions

: Not by its user agent, which any client can send. Check the address: it has to be in one of the IP lists Google publishes, or have a reverse DNS name on Google's crawler domain that resolves back to the same address. The same check works for every crawler whose operator publishes a list or a reverse DNS rule.
: Decide by the result of the check, never by the user agent. Use each operator's current list rather than a copy made once, because operators add and retire ranges. Require the forward lookup after reverse DNS, and do not treat every name on an operator's cloud domain as its crawler: customers rent machines there too.
: Their operators publish no IP list, no reverse DNS rule and no signing keys, so nothing outside the request can confirm them, and the name in the log is only a claim. Some operators publish only the network their crawlers run in, and then the check can say no more than that the address belongs to it.
: The operator's list could not be loaded, or DNS did not answer in time, so there was nothing to decide by. Run the check again in a minute. An answer is given only when the operator's own source confirms or rules out the address.