Crawler directory
Other bots
User agent only
Meta
facebookexternalhit
About this crawler
Fetches pages shared on Facebook, Instagram or Messenger to build link previews.
Usually means: Someone shared your link. The crawler reads its title, description and thumbnail. It can bypass robots.txt for security checks.
User agent
facebookexternalhitWe match these as user-agent tokens, not full browser strings. Providers may add surrounding version or product text.
IP verification
This crawler is matched on its user agent only, because no machine-readable list of its IP addresses is available to us. Because user agents can be spoofed, it is shown as “claimed”, not verified.
Discover when bots and AI assistants crawl your website.
Add a small server-side snippet to see which AI assistants, search engines and training crawlers request your pages, including pages that return 404.
- ChatGPT, Claude, Perplexity and Gemini traffic in one view
- Verified against each provider’s published IP ranges
- No visitor cookies and no crawler IP addresses stored