Crawler directory
Search indexes
User agent only
Meta
meta-webindexer
About this crawler
Navigates the web to improve Meta AI search result quality.
Usually means: Allowing it helps Meta AI cite and link to your content in its answers.
User agent
meta-webindexerWe match these as user-agent tokens, not full browser strings. Providers may add surrounding version or product text.
IP verification
This crawler is matched on its user agent only, because no machine-readable list of its IP addresses is available to us. Because user agents can be spoofed, it is shown as “claimed”, not verified.
Discover when bots and AI assistants crawl your website.
Add a small server-side snippet to see which AI assistants, search engines and training crawlers request your pages, including pages that return 404.
- ChatGPT, Claude, Perplexity and Gemini traffic in one view
- Verified against each provider’s published IP ranges
- No visitor cookies and no crawler IP addresses stored