meta.ai
robots.txt
Robots Exclusion Standard data for meta.ai
Resource Scan
Scan Details
| Site Domain | meta.ai |
| Base Domain | meta.ai |
| Scan Status | Ok |
| Last Scan | 2024-11-08T22:42:02+00:00 |
| Next Scan | 2024-11-22T22:42:02+00:00 |
Last Scan
| Scanned | 2024-11-08T22:42:02+00:00 |
| URL | https://meta.ai/robots.txt |
| Redirect | https://www.meta.ai/robots.txt |
| Redirect Domain | www.meta.ai |
| Redirect Base | meta.ai |
| Domain IPs | 157.240.7.20, 2a03:2880:f34b:8d:face:b00c:0:2 |
| Redirect IPs | 157.240.7.20, 2a03:2880:f34b:8d:face:b00c:0:2 |
| Response IP | 157.240.235.15 |
| Found | Yes |
| Hash | 40832c58ee9c8a42c72691ceaf7e5c891f51fb5a8331ab4f4232f8bc9b15f12c |
| SimHash | ea1ed381c5a7 |
Groups
applebot
| Rule | Path |
|---|---|
| Disallow | /ajax/ |
| Disallow | /client_error/ |
| Disallow | /logging/ |
| Disallow | /publicapi/ |
| Disallow | /qp/batch_fetch_web/ |
| Disallow | /query/ |
bingbot
| Rule | Path |
|---|---|
| Disallow | /ajax/ |
| Disallow | /client_error/ |
| Disallow | /logging/ |
| Disallow | /publicapi/ |
| Disallow | /qp/batch_fetch_web/ |
| Disallow | /query/ |
discordbot
| Rule | Path |
|---|---|
| Disallow | /ajax/ |
| Disallow | /client_error/ |
| Disallow | /logging/ |
| Disallow | /publicapi/ |
| Disallow | /qp/batch_fetch_web/ |
| Disallow | /query/ |
duckduckbot
| Rule | Path |
|---|---|
| Disallow | /ajax/ |
| Disallow | /client_error/ |
| Disallow | /logging/ |
| Disallow | /publicapi/ |
| Disallow | /qp/batch_fetch_web/ |
| Disallow | /query/ |
facebookexternalhit
| Rule | Path |
|---|---|
| Disallow | /ajax/ |
| Disallow | /client_error/ |
| Disallow | /logging/ |
| Disallow | /publicapi/ |
| Disallow | /qp/batch_fetch_web/ |
| Disallow | /query/ |
googlebot
| Rule | Path |
|---|---|
| Disallow | /ajax/ |
| Disallow | /client_error/ |
| Disallow | /logging/ |
| Disallow | /publicapi/ |
| Disallow | /qp/batch_fetch_web/ |
| Disallow | /query/ |
googlebot-image
| Rule | Path |
|---|---|
| Disallow | /ajax/ |
| Disallow | /client_error/ |
| Disallow | /logging/ |
| Disallow | /publicapi/ |
| Disallow | /qp/batch_fetch_web/ |
| Disallow | /query/ |
googlebot-video
| Rule | Path |
|---|---|
| Disallow | /ajax/ |
| Disallow | /client_error/ |
| Disallow | /logging/ |
| Disallow | /publicapi/ |
| Disallow | /qp/batch_fetch_web/ |
| Disallow | /query/ |
googlebot-news
| Rule | Path |
|---|---|
| Disallow | /ajax/ |
| Disallow | /client_error/ |
| Disallow | /logging/ |
| Disallow | /publicapi/ |
| Disallow | /qp/batch_fetch_web/ |
| Disallow | /query/ |
google-inspectiontool
| Rule | Path |
|---|---|
| Disallow | /ajax/ |
| Disallow | /client_error/ |
| Disallow | /logging/ |
| Disallow | /publicapi/ |
| Disallow | /qp/batch_fetch_web/ |
| Disallow | /query/ |
linkedinbot
| Rule | Path |
|---|---|
| Disallow | /ajax/ |
| Disallow | /client_error/ |
| Disallow | /logging/ |
| Disallow | /publicapi/ |
| Disallow | /qp/batch_fetch_web/ |
| Disallow | /query/ |
msnbot
| Rule | Path |
|---|---|
| Disallow | /ajax/ |
| Disallow | /client_error/ |
| Disallow | /logging/ |
| Disallow | /publicapi/ |
| Disallow | /qp/batch_fetch_web/ |
| Disallow | /query/ |
naverbot
| Rule | Path |
|---|---|
| Disallow | /ajax/ |
| Disallow | /client_error/ |
| Disallow | /logging/ |
| Disallow | /publicapi/ |
| Disallow | /qp/batch_fetch_web/ |
| Disallow | /query/ |
pinterestbot
| Rule | Path |
|---|---|
| Disallow | /ajax/ |
| Disallow | /client_error/ |
| Disallow | /logging/ |
| Disallow | /publicapi/ |
| Disallow | /qp/batch_fetch_web/ |
| Disallow | /query/ |
screaming frog seo spider
| Rule | Path |
|---|---|
| Disallow | /ajax/ |
| Disallow | /client_error/ |
| Disallow | /logging/ |
| Disallow | /publicapi/ |
| Disallow | /qp/batch_fetch_web/ |
| Disallow | /query/ |
seznambot
| Rule | Path |
|---|---|
| Disallow | /ajax/ |
| Disallow | /client_error/ |
| Disallow | /logging/ |
| Disallow | /publicapi/ |
| Disallow | /qp/batch_fetch_web/ |
| Disallow | /query/ |
slurp
| Rule | Path |
|---|---|
| Disallow | /ajax/ |
| Disallow | /client_error/ |
| Disallow | /logging/ |
| Disallow | /publicapi/ |
| Disallow | /qp/batch_fetch_web/ |
| Disallow | /query/ |
teoma
| Rule | Path |
|---|---|
| Disallow | /ajax/ |
| Disallow | /client_error/ |
| Disallow | /logging/ |
| Disallow | /publicapi/ |
| Disallow | /qp/batch_fetch_web/ |
| Disallow | /query/ |
telegrambot
| Rule | Path |
|---|---|
| Disallow | /ajax/ |
| Disallow | /client_error/ |
| Disallow | /logging/ |
| Disallow | /publicapi/ |
| Disallow | /qp/batch_fetch_web/ |
| Disallow | /query/ |
twitterbot
| Rule | Path |
|---|---|
| Disallow | /ajax/ |
| Disallow | /client_error/ |
| Disallow | /logging/ |
| Disallow | /publicapi/ |
| Disallow | /qp/batch_fetch_web/ |
| Disallow | /query/ |
yeti
| Rule | Path |
|---|---|
| Disallow | /ajax/ |
| Disallow | /client_error/ |
| Disallow | /logging/ |
| Disallow | /publicapi/ |
| Disallow | /qp/batch_fetch_web/ |
| Disallow | /query/ |
*
| Rule | Path |
|---|---|
| Disallow | / |