meta.ai
robots.txt
Robots Exclusion Standard data for meta.ai
Resource Scan
Scan Details
| Site Domain | meta.ai |
| Base Domain | meta.ai |
| Scan Status | Ok |
| Last Scan | 2024-05-28T22:40:22+00:00 |
| Next Scan | 2024-06-27T22:40:22+00:00 |
Last Scan
| Scanned | 2024-05-28T22:40:22+00:00 |
| URL | https://meta.ai/robots.txt |
| Redirect | https://www.meta.ai/robots.txt |
| Redirect Domain | www.meta.ai |
| Redirect Base | meta.ai |
| Domain IPs | 157.240.7.20, 2a03:2880:f00c:100:face:b00c:0:2 |
| Redirect IPs | 157.240.15.1, 2a03:2880:f00c:100:face:b00c:0:2 |
| Response IP | 157.240.235.15 |
| Found | Yes |
| Hash | b0efa46affc236bdabe49a5c2293ae6649c9ee5d327c672ffce996f87325e701 |
| SimHash | 2817d800c583 |
Groups
applebot
| Rule | Path |
|---|---|
| Disallow | /ajax/ |
| Disallow | /client_error/ |
| Disallow | /logging/ |
| Disallow | /publicapi/ |
| Disallow | /qp/batch_fetch_web/ |
| Disallow | /query/ |
baiduspider
| Rule | Path |
|---|---|
| Disallow | /ajax/ |
| Disallow | /client_error/ |
| Disallow | /logging/ |
| Disallow | /publicapi/ |
| Disallow | /qp/batch_fetch_web/ |
| Disallow | /query/ |
bingbot
| Rule | Path |
|---|---|
| Disallow | /ajax/ |
| Disallow | /client_error/ |
| Disallow | /logging/ |
| Disallow | /publicapi/ |
| Disallow | /qp/batch_fetch_web/ |
| Disallow | /query/ |
duckduckbot
| Rule | Path |
|---|---|
| Disallow | /ajax/ |
| Disallow | /client_error/ |
| Disallow | /logging/ |
| Disallow | /publicapi/ |
| Disallow | /qp/batch_fetch_web/ |
| Disallow | /query/ |
facebookexternalhit
| Rule | Path |
|---|---|
| Disallow | /ajax/ |
| Disallow | /client_error/ |
| Disallow | /logging/ |
| Disallow | /publicapi/ |
| Disallow | /qp/batch_fetch_web/ |
| Disallow | /query/ |
googlebot
| Rule | Path |
|---|---|
| Disallow | /ajax/ |
| Disallow | /client_error/ |
| Disallow | /logging/ |
| Disallow | /publicapi/ |
| Disallow | /qp/batch_fetch_web/ |
| Disallow | /query/ |
google-extended
| Rule | Path |
|---|---|
| Disallow | / |
| Disallow | /ajax/ |
| Disallow | /client_error/ |
| Disallow | /logging/ |
| Disallow | /publicapi/ |
| Disallow | /qp/batch_fetch_web/ |
| Disallow | /query/ |
gptbot
| Rule | Path |
|---|---|
| Disallow | / |
| Disallow | /ajax/ |
| Disallow | /client_error/ |
| Disallow | /logging/ |
| Disallow | /publicapi/ |
| Disallow | /qp/batch_fetch_web/ |
| Disallow | /query/ |
ia_archiver
| Rule | Path |
|---|---|
| Disallow | /ajax/ |
| Disallow | /client_error/ |
| Disallow | /logging/ |
| Disallow | /publicapi/ |
| Disallow | /qp/batch_fetch_web/ |
| Disallow | /query/ |
msnbot
| Rule | Path |
|---|---|
| Disallow | /ajax/ |
| Disallow | /client_error/ |
| Disallow | /logging/ |
| Disallow | /publicapi/ |
| Disallow | /qp/batch_fetch_web/ |
| Disallow | /query/ |
naverbot
| Rule | Path |
|---|---|
| Disallow | /ajax/ |
| Disallow | /client_error/ |
| Disallow | /logging/ |
| Disallow | /publicapi/ |
| Disallow | /qp/batch_fetch_web/ |
| Disallow | /query/ |
seznambot
| Rule | Path |
|---|---|
| Disallow | /ajax/ |
| Disallow | /client_error/ |
| Disallow | /logging/ |
| Disallow | /publicapi/ |
| Disallow | /qp/batch_fetch_web/ |
| Disallow | /query/ |
slurp
| Rule | Path |
|---|---|
| Disallow | /ajax/ |
| Disallow | /client_error/ |
| Disallow | /logging/ |
| Disallow | /publicapi/ |
| Disallow | /qp/batch_fetch_web/ |
| Disallow | /query/ |
teoma
| Rule | Path |
|---|---|
| Disallow | /ajax/ |
| Disallow | /client_error/ |
| Disallow | /logging/ |
| Disallow | /publicapi/ |
| Disallow | /qp/batch_fetch_web/ |
| Disallow | /query/ |
twitterbot
| Rule | Path |
|---|---|
| Disallow | /ajax/ |
| Disallow | /client_error/ |
| Disallow | /logging/ |
| Disallow | /publicapi/ |
| Disallow | /qp/batch_fetch_web/ |
| Disallow | /query/ |
yandex
| Rule | Path |
|---|---|
| Disallow | /ajax/ |
| Disallow | /client_error/ |
| Disallow | /logging/ |
| Disallow | /publicapi/ |
| Disallow | /qp/batch_fetch_web/ |
| Disallow | /query/ |
yeti
| Rule | Path |
|---|---|
| Disallow | /ajax/ |
| Disallow | /client_error/ |
| Disallow | /logging/ |
| Disallow | /publicapi/ |
| Disallow | /qp/batch_fetch_web/ |
| Disallow | /query/ |
*
| Rule | Path |
|---|---|
| Disallow | / |