instagram.org
robots.txt
Robots Exclusion Standard data for instagram.org
Resource Scan
Scan Details
Site Domain | instagram.org |
Base Domain | instagram.org |
Scan Status | Ok |
Last Scan | 2024-05-18T02:01:27+00:00 |
Next Scan | 2024-06-17T02:01:27+00:00 |
Last Scan
Scanned | 2024-05-18T02:01:27+00:00 |
URL | http://instagram.org/robots.txt |
Redirect | https://www.instagram.com/robots.txt |
Redirect Domain | www.instagram.com |
Redirect Base | instagram.com |
Domain IPs | 157.240.13.14, 2a03:2880:f00c:20d:face:b00c:0:2 |
Redirect IPs | 2a03:2880:f20c:e5:face:b00c:0:4420, 31.13.95.174 |
Response IP | 157.240.235.174 |
Found | Yes |
Hash | 258d3fcd8a1d2f7ec562e3facee67d47f17a5bd9c7fb8d41615d3d257b5f881d |
SimHash | 785151516785 |
Groups
applebot
Rule | Path |
---|---|
Disallow | /ajax/ |
Disallow | /client_error/ |
Disallow | /logging/ |
Disallow | /publicapi/ |
Disallow | /qp/batch_fetch_web/ |
Disallow | /query/ |
baiduspider
Rule | Path |
---|---|
Disallow | /ajax/ |
Disallow | /client_error/ |
Disallow | /logging/ |
Disallow | /publicapi/ |
Disallow | /qp/batch_fetch_web/ |
Disallow | /query/ |
bingbot
Rule | Path |
---|---|
Disallow | /*/c/ |
Disallow | /*/comments/ |
Disallow | /*/liked_by/ |
Disallow | /ajax/ |
Disallow | /client_error/ |
Disallow | /logging/ |
Disallow | /publicapi/ |
Disallow | /qp/batch_fetch_web/ |
Disallow | /query/ |
duckduckbot
Rule | Path |
---|---|
Disallow | /ajax/ |
Disallow | /client_error/ |
Disallow | /logging/ |
Disallow | /publicapi/ |
Disallow | /qp/batch_fetch_web/ |
Disallow | /query/ |
googlebot
Rule | Path |
---|---|
Disallow | /*/c/ |
Disallow | /*/comments/ |
Disallow | /*/liked_by/ |
Disallow | /ajax/ |
Disallow | /client_error/ |
Disallow | /logging/ |
Disallow | /publicapi/ |
Disallow | /qp/batch_fetch_web/ |
Disallow | /query/ |
google-extended
Rule | Path |
---|---|
Disallow | / |
Disallow | /ajax/ |
Disallow | /client_error/ |
Disallow | /logging/ |
Disallow | /publicapi/ |
Disallow | /qp/batch_fetch_web/ |
Disallow | /query/ |
gptbot
Rule | Path |
---|---|
Disallow | / |
Disallow | /ajax/ |
Disallow | /client_error/ |
Disallow | /logging/ |
Disallow | /publicapi/ |
Disallow | /qp/batch_fetch_web/ |
Disallow | /query/ |
ia_archiver
Rule | Path |
---|---|
Disallow | /ajax/ |
Disallow | /client_error/ |
Disallow | /logging/ |
Disallow | /publicapi/ |
Disallow | /qp/batch_fetch_web/ |
Disallow | /query/ |
msnbot
Rule | Path |
---|---|
Disallow | /ajax/ |
Disallow | /client_error/ |
Disallow | /logging/ |
Disallow | /publicapi/ |
Disallow | /qp/batch_fetch_web/ |
Disallow | /query/ |
naverbot
Rule | Path |
---|---|
Disallow | /*/c/ |
Disallow | /*/comments/ |
Disallow | /*/liked_by/ |
Disallow | /ajax/ |
Disallow | /client_error/ |
Disallow | /logging/ |
Disallow | /publicapi/ |
Disallow | /qp/batch_fetch_web/ |
Disallow | /query/ |
seznambot
Rule | Path |
---|---|
Disallow | /ajax/ |
Disallow | /client_error/ |
Disallow | /logging/ |
Disallow | /publicapi/ |
Disallow | /qp/batch_fetch_web/ |
Disallow | /query/ |
slurp
Rule | Path |
---|---|
Disallow | /ajax/ |
Disallow | /client_error/ |
Disallow | /logging/ |
Disallow | /publicapi/ |
Disallow | /qp/batch_fetch_web/ |
Disallow | /query/ |
teoma
Rule | Path |
---|---|
Disallow | /ajax/ |
Disallow | /client_error/ |
Disallow | /logging/ |
Disallow | /publicapi/ |
Disallow | /qp/batch_fetch_web/ |
Disallow | /query/ |
twitterbot
Rule | Path |
---|---|
Disallow | /ajax/ |
Disallow | /client_error/ |
Disallow | /logging/ |
Disallow | /publicapi/ |
Disallow | /qp/batch_fetch_web/ |
Disallow | /query/ |
yandex
Rule | Path |
---|---|
Disallow | /ajax/ |
Disallow | /client_error/ |
Disallow | /logging/ |
Disallow | /publicapi/ |
Disallow | /qp/batch_fetch_web/ |
Disallow | /query/ |
yeti
Rule | Path |
---|---|
Disallow | /ajax/ |
Disallow | /client_error/ |
Disallow | /logging/ |
Disallow | /publicapi/ |
Disallow | /qp/batch_fetch_web/ |
Disallow | /query/ |
*
Rule | Path |
---|---|
Disallow | / |
Comments