targetadmission.com
robots.txt

Robots Exclusion Standard data for targetadmission.com

Resource Scan

Scan Details

Site Domain targetadmission.com
Base Domain targetadmission.com
Scan Status Ok
Last Scan2024-11-14T16:48:18+00:00
Next Scan 2024-11-21T16:48:18+00:00

Last Scan

Scanned2024-11-14T16:48:18+00:00
URL https://targetadmission.com/robots.txt
Redirect https://www.targetadmission.com/robots.txt
Redirect Domain www.targetadmission.com
Redirect Base targetadmission.com
Domain IPs 104.21.57.33, 172.67.188.236, 2606:4700:3032::6815:3921, 2606:4700:3034::ac43:bcec
Redirect IPs 104.21.57.33, 172.67.188.236, 2606:4700:3032::6815:3921, 2606:4700:3034::ac43:bcec
Response IP 104.21.57.33
Found Yes
Hash 786c2f6b8f1a45461ef053af03b1087afe01e030d8e201b6df46c8ea825f6863
SimHash ee0ae8e6ca97

Groups

*

Rule Path
Disallow /colleges/bsc-nautical-science*

bingbot

Rule Path
Disallow /*-courses?id=
Disallow /colleges/bsc-nautical-science*

Other Records

Field Value
crawl-delay 15

yandex

Rule Path
Disallow /

seekport crawler

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /*-courses?id=
Disallow /colleges/bsc-nautical-science*

Other Records

Field Value
crawl-delay 20

Comments

  • Disallow: /*-courses?id=