pacwestbc.prestosports.com
robots.txt

Robots Exclusion Standard data for pacwestbc.prestosports.com

Resource Scan

Scan Details

Site Domain pacwestbc.prestosports.com
Base Domain prestosports.com
Scan Status Ok
Last Scan2024-10-09T11:17:21+00:00
Next Scan 2024-11-08T11:17:21+00:00

Last Scan

Scanned2024-10-09T11:17:21+00:00
URL https://pacwestbc.prestosports.com/robots.txt
Domain IPs 18.246.126.30, 35.165.38.145, 52.32.188.247, 52.42.114.30
Response IP 52.42.114.30
Found Yes
Hash 5384c90a8f1903a1e18b6ec686452644462cf557df08cadced82f4127e1e6a72
SimHash 6875d002ceb9

Groups

american-univ-crawler (enterprise; s5-dwrrj5kwb2naa; nguyen@american.edu)

Rule Path
Disallow /

cstv search crawler

Rule Path
Disallow /

*

Rule Path
Disallow /cgi-bin/
Disallow /_private/
Disallow /_vti_bin/
Disallow /_vti_cnf/
Disallow /_vti_log/
Disallow /_vti_pvt/
Disallow /_vti_txt/
Disallow /reports/
Disallow /admin/
Disallow /action/

Other Records

Field Value
crawl-delay 5

msnbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

msnbot-media

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

twitterbot

Rule Path
Disallow

ahrefsbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

semrushbot-sa

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

rogerbot

Rule Path
Disallow /

piplbot

Rule Path
Disallow /

yandex

Rule Path
Disallow /

baidu

Rule Path
Disallow /

Comments

  • Managed by PrestoSports sysadmin@prestosports.com