drexel.edu
robots.txt

Robots Exclusion Standard data for drexel.edu

Resource Scan

Scan Details

Site Domain drexel.edu
Base Domain drexel.edu
Scan Status Ok
Last Scan2024-10-21T15:29:29+00:00
Next Scan 2024-11-20T15:29:29+00:00

Last Scan

Scanned2024-10-21T15:29:29+00:00
URL https://drexel.edu/robots.txt
Domain IPs 144.118.67.10
Response IP 144.118.67.10
Found Yes
Hash 302c24ce44d21c8347f3e7e2d2c5d5ac497b3d12ce88968a1e5785d052980af1
SimHash 8f35d868ab52

Groups

amazonbot

Rule Path
Disallow /

*

Rule Path
Disallow /search
Disallow /*/search
Disallow /search/*
Disallow /*/search/*
Disallow /coas/search
Disallow /api/du/search
Disallow /Search
Disallow /*/Search
Disallow /Search/*
Disallow /*/Search/*
Disallow /coas/Search
Disallow /api/du/Search
Disallow /SEARCH
Disallow /*/SEARCH
Disallow /SEARCH/*
Disallow /*/SEARCH/*
Disallow /coas/SEARCH
Disallow /api/du/SEARCH