medicalxpress.com
robots.txt

Robots Exclusion Standard data for medicalxpress.com

Resource Scan

Scan Details

Site Domain medicalxpress.com
Base Domain medicalxpress.com
Scan Status Ok
Last Scan2024-05-23T08:17:03+00:00
Next Scan 2024-05-30T08:17:03+00:00

Last Scan

Scanned2024-05-23T08:17:03+00:00
URL https://medicalxpress.com/robots.txt
Domain IPs 2001:48c8:13:5::53, 72.251.233.233
Response IP 72.251.233.233
Found Yes
Hash ff2db196e1e4a3b5892d7631f2f44f8463df322aac225ac84664d944db370bd0
SimHash 6205d8827715

Groups

googlebot

Rule Path
Disallow

googlebot-image

Rule Path
Disallow

mediapartners-google/2.1

Rule Path
Disallow

mediapartners-google*

Rule Path
Disallow

msnbot

Rule Path
Disallow

slurp

Rule Path
Disallow

Other Records

Field Value
crawl-delay 1

yahoo-mmcrawler

Rule Path
Disallow

yahoo-blogs/v3.9

Rule Path
Disallow

gigabot

Rule Path
Disallow

scrubby

Rule Path
Disallow

robozilla

Rule Path
Disallow

nutch

Rule Path
Disallow

ia_archiver

Rule Path
Disallow

psbot

Rule Path
Disallow

asterias

Rule Path
Disallow

*

Rule Path
Disallow /openx/