chem-agilent.com
robots.txt

Robots Exclusion Standard data for chem-agilent.com

Resource Scan

Scan Details

Site Domain chem-agilent.com
Base Domain chem-agilent.com
Scan Status Ok
Last Scan2024-10-30T16:06:40+00:00
Next Scan 2024-11-29T16:06:40+00:00

Last Scan

Scanned2024-10-30T16:06:40+00:00
URL https://chem-agilent.com/robots.txt
Redirect https://www.chem-agilent.com/robots.txt
Redirect Domain www.chem-agilent.com
Redirect Base chem-agilent.com
Domain IPs 27.133.240.17
Redirect IPs 27.133.240.17
Response IP 27.133.240.17
Found Yes
Hash 1d282c4c0c21c82092dd49546f24e4d320cfb6782e3cb1792efbd0ec71400a0b
SimHash 0450c9b4e212

Groups

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

baiduspider

Rule Path
Disallow /

baiduspider-image

Rule Path
Disallow /

baiduspider-video

Rule Path
Disallow /

baiduspider-news

Rule Path
Disallow /

baiduspider-favo

Rule Path
Disallow /

baiduspider-cpro

Rule Path
Disallow /

baiduspider-ads

Rule Path
Disallow /

*

Rule Path
Disallow /webinar/