matthewjockers.net
robots.txt

Robots Exclusion Standard data for matthewjockers.net

Resource Scan

Scan Details

Site Domain matthewjockers.net
Base Domain matthewjockers.net
Scan Status Ok
Last Scan2025-10-25T06:51:42+00:00
Next Scan 2025-11-24T06:51:42+00:00

Last Scan

Scanned2025-10-25T06:51:42+00:00
URL https://matthewjockers.net/robots.txt
Redirect https://www.matthewjockers.net/robots.txt
Redirect Domain www.matthewjockers.net
Redirect Base matthewjockers.net
Domain IPs 69.163.180.96
Redirect IPs 69.163.180.96
Response IP 69.163.180.96
Found Yes
Hash e27f7d630172518190d9c62466cc880beb3dee32bcbb3fd650c6c87dd05d1f74
SimHash ab58dc908066

Groups

jeeves

Rule Path
Disallow /

slurp

Rule Path
Disallow /

yahoo-mmcrawler

Rule Path
Disallow /

msnbot

Rule Path
Disallow /

psbot

Rule Path
Disallow /

*

Rule Path
Disallow /IAW
Disallow /temp
Disallow /wp-admin
Disallow /wp-content
Disallow /wp-includes
Disallow /*.zip$
Disallow /*.Zip$
Disallow /*.ZIP$

Other Records

Field Value
crawl-delay 10.0