instructables.com
robots.txt

Robots Exclusion Standard data for instructables.com

Resource Scan

Scan Details

Site Domain instructables.com
Base Domain instructables.com
Scan Status Ok
Last Scan2024-09-19T15:59:27+00:00
Next Scan 2024-10-19T15:59:27+00:00

Last Scan

Scanned2024-09-19T15:59:27+00:00
URL https://instructables.com/robots.txt
Redirect https://www.instructables.com/robots.txt
Redirect Domain www.instructables.com
Redirect Base instructables.com
Domain IPs 151.101.1.105, 151.101.129.105, 151.101.193.105, 151.101.65.105
Redirect IPs 151.101.1.105, 151.101.129.105, 151.101.193.105, 151.101.65.105, 2a04:4e42:200::361, 2a04:4e42:400::361, 2a04:4e42:600::361, 2a04:4e42::361
Response IP 199.232.45.105
Found Yes
Hash 6e48d7df0d6cafbfed9b8836e250bd5b3d8deb7f42cdd27dc15c1e455d5fd691
SimHash 6b64fd75c775

Groups

mediapartners-google*

Rule Path
Disallow

slurp

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 1

adsbot-google*

Rule Path
Disallow

*

Rule Path
Disallow /*.pdf$
Disallow /*.txt$
Disallow /*.html$
Disallow /file/*
Disallow /howto/*
Disallow /image/*

Other Records

Field Value
sitemap https://www.instructables.com/sitemap.xml