home-remodeling-directory.com
robots.txt

Robots Exclusion Standard data for home-remodeling-directory.com

Resource Scan

Scan Details

Site Domain home-remodeling-directory.com
Base Domain home-remodeling-directory.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonCouldn't connect to server.
Last Scan2024-05-16T10:18:43+00:00
Next Scan 2024-07-15T10:18:43+00:00

Last Successful Scan

Scanned2024-03-18T10:17:04+00:00
URL https://home-remodeling-directory.com/robots.txt
Redirect https://www.home-remodeling-directory.com/robots.txt
Redirect Domain www.home-remodeling-directory.com
Redirect Base home-remodeling-directory.com
Domain IPs 104.21.34.120, 172.67.160.185, 2606:4700:3030::6815:2278, 2606:4700:3031::ac43:a0b9
Redirect IPs 104.21.34.120, 172.67.160.185, 2606:4700:3030::6815:2278, 2606:4700:3031::ac43:a0b9
Response IP 104.21.34.120
Found Yes
Hash 43c78b9b187fc45a48ae6cde7063a8e43008da3026e5c2ee97e75fa209e01ff9
SimHash e91c5a40c7f7

Groups

*

Rule Path
Allow /
Allow /wp-content/*.css
Disallow /cgi-bin
Disallow /wp-content/themes/
Disallow /tag/*/page/
Disallow /tag/*/feed/
Disallow /page/
Disallow /comments/
Disallow /xmlrpc.php
Disallow /trackback
Allow /feed/$
Disallow /feed/
Disallow /*/feed/$
Disallow /*/feed/rss/$
Disallow /*/*/trackback/$
Disallow /*/*/*/feed/$
Disallow /*/*/*/feed/rss/$

duckduckbot

Rule Path
Disallow /Sitemap

googlebot

Rule Path
Disallow /Sitemap

sogou

Rule Path
Disallow /Sitemap

bingbot

Rule Path
Disallow /Sitemap

msiecrawler

Rule Path
Disallow /

webcopier

Rule Path
Disallow /

httrack

Rule Path
Disallow /

microsoft.url.control

Rule Path
Disallow /

zao

Rule Path
Disallow /

sitecheck.internetseer.com

Rule Path
Disallow /

sitesnagger

Rule Path
Disallow /

webstripper

Rule Path
Disallow /

webcopier

Rule Path
Disallow /

offline explorer

Rule Path
Disallow /

teleport

Rule Path
Disallow /

webzip

Rule Path
Disallow /

microsoft.url.control

Rule Path
Disallow /

larbin

Rule Path
Disallow /

zyborg

Rule Path
Disallow /

wget

Rule Path
Disallow /

grub-client

Rule Path
Disallow /

k2spider

Rule Path
Disallow /

webreaper

Rule Path
Disallow /

Other Records

Field Value
sitemap https://www.home-remodeling-directory.com/sitemapindex.xml