andersenwindows.com
robots.txt

Robots Exclusion Standard data for andersenwindows.com

Resource Scan

Scan Details

Site Domain andersenwindows.com
Base Domain andersenwindows.com
Scan Status Ok
Last Scan2024-06-23T07:17:44+00:00
Next Scan 2024-07-23T07:17:44+00:00

Last Scan

Scanned2024-06-23T07:17:44+00:00
URL https://andersenwindows.com/robots.txt
Redirect https://www.andersenwindows.com/robots.txt
Redirect Domain www.andersenwindows.com
Redirect Base andersenwindows.com
Domain IPs 76.76.21.241, 76.76.21.9
Redirect IPs 76.76.21.123, 76.76.21.142
Response IP 76.76.21.123
Found Yes
Hash 01f86e1b7fa85c87f0af2dc996c8e18a7b829413fe0cf15c2f70f1b78dd1e738
SimHash 48160b708f80

Groups

*

Rule Path Comment
Disallow /search/ Block search results for site search or learning navigation with facets chosen (Temporarily block the whole directory)
Disallow /search/search* Block site search results
Disallow /search/go* Block site search results
Disallow /search/popular/ Block popular site search results
Disallow /business/search Block search results for Where to Buy or Find a Service Provider
Disallow /*.swf$ Block files ending .swf (Flash media)
Disallow /*.doc$ Block files ending .doc (Word document)
Disallow /*.dwg$ Block files ending .dwg
Disallow /sitecore/content/ Don't allow access to Sitecore directory (Include meta robots noindex,nofolow tag)
Disallow /thd-connect Block /thd-connect from being indexed
Disallow /api/ Block /api from being indexed
Disallow *?frameColor -
Disallow *?exteriorTrim -
Disallow *?exteriorTrimColor -
Disallow *?frameColorExt -
Disallow *?frameStain -
Disallow *?hardware -
Disallow *?grilleStyle -
Disallow *?hardwareOptions -
Disallow *?side -
Disallow *?exteriorSash -
Disallow *?reside -
Disallow *?widIn -
Disallow *?hgtIn -
Disallow *?grilleLightsWide -
Disallow *?grilleLightsHigh -
Disallow *?grilleSpacing -
Disallow *?glass -

ahrefsbot

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

sitebot

Rule Path
Disallow /

megaindex

Rule Path
Disallow /

Other Records

Field Value
sitemap https://www.andersenwindows.com/sitemap.xml

Comments

  • Block crawlers from accessing facet pages
  • Block malicious bots
  • SITEMAPS