americas.rwe.com
robots.txt

Robots Exclusion Standard data for americas.rwe.com

Resource Scan

Scan Details

Site Domain americas.rwe.com
Base Domain rwe.com
Scan Status Ok
Last Scan2024-05-31T07:15:30+00:00
Next Scan 2024-06-30T07:15:30+00:00

Last Scan

Scanned2024-05-31T07:15:30+00:00
URL https://americas.rwe.com/robots.txt
Domain IPs 128.65.209.37
Response IP 128.65.209.37
Found Yes
Hash 89c5da07381f5787c2cba1bda39397a67022190234c68d59c887b3f5c1c9c2b8
SimHash 12752b6d2631

Groups

chatgpt-user

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

yisouspider

Rule Path
Disallow /

xovibot

Rule Path
Disallow /

admantxbot

Rule Path
Disallow /

proximic

Rule Path
Disallow /

wordpress

Rule Path
Disallow /wp-admin/

sistrix crawler

Rule Path
Disallow /

dataforseobot

Rule Path
Disallow /

thinkers-bot

Rule Path
Disallow /

siteimprovebot

Rule Path
Disallow /

siteimprovebot-crawler

Rule Path
Disallow /

alittle client

Rule Path
Disallow /

*

Rule Path
Disallow /App_Browsers/
Disallow /App_Config/
Disallow /App_Start/
Disallow /Areas/
Disallow /Artifacts/
Disallow /bin/
Disallow /Content/
Disallow /layouts/
Disallow /Properties/
Disallow /Scripts/
Disallow /sitecore/
Disallow /sitecore_files/
Disallow /temp/
Disallow /upload/
Disallow /Views/
Disallow /xsl/
Disallow */sitecore/Inhalt/*
Disallow */sitecore/Content/*
Disallow /Default.aspx
Disallow /default.css
Disallow /default.htm.sitedown
Disallow /default.js
Disallow /Web.config.install.xdt
Disallow /Web.config.uninstall.xdt
Disallow /webedit.css
Disallow /Global.asax
Disallow /lb.txt
Disallow /web.config
Allow /

Comments

  • directory
  • files

Warnings

  • 1 invalid line.