solarreviews.com
robots.txt

Robots Exclusion Standard data for solarreviews.com

Resource Scan

Scan Details

Site Domain solarreviews.com
Base Domain solarreviews.com
Scan Status Ok
Last Scan2025-02-15T18:56:32+00:00
Next Scan 2025-03-17T18:56:32+00:00

Last Scan

Scanned2025-02-15T18:56:32+00:00
URL https://solarreviews.com/robots.txt
Redirect https://www.solarreviews.com/robots.txt
Redirect Domain www.solarreviews.com
Redirect Base solarreviews.com
Domain IPs 104.239.244.244
Redirect IPs 151.101.130.132, 151.101.194.132, 151.101.2.132, 151.101.66.132
Response IP 199.232.46.132
Found Yes
Hash 6cff36e28c9b5136efb03d0de4b4c931b1c04fd1c1d3f9b9ce05efdb9121448b
SimHash e44ec8707a39

Groups

*

Rule Path
Allow /remote/link
Allow /remote/companydisplayaddresses
Allow /remote/viewthumbnail_bootstrap
Disallow /remote/
Disallow /advertising/
Disallow /external/
Disallow /generator/
Disallow /monitor/
Disallow /utilities/
Disallow /login/
Disallow /company-registration/
Disallow /landing/

*

Rule Path
Disallow /content/page/*/$
Allow /content/page/*/*.*$

ahrefsbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 1

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 1

yandex

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

slurp

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

istellabot

Rule Path
Disallow /

istellabot/1.01.18

Rule Path
Disallow /

istellabot/1.01.18 +http://www.tiscali.it/

Rule Path
Disallow /

istellabot/1.10.2 +http://www.tiscali.it/

Rule Path
Disallow /

mozilla/5.0 (compatible; istellabot/1.01.18 +http://www.tiscali.it/)

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

woobot

Rule Path
Disallow /

woobot/1.1

Rule Path
Disallow /

woobot/2.0

Rule Path
Disallow /

Other Records

Field Value
sitemap https://www.solarreviews.com/sitemapindex.xml

Comments

  • robots.txt
  • Disallow parent content pages