carboners.cat
robots.txt

Robots Exclusion Standard data for carboners.cat

Resource Scan

Scan Details

Site Domain carboners.cat
Base Domain carboners.cat
Scan Status Ok
Last Scan2025-11-12T02:57:26+00:00
Next Scan 2025-12-12T02:57:26+00:00

Last Scan

Scanned2025-11-12T02:57:26+00:00
URL http://carboners.cat/robots.txt
Redirect https://carbonersterrassa.weebly.com/robots.txt
Redirect Domain carbonersterrassa.weebly.com
Redirect Base weebly.com
Domain IPs 34.77.10.20
Redirect IPs 74.115.51.8, 74.115.51.9
Response IP 74.115.51.9
Found Yes
Hash 1fbcb31b7f43364ec9ae5f7536cf0fff11e4ccf5ca6b2e22d21e902a606ec474
SimHash 6954dc746f93

Groups

nerdybot

Rule Path
Disallow /

dotbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

*

Rule Path
Disallow /ajax/
Disallow /apps/

Other Records

Field Value
sitemap https://carbonersterrassa.weebly.com/sitemap.xml