industrialcafe.it
robots.txt
Robots Exclusion Standard data for industrialcafe.it
Resource Scan
Scan Details
| Site Domain | industrialcafe.it |
| Base Domain | industrialcafe.it |
| Scan Status | Ok |
| Last Scan | 2026-02-03T00:38:25+00:00 |
| Next Scan | 2026-02-10T00:38:25+00:00 |
Last Scan
| Scanned | 2026-02-03T00:38:25+00:00 |
| URL | https://industrialcafe.it/robots.txt |
| Redirect | https://www.industrialcafe.it/robots.txt |
| Redirect Domain | www.industrialcafe.it |
| Redirect Base | industrialcafe.it |
| Domain IPs | 104.21.70.3, 172.67.217.41, 2606:4700:3031::6815:4603, 2606:4700:3037::ac43:d929 |
| Redirect IPs | 104.21.70.3, 172.67.217.41, 2606:4700:3031::6815:4603, 2606:4700:3037::ac43:d929 |
| Response IP | 104.21.70.3 |
| Found | Yes |
| Hash | fc35d26d18c98e7a31333f8a597b7c338e94f39545e3f9430f515e868a633469 |
| SimHash | 5b9d5c322ab1 |
Groups
ahrefsbot
aspiegelbot
blexbot
barkrowler
dotbot
mj12bot
mauibot
nimbostratus-bot
petalbot
semrushbot
seznambot
sogou
serpstatbot
trendiction
textbulkerbot
No rules defined. All paths allowed.
Other Records
| Field | Value |
|---|---|
| crawl-delay | 180 |
*
| Rule | Path |
|---|---|
| Disallow | /wp-admin/ |
| Disallow | /cdn-cgi/* |
| Disallow | /cdn-fpw/* |
| Disallow | */1000$ |
| Allow | /wp-admin/admin-ajax.php |
| Allow | /*css?* |
| Allow | /*js?* |
Other Records
| Field | Value |
|---|---|
| sitemap | https://butzbacher-frischemarkt.de/sitemap_index.xml |
| sitemap | https://butzbacher-frischemarkt.de/post-sitemap.xml |
| sitemap | https://butzbacher-frischemarkt.de/sitemap-news.xml |
Warnings
- 1 invalid line.