fleminghowland.com
robots.txt

Robots Exclusion Standard data for fleminghowland.com

Resource Scan

Scan Details

Site Domain fleminghowland.com
Base Domain fleminghowland.com
Scan Status Ok
Last Scan2025-03-26T16:04:23+00:00
Next Scan 2025-04-25T16:04:23+00:00

Last Scan

Scanned2025-03-26T16:04:23+00:00
URL https://fleminghowland.com/robots.txt
Domain IPs 104.26.0.96, 104.26.1.96, 172.67.71.146, 2606:4700:20::681a:160, 2606:4700:20::681a:60, 2606:4700:20::ac43:4792
Response IP 104.26.0.96
Found Yes
Hash a28aebbb88eef3e924ddef8f153b1ad46c503f946c0fd296de334ae11f0d355d
SimHash c00059057311

Groups

scrapy

Rule Path
Allow /

*

Rule Path
Disallow /wp/wp-admin/
Disallow /my-account
Disallow /filterable-categories/
Disallow /de/filterable-categories/
Disallow /fr/filterable-categories/
Disallow /fr/categorie-produit/
Disallow /product-category/
Disallow /de/produkt-kategorie/
Disallow /tag/
Disallow /fr/tag/
Disallow /de/tag/
Disallow /de/faq-categories/
Disallow /fr/faq-categories/
Disallow /faq-categories/
Disallow /send-estimate/
Allow /wp/wp-admin/admin-ajax.php
Allow /wp/wp-content/uploads/

Other Records

Field Value
crawl-delay 10