petbase.eu
robots.txt

Robots Exclusion Standard data for petbase.eu

Resource Scan

Scan Details

Site Domain petbase.eu
Base Domain petbase.eu
Scan Status Ok
Last Scan2026-01-29T13:38:38+00:00
Next Scan 2026-02-28T13:38:38+00:00

Last Scan

Scanned2026-01-29T13:38:38+00:00
URL https://petbase.eu/robots.txt
Domain IPs 104.26.8.45, 104.26.9.45, 172.67.68.124, 2606:4700:20::681a:82d, 2606:4700:20::681a:92d, 2606:4700:20::ac43:447c
Response IP 104.26.8.45
Found Yes
Hash 7ae5be3c16816605975baf46585bab53214c8fe19adaaeee370f81acb7bcbb2f
SimHash 658a3c57c020

Groups

*

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20

Other Records

Field Value
sitemap https://www.petbase.eu/page-sitemap.xml
sitemap https://www.petbase.eu/post-sitemap.xml
sitemap https://www.petbase.eu/product-sitemap.xml
sitemap https://www.petbase.eu/category-sitemap.xml
sitemap https://www.petbase.eu/post_tag-sitemap.xml
sitemap https://www.petbase.eu/product_cat-sitemap.xml