spiru.be
robots.txt

Robots Exclusion Standard data for spiru.be

Resource Scan

Scan Details

Site Domain spiru.be
Base Domain spiru.be
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2025-06-05T06:19:38+00:00
Next Scan 2025-09-03T06:19:38+00:00

Last Successful Scan

Scanned2023-10-22T04:39:13+00:00
URL https://spiru.be/robots.txt
Domain IPs 104.26.14.232, 104.26.15.232, 172.67.72.91, 2606:4700:20::681a:ee8, 2606:4700:20::681a:fe8, 2606:4700:20::ac43:485b
Response IP 104.26.14.232
Found Yes
Hash c8e4d68ca6a9dcdad14b932b1c24e6d2963921ed34b511023169d2768be765ee
SimHash 609c5c55cbb6

Groups

*

Rule Path
Disallow /wp-admin/
Disallow /*?*add-to-cart=*
Allow /wp-admin/admin-ajax.php

googlebot

Rule Path
Disallow

googlebot-image

Rule Path
Disallow

pinterestbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 0.8

Other Records

Field Value
sitemap https://spiru.be/sitemap.xml