thepiel.com
robots.txt

Robots Exclusion Standard data for thepiel.com

Resource Scan

Scan Details

Site Domain thepiel.com
Base Domain thepiel.com
Scan Status Ok
Last Scan2025-10-19T05:44:39+00:00
Next Scan 2025-11-18T05:44:39+00:00

Last Scan

Scanned2025-10-19T05:44:39+00:00
URL https://thepiel.com/robots.txt
Domain IPs 104.26.8.59, 104.26.9.59, 172.67.75.163, 2606:4700:20::681a:83b, 2606:4700:20::681a:93b, 2606:4700:20::ac43:4ba3
Response IP 104.26.9.59
Found Yes
Hash ade8aa69748c0b1d6587b58596260b90e3e50380c5921f0d3451b52c6925ef40
SimHash 69045c068391

Groups

*

Rule Path
Allow
Disallow /admin/

Other Records

Field Value
crawl-delay 10

Other Records

Field Value
sitemap https://www.thepiel.com/sitemap.xml
sitemap https://www.thepiel.com/sitemap_news.xml
sitemap https://www.thepiel.com/sitemap_mobile.xml
sitemap https://www.thepiel.com/sitemap_images.xml
sitemap https://www.thepiel.com/sitemap_video.xml