podyart.com
robots.txt

Robots Exclusion Standard data for podyart.com

Resource Scan

Scan Details

Site Domain podyart.com
Base Domain podyart.com
Scan Status Ok
Last Scan2025-05-13T09:33:15+00:00
Next Scan 2025-06-12T09:33:15+00:00

Last Scan

Scanned2025-05-13T09:33:15+00:00
URL https://podyart.com/robots.txt
Domain IPs 104.21.9.129, 172.67.130.222, 2606:4700:3030::ac43:82de, 2606:4700:3031::6815:981
Response IP 172.67.130.222
Found Yes
Hash 32e045a81b9d3cb901a814c0fba0ee011f75e32f26f7d483627cca5f36cc8811
SimHash 4040da144313

Groups

scrapy

Rule Path
Allow /
Disallow /en/

Warnings

  • `user-agent` is not a known field.