themescene.tv
robots.txt

Robots Exclusion Standard data for themescene.tv

Resource Scan

Scan Details

Site Domain themescene.tv
Base Domain themescene.tv
Scan Status Ok
Last Scan2024-09-29T01:27:33+00:00
Next Scan 2024-10-06T01:27:33+00:00

Last Scan

Scanned2024-09-29T01:27:33+00:00
URL https://themescene.tv/robots.txt
Domain IPs 104.21.8.102, 172.67.157.72, 2606:4700:3030::ac43:9d48, 2606:4700:3033::6815:866
Response IP 172.67.157.72
Found Yes
Hash 37a03323eeddda3d16f9ad22c98a1ddf078418fd8b66a5252ae15e33cca777f4
SimHash 65105c52c191

Groups

*

Rule Path
Disallow /cdn-cgi/

Other Records

Field Value
crawl-delay 20