corruptbox.net
robots.txt

Robots Exclusion Standard data for corruptbox.net

Resource Scan

Scan Details

Site Domain corruptbox.net
Base Domain corruptbox.net
Scan Status Ok
Last Scan2025-10-29T19:16:31+00:00
Next Scan 2025-11-05T19:16:31+00:00

Last Scan

Scanned2025-10-29T19:16:31+00:00
URL https://corruptbox.net/robots.txt
Domain IPs 104.21.88.229, 172.67.153.182, 2606:4700:3032::6815:58e5, 2606:4700:3037::ac43:99b6
Response IP 172.67.153.182
Found Yes
Hash 737a24f1cb2b321f548ca84df2b8793d2daecf2a6acbedc974b54bd1fa4e6444
SimHash 11445c51ca12

Groups

*

Rule Path
Allow /
Allow /en/
Disallow /Base-Template/
Disallow /css/
Disallow /js/
Disallow /img/
Disallow /node_modules/
Disallow /.git/

Other Records

Field Value
crawl-delay 10

Other Records

Field Value
sitemap https://corruptbox.net/sitemap.xml

Comments

  • robots.txt for https://corruptbox.net
  • 站点地图
  • 爬虫限制
  • 禁止访问的目录