ctftime.org
robots.txt

Robots Exclusion Standard data for ctftime.org

Resource Scan

Scan Details

Site Domain ctftime.org
Base Domain ctftime.org
Scan Status Ok
Last Scan2025-09-28T15:27:50+00:00
Next Scan 2025-10-05T15:27:50+00:00

Last Scan

Scanned2025-09-28T15:27:50+00:00
URL https://ctftime.org/robots.txt
Domain IPs 104.21.96.2, 172.67.171.235, 2606:4700:3034::6815:6002, 2606:4700:3035::ac43:abeb
Response IP 172.67.171.235
Found Yes
Hash 46d0b8bf29de378d48953cbb03fe5cd3ba46f989ab5836288bb2603301297181
SimHash 007c9d73eeb1

Groups

*

Rule Path
Disallow /user/
Disallow /task/tags/
Disallow /login/
Disallow /export/
Disallow /dating/

Other Records

Field Value
crawl-delay 10

dotbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

semrushbot-sa

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

mail.ru_bot

Rule Path
Disallow /

mail.ru

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

mail.ru_bot

Rule Path
Disallow /

Comments

  • public data for rating experiments
  • secret page