timeform.com
robots.txt

Robots Exclusion Standard data for timeform.com

Resource Scan

Scan Details

Site Domain timeform.com
Base Domain timeform.com
Scan Status Failed
Failure ReasonScan timed out.
Last Scan2024-09-03T18:15:38+00:00
Next Scan 2024-12-02T18:15:38+00:00

Last Successful Scan

Scanned2024-04-14T16:07:44+00:00
URL https://timeform.com/robots.txt
Redirect https://www.timeform.com/robots.txt
Redirect Domain www.timeform.com
Redirect Base timeform.com
Domain IPs 13.107.213.59, 13.107.246.59
Redirect IPs 13.107.213.59, 13.107.246.59, 2620:1ec:46::59, 2620:1ec:bdf::59
Response IP 13.107.213.59
Found Yes
Hash 2cce1b035cb19b73d1287d63a0d910ea42540bf0bb7875a9c25205140e62204f
SimHash 5966c2caf599

Groups

*
*

Rule Path
Disallow /something-has-gone-wrong
Disallow /horse-racing/something-has-gone-wrong
Disallow /horse-racing/account/sign-in
Disallow /free-bets/reroute/

yandex
baiduspider
semrushbot
petalbot
mj12bot
blexbot
dotbot
ahrefsbot
gptbot

Rule Path
Disallow /

Other Records

Field Value
sitemap https://www.timeform.com/sitemap.xml

Comments

  • Don't index the reroute or error url's
  • Block access to whole site for badly behaved crawlers