Log In Sign Up

todayspredict.com
robots.txt

Robots Exclusion Standard data for todayspredict.com

Resource Scan

Scan Details

Site Domain todayspredict.com
Base Domain todayspredict.com
Scan Status Ok
Last Scan2025-12-03T17:10:24+00:00
Next Scan 2025-12-10T17:10:24+00:00

Last Scan

Scanned2025-12-03T17:10:24+00:00
URL https://todayspredict.com/robots.txt
Domain IPs 104.21.38.12, 172.67.216.204, 2606:4700:3030::ac43:d8cc, 2606:4700:3037::6815:260c
Response IP 104.21.38.12
Found Yes
Hash 32b6421a760cac900f1cf74dc0a1f1f4911a50f0b7bd15c9d6b2fa388e3145f5
SimHash 2c1c541606e0

Groups

*

Rule Path
Disallow
Disallow /admin/
Disallow /login/
Disallow /private/
Disallow /tmp/
Disallow /*?type=
Disallow /*?page=
Disallow /*?dt=
Disallow /*?date=
Disallow /*index.php

Other Records

Field Value
crawl-delay 10

googlebot

Rule Path
Allow /

bingbot

Rule Path
Allow /

badbot

Rule Path
Disallow /
Disallow /*?sessionid=
Disallow /*?sort=

*

Rule Path
Allow /*.css$
Allow /*.js$
Allow /*.jpg$
Allow /*.png$
Allow /*.gif$
Allow /*.svg$
Allow /*.xml$
Allow /*.json$
Allow /*.txt$
Back to top

Other Records

Field Value
sitemap https://todayspredict.com/sitemap.xml
Back to top

Comments

  • Disallow specific directories
  • Disallow URLs with specific query parameters
  • Disallow URLs with index.php in the path
  • Specific user-agent rules
  • Disallow sessionid and sort query parameters
  • Allow specific file types
Back to top