tuitionrewards.com
robots.txt

Robots Exclusion Standard data for tuitionrewards.com

Resource Scan

Scan Details

Site Domain tuitionrewards.com
Base Domain tuitionrewards.com
Scan Status Ok
Last Scan2024-09-03T03:34:36+00:00
Next Scan 2024-10-03T03:34:36+00:00

Last Scan

Scanned2024-09-03T03:34:36+00:00
URL https://tuitionrewards.com/robots.txt
Redirect https://www.tuitionrewards.com/robots.txt
Redirect Domain www.tuitionrewards.com
Redirect Base tuitionrewards.com
Domain IPs 66.252.104.206
Redirect IPs 66.252.104.206
Response IP 66.252.104.206
Found Yes
Hash c429ed291019395cfed90ed452a11eba2cc6398031677d0e6f8e1459991972ee
SimHash 73de0854bbd0

Groups

ahrefsbot

Rule Path
Disallow /

blekkobot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

ccbot

Rule Path
Disallow /

claudebot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

dow jones searchbot

Rule Path
Disallow /

grapeshot

Rule Path
Disallow /

grapeshotcrawler

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

panscient.com

Rule Path
Disallow /

seznambot

Rule Path
Disallow /

smtbot

Rule Path
Disallow /

yandexbot

Rule Path
Disallow /

*

Rule Path
Disallow /.vscode/
Disallow /.git/

Other Records

Field Value
sitemap https://www.tuitionrewards.com/sitemap.xml

Comments

  • Limit 1 page every 5 seconds
  • Disallow: /assets/
  • Disallow: /*?p=newsroom&class=*&pkid=*&alias=*