gb.education.com
robots.txt

Robots Exclusion Standard data for gb.education.com

Resource Scan

Scan Details

Site Domain gb.education.com
Base Domain education.com
Scan Status Ok
Last Scan2024-09-22T13:22:15+00:00
Next Scan 2024-10-22T13:22:15+00:00

Last Scan

Scanned2024-09-22T13:22:15+00:00
URL https://gb.education.com/robots.txt
Redirect https://www.education.com:443/robots.txt
Redirect Domain www.education.com
Redirect Base education.com
Domain IPs 23.21.151.220, 3.94.43.22, 34.193.243.200
Redirect IPs 104.18.42.23, 172.64.145.233, 2606:4700:4400::6812:2a17, 2606:4700:4400::ac40:91e9
Response IP 104.18.42.23
Found Yes
Hash d89fce262db2ae867ac9adce5f78464e189eaf31f5763a9806b7b5e424fc3270
SimHash 2b9ef151cb11

Groups

*

Rule Path
Disallow /admin/
Disallow /results/
Disallow /print/
Disallow /toast/
Disallow /downloading.php
Disallow /*.pdf$
Disallow /download/
Disallow /pdf/
Disallow /profile/
Disallow /openx/
Disallow /openads/
Disallow /schoolfinder/searchresult/
Disallow /widget/
Disallow /lesson-related-content/
Disallow /summer-sale/
Disallow /*?q=
Disallow /*%26q%3D

slurp

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 0.5

ahrefsbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 1

grapeshot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 1

rogerbot

Rule Path
Disallow /about/
Disallow /getlocal/
Disallow /privacy/
Disallow /tags/
Disallow /reference/
Disallow /schoolfinder/
Disallow /topic/
Disallow /study-help/
Disallow /answers/

gptbot

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

applebot-extended

Rule Path
Disallow /

Other Records

Field Value
sitemap https://www.education.com/sitemapindex.xml