etymonline.com
robots.txt

Robots Exclusion Standard data for etymonline.com

Resource Scan

Scan Details

Site Domain etymonline.com
Base Domain etymonline.com
Scan Status Ok
Last Scan2024-11-14T23:48:53+00:00
Next Scan 2024-11-21T23:48:53+00:00

Last Scan

Scanned2024-11-14T23:48:53+00:00
URL https://etymonline.com/robots.txt
Redirect https://www.etymonline.com/robots.txt
Redirect Domain www.etymonline.com
Redirect Base etymonline.com
Domain IPs 18.209.254.197
Redirect IPs 18.209.254.197
Response IP 18.209.254.197
Found Yes
Hash 878634edd324df52327003714d196d0918c59c40468fac189fa507743d6566a3
SimHash 781cb596eac1

Groups

googlebot

Rule Path
Disallow /admin/
Disallow /app/feedback

bingbot

Rule Path
Disallow /classic
Disallow /classic/
Disallow /admin/
Disallow /app/feedback

ccbot

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

semrushbot-ba

Rule Path
Disallow /

semrushbot-sa

Rule Path
Disallow /

siteauditbot

Rule Path
Disallow /

splitsignalbot

Rule Path
Disallow /

semrushbot-coub

Rule Path
Disallow /

semrushbot-si

Rule Path
Disallow /

semrushbot-swa

Rule Path
Disallow /

semrushbot-ct

Rule Path
Disallow /

semrushbot-bm

Rule Path
Disallow /

semrushbot-seoab

Rule Path
Disallow /

teleport

Rule Path
Disallow /

teleportpro

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

*

Rule Path
Allow /search
Allow /word/
Disallow /web/login
Disallow /classic
Disallow /classic/
Disallow /admin/
Disallow /app/feedback

Other Records

Field Value
sitemap https://www.etymonline.com/landings.xml
sitemap https://www.etymonline.com/sitemap.xml
sitemap https://www.etymonline.com/sitemap-article.xml
sitemap https://www.etymonline.com/locale-sitemap/home.xml
sitemap https://www.etymonline.com/sitemaps/words-locale.xml