Log In Sign Up

vistra.com
robots.txt

Robots Exclusion Standard data for vistra.com

Resource Scan

Scan Details

Site Domain vistra.com
Base Domain vistra.com
Scan Status Ok
Last Scan2024-05-20T16:45:30+00:00
Next Scan 2024-06-19T16:45:30+00:00

Last Scan

Scanned2024-05-20T16:45:30+00:00
URL https://vistra.com/robots.txt
Redirect https://www.vistra.com:443/robots.txt
Redirect Domain www.vistra.com
Redirect Base vistra.com
Domain IPs 75.2.123.182, 99.83.156.75
Redirect IPs 104.22.66.214, 104.22.67.214, 172.67.9.213, 2606:4700:10::6816:42d6, 2606:4700:10::6816:43d6, 2606:4700:10::ac43:9d5
Response IP 104.22.67.214
Found Yes
Hash 71e70b209c40fb10d28b0b8bcc3f896d019080a17454cd9d75a63fdf90fcab38
SimHash 29061d02cd51

Groups

chatgpt-user

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

googlebot-news

Rule Path Comment
Disallow /people/ -
Disallow /*/people/ -
Disallow /core/ -
Disallow /profiles/ -
Disallow /*/profiles/ -
Disallow /README.txt -
Disallow /web.config -
Disallow /admin/ -
Disallow /*/admin/ -
Disallow /comment/reply/ -
Disallow /*/comment/reply/ -
Disallow /filter/tips -
Disallow /*/filter/tips -
Disallow /node/ PP changed from /node/add/
Disallow /*/node/ -
Disallow /search/? PP added "?"
Disallow /*/search/? -
Disallow /user/ PP added
Disallow /*/user/ -
Disallow /user/register/ -
Disallow /*/user/register/ -
Disallow /user/password/ -
Disallow /*/user/password/ -
Disallow /user/login/ -
Disallow /*/user/login/ -
Disallow /user/logout/ -
Disallow /*/user/logout/ -
Disallow /term-listing/ -
Disallow /*/term-listing/ -
Disallow /print/ -
Disallow /*/print/ -
Allow /*.js -
Allow /*.css -
Allow /*.gif -
Allow /*.jpeg -
Allow /*.jpg -
Allow /*.png -
Allow /*.svg -
Allow /*.woff2 -
Allow /*utm_campaign%3Dgoogle_search_box -

*

Rule Path Comment
Disallow /core/ -
Disallow /profiles/ -
Disallow /*/profiles/ -
Disallow /README.txt -
Disallow /web.config -
Disallow /admin/ -
Disallow /*/admin/ -
Disallow /comment/reply/ -
Disallow /*/comment/reply/ -
Disallow /filter/tips -
Disallow /*/filter/tips -
Disallow /node/ PP changed from /node/add/
Disallow /*/node/ -
Disallow /search/? PP added "?"
Disallow /*/search/? -
Disallow /user/ PP added
Disallow /*/user/ -
Disallow /user/register/ -
Disallow /*/user/register/ -
Disallow /user/password/ -
Disallow /*/user/password/ -
Disallow /user/login/ -
Disallow /*/user/login/ -
Disallow /user/logout/ -
Disallow /*/user/logout/ -
Disallow /term-listing/ -
Disallow /*/term-listing/ -
Disallow /print/ -
Disallow /*/print/ -
Allow /*.js -
Allow /*.css -
Allow /*.gif -
Allow /*.jpeg -
Allow /*.jpg -
Allow /*.png -
Allow /*.svg -
Allow /*.woff2 -
Allow /*utm_campaign%3Dgoogle_search_box -

meanpathbot
megaindex.ru/2.0
megaindex.ru/
ahrefsbot
sogou web spider
qwantify
barkrowler
petalbot

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap https://www.vistra.com/sitemap.xml
Back to top

Comments

  • https://www.vistra.com/robots.txt
  • This file is to prevent the crawling and indexing of certain parts
  • of your site by web crawlers and spiders run by sites like Yahoo!
  • and Google. By telling these "robots" where not to go on your site,
  • you save bandwidth and server resources.
  • This file will be ignored unless it is at the root of your host:
  • Used: https://www.vistra.com/robots.txt
  • Ignored: https://www.vistra.com/site/robots.txt
  • For more information about the robots.txt standard, see:
  • http://www.robotstxt.org/robotstxt.html
  • Start: Block ChatGPT crawler
  • End
  • Directories
  • Files
  • Paths (clean URLs)
  • Paths (no clean URLs)
  • Disallow: /de/taxonomy/term/ #canonicalised so not deindexed
  • Disallow: /zh-hans/taxonomy/term/ #canonicalised so not deindexed
  • CSS, JS, Images
  • Directories
  • Files
  • Paths (clean URLs)
  • Paths (no clean URLs)
  • Disallow: /de/taxonomy/term/ #canonicalised so not deindexed
  • Disallow: /zh-hans/taxonomy/term/ #canonicalised so not deindexed
  • CSS, JS, Images
  • BB Additional rules
Back to top