moneyshow.com
robots.txt

Robots Exclusion Standard data for moneyshow.com

Resource Scan

Scan Details

Site Domain moneyshow.com
Base Domain moneyshow.com
Scan Status Ok
Last Scan2024-10-08T15:36:13+00:00
Next Scan 2024-10-15T15:36:13+00:00

Last Scan

Scanned2024-10-08T15:36:13+00:00
URL https://moneyshow.com/robots.txt
Redirect https://www.moneyshow.com/robots.txt
Redirect Domain www.moneyshow.com
Redirect Base moneyshow.com
Domain IPs 20.62.250.143
Redirect IPs 20.62.250.143
Response IP 20.62.250.143
Found Yes
Hash 8c62002ec554b274a5c968a3a73ed3208fdc6c70aa8da55ac6bde3fd07f217aa
SimHash 414ce905c552

Groups

*

Rule Path
Disallow /aspnet_client/
Disallow /bin/
Disallow /config/
Disallow /App_Data/
Disallow /shopping-cart/
Disallow /do-not-sell-my-information/
Disallow /partials/
Disallow /dashboard/
Disallow /video/bcf9e99a88644355b1e89b32be3dd1d71/

discoverybot

Rule Path
Disallow /

megaindex.ru/2.0

Rule Path
Disallow /

megaindex.ru

Rule Path
Disallow /

megaindex.ru

Rule Path
Disallow /

yandex

Rule Path
Disallow /

urlblacklistskafkaworkers-urlchecksworker

Rule Path
Disallow /

baiduspider
baiduspider-video
baiduspider-image

Rule Path
Disallow /

youdaobot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

punkspider/5.0

Rule Path
Disallow /

Other Records

Field Value
sitemap https://www.moneyshow.com/sitemap.xml
sitemap https://www.moneyshow.com/google-news-sitemap.xml

Comments

  • Disallow: /quotes/ MG commented out 1/8/24 to allow for it to be crawled again
  • Disallow: /library/ MG commented out 1/17/24 to allow for it to be crawled
  • Block MegaIndex.ru
  • Block YandexBot
  • Block Hubspot
  • Block Baidu
  • Block Youdao
  • Block Majestic
  • PunkSpider