fool.com
robots.txt

Robots Exclusion Standard data for fool.com

Resource Scan

Scan Details

Site Domain fool.com
Base Domain fool.com
Scan Status Ok
Last Scan2024-11-16T20:13:08+00:00
Next Scan 2024-11-23T20:13:08+00:00

Last Scan

Scanned2024-11-16T20:13:08+00:00
URL https://fool.com/robots.txt
Redirect https://www.fool.com/robots.txt
Redirect Domain www.fool.com
Redirect Base fool.com
Domain IPs 104.18.24.135, 104.18.25.135
Redirect IPs 104.18.40.166, 172.64.147.90, 2606:4700:4400::6812:28a6, 2606:4700:4400::ac40:935a
Response IP 172.64.147.90
Found Yes
Hash 84b7d5ba493bb5b5627cf318df4541b35d25bfcd05ed50a6682879c494f1a634
SimHash e1542b780472

Groups

mauibot

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

claudebot

Rule Path
Disallow /

*

Rule Path
Allow /a/feeds/
Disallow /3910
Disallow /a/
Disallow /account/
Disallow /Account/
Disallow /admin
Disallow /Admin
Disallow /amptest/
Disallow /art/
Disallow /Art/
Disallow /auth/
Disallow /betaboards/
Disallow /cdn-cgi/
Disallow /Common/
Disallow /dubs/
Disallow /ecap/
Disallow /ext-content/
Disallow /external/
Disallow /feeds
Disallow /Feeds
Disallow /fool/free-report
Disallow /foolpics
Disallow /foolPics
Disallow /FoolPics
Disallow /free-report
Disallow /help
Disallow /Help
Disallow /includes
Disallow /Includes
Disallow /investing/businesswire/
Disallow /investing/fiercemarkets/
Disallow /investing/FierceMarkets/
Disallow /investing/stocks/
Disallow /investor-alert/
Disallow /localize
Disallow /Localize
Disallow /login/
Disallow /mailemergency
Disallow /mailEmergency
Disallow /MailEmergency
Disallow /Marketing/EcapSurvey.aspx
Disallow /mms
Disallow /news/commentary/
Disallow /news/xt
Disallow /News/Xt
Disallow /News/XT
Disallow /newsletters/*
Disallow /nexus/
Disallow /now50/
Disallow /offers/
Disallow /order-results*
Disallow /order/
Disallow /p/*
Disallow /p/60/
Disallow /partners
Disallow /Partners
Disallow /pegulator
Disallow /Pegulator
Disallow /premium-reports
Disallow /private
Disallow /Private
Disallow /quote/failed-lookup/
Disallow /quote/unknown/
Disallow /reports
Disallow /scripts
Disallow /Scripts
Disallow /search
Disallow /Search
Disallow /secure/
Disallow /server
Disallow /Server
Disallow /shop/
Disallow /Shop/Download/Event/
Disallow /Shop/OrderResults.aspx
Disallow /Shop/OrderResults.aspx*
Disallow /Shop/OrderResults*
Disallow /snap
Disallow /Snap
Disallow /stock-ranker/
Disallow /test
Disallow /Test
Disallow /thank-you/
Disallow /the-ascent/search/
Disallow /video-alert
Disallow /vip/

Other Records

Field Value
sitemap https://www.fool.com/sitemap/
sitemap https://www.fool.com/wag-sitemap.xml
sitemap https://www.fool.com/money/sitemap.xml
sitemap https://www.fool.com/hreflang-tag-sitemap.xml

Comments

  • /robots.txt file for http://www.fool.com/ (prod)
  • MauiBot
  • ChatGPT
  • else