pool.tvasports.ca
robots.txt

Robots Exclusion Standard data for pool.tvasports.ca

Resource Scan

Scan Details

Site Domain pool.tvasports.ca
Base Domain tvasports.ca
Scan Status Ok
Last Scan2024-06-21T13:36:25+00:00
Next Scan 2024-06-28T13:36:25+00:00

Last Scan

Scanned2024-06-21T13:36:25+00:00
URL http://pool.tvasports.ca/robots.txt
Redirect https://www.poolexpert.com/robots.txt
Redirect Domain www.poolexpert.com
Redirect Base poolexpert.com
Domain IPs 52.7.64.51, 52.87.143.246
Redirect IPs 52.7.64.51, 52.87.143.246
Response IP 52.7.64.51
Found Yes
Hash 9dc9536a8a611b97a16756a64cd810511fd9b2978adaf6eee42b45aebdd77e14
SimHash 6104c7f17d67

Groups

*

Rule Path
Allow /ads.txt
Allow /app-ads.txt
Disallow /

Other Records

Field Value
crawl-delay 600

duckduckbot
duckduck
bingbot
googlebot
googlebot-image
googlebot-news
googlebot-video
googlebot-mobile
adsbot-google
mediapartners-google
storebot-google
google-inspectiontool
googleother
google-extended
adsbot-google-mobile
adidxbot
bingpreview
msnbot
msnbot-media
slurp
yahoo-blogs
yahoo-mmcrawler
yandex
yandexbot
twitterbot
facebookexternalhit
proximic
mj12
mj12bot
semrush
semrushbot
dot
dotbot
applebot
linkedinbot
pinterestbot
facebookbot
facebookcatalog
amazon-advertising-ad-standards-bot

Rule Path
Disallow /*.less
Disallow /admin/
Disallow /bin/
Disallow /cache/
Disallow /cache
Disallow /change_lang.aspx
Disallow /classes/
Disallow /cls/
Disallow /components/
Disallow /constants/
Disallow /controls/
Disallow /cookie_sync
Disallow /en/change_lang.aspx
Disallow /en/facebookLink.aspx
Disallow /en/facebookLogin.aspx
Disallow /en/facebookPost.aspx
Disallow /en/facebookUnlink.aspx
Disallow /en/gpoollogin.aspx
Disallow /en/gpoollogout.aspx
Disallow /en/home_wl.aspx
Disallow /en/empty.aspx
Disallow /en/maintenance.aspx
Disallow /en/newsref.aspx
Disallow /en/parallel.aspx
Disallow /en/userlogin.aspx
Disallow /en/userlogout.aspx
Disallow /facebookLink.aspx
Disallow /facebookLogin.aspx
Disallow /facebookPost.aspx
Disallow /facebookredirect.axd
Disallow /facebookUnlink.aspx
Disallow /fr/change_lang.aspx
Disallow /fr/facebookLink.aspx
Disallow /fr/facebookLogin.aspx
Disallow /fr/facebookPost.aspx
Disallow /fr/facebookUnlink.aspx
Disallow /fr/gpoollogin.aspx
Disallow /fr/gpoollogout.aspx
Disallow /fr/home_wl.aspx
Disallow /fr/empty.aspx
Disallow /fr/maintenance.aspx
Disallow /fr/newsref.aspx
Disallow /fr/parallel.aspx
Disallow /fr/userlogin.aspx
Disallow /fr/userlogout.aspx
Disallow /gpoollogin.aspx
Disallow /gpoollogout.aspx
Disallow /home_wl.aspx
Disallow /empty.aspx
Disallow /includes/
Disallow /maintenance.aspx
Disallow /newsref.aspx
Disallow /openrtb2/auction
Disallow /parallel.aspx
Disallow /request.ashx
Disallow /resources/
Disallow /Telerik.RadUploadProgressHandler.ashx
Disallow /Telerik.Web.UI.DialogHandler.aspx
Disallow /therest.ashx
Disallow /userlogin.aspx
Disallow /userlogout.aspx
Disallow /v01/
Disallow /v02/
Disallow /v03/

Other Records

Field Value
crawl-delay 5

Other Records

Field Value
sitemap https://www.poolexpert.com/sitemaps/sitemap-index.xml

Comments

  • disallow all
  • but allow only important bots
  • Disallow: /ChartImage.axd
  • Disallow: /client.ashx
  • Disallow: /en/socialshare.aspx
  • Disallow: /fr/socialshare.aspx
  • Disallow: /ScriptResource.axd
  • Disallow: /socialshare.aspx
  • Disallow: /Telerik.Web.UI.SpellCheckHandler.axd
  • Disallow: /Telerik.Web.UI.WebResource.axd
  • Disallow: /WebResource.axd