telerik.com
robots.txt

Robots Exclusion Standard data for telerik.com

Resource Scan

Scan Details

Site Domain telerik.com
Base Domain telerik.com
Scan Status Ok
Last Scan2024-06-13T15:22:28+00:00
Next Scan 2024-06-27T15:22:28+00:00

Last Scan

Scanned2024-06-13T15:22:28+00:00
URL https://telerik.com/robots.txt
Redirect https://www.telerik.com/robots.txt
Redirect Domain www.telerik.com
Redirect Base telerik.com
Domain IPs 50.56.19.112
Redirect IPs 50.56.19.112
Response IP 50.56.19.112
Found Yes
Hash 1da5987ee178a3bd97844743fd404619d736ba1263601d2042b6c81b97d6fada
SimHash 65374fb64684

Groups

*
googlebot
googlebot-news
googlebot-image
googlebot-video
googlebot-mobile
adsbot-google

Rule Path
Disallow /ArticleFileDownload.aspx
Disallow /registration-login/
Disallow /search
Disallow /account/default.aspx
Disallow /t3mp0mAt/
Disallow /help/search.aspx
Disallow /*aspxerrorpath*
Disallow /login/v2*
Disallow /account*
Disallow *tagId*
Disallow /api/v1/*
Disallow /forums/profile/*
Disallow *answered%3D*

bingbot
msnbot
msnbot-media
adidxbot

Rule Path
Disallow /ArticleFileDownload.aspx
Disallow /registration-login/
Disallow /search
Disallow /account/default.aspx
Disallow /t3mp0mAt/
Disallow /help/search.aspx
Disallow /*aspxerrorpath*
Disallow /login/v2*
Disallow /account*
Disallow *tagId*
Disallow /api/v1/*
Disallow /forums/profile/*
Disallow *answered%3D*

yandex bot

Rule Path
Disallow /ArticleFileDownload.aspx
Disallow /registration-login/
Disallow /search
Disallow /account/default.aspx
Disallow /t3mp0mAt/
Disallow /help/search.aspx
Disallow /*aspxerrorpath*
Disallow /login/v2*
Disallow /account*
Disallow *tagId*
Disallow /api/v1/*
Disallow /forums/profile/*
Disallow *answered%3D*

Other Records

Field Value
crawl-delay 10

slurp

Rule Path
Disallow /ArticleFileDownload.aspx
Disallow /registration-login/
Disallow /search
Disallow /account/default.aspx
Disallow /t3mp0mAt/
Disallow /help/search.aspx
Disallow /*aspxerrorpath*
Disallow /login/v2*
Disallow /account*
Disallow *tagId*
Disallow /api/v1/*
Disallow /forums/profile/*
Disallow *answered%3D*

Other Records

Field Value
crawl-delay 10

baiduspider
baiduspider-image
baiduspider-mobile
baiduspider-news
baiduspider-video

Rule Path
Disallow /ArticleFileDownload.aspx
Disallow /registration-login/
Disallow /search
Disallow /account/default.aspx
Disallow /t3mp0mAt/
Disallow /help/search.aspx
Disallow /*aspxerrorpath*
Disallow /login/v2*
Disallow /account*
Disallow *tagId*
Disallow /api/v1/*
Disallow /forums/profile/*
Disallow *answered%3D*

facebookexternalhit/1.1 (+http://www.facebook.com/externalhit_uatext.php)

Rule Path
Disallow /ArticleFileDownload.aspx
Disallow /registration-login/
Disallow /search
Disallow /account/default.aspx
Disallow /t3mp0mAt/
Disallow /help/search.aspx
Disallow /*aspxerrorpath*
Disallow /login/v2*
Disallow /account*
Disallow *tagId*
Disallow /api/v1/*
Disallow /forums/profile/*
Disallow *answered%3D*

Other Records

Field Value
sitemap https://www.telerik.com/sitemap.xml
sitemap https://www.telerik.com/sitemap_images.xml
sitemap https://www.telerik.com/sitemap_video.xml

Comments

  • All robots will spider the domain
  • Image Sitemap
  • Video Sitemap