tech.sportskeeda.com
robots.txt

Robots Exclusion Standard data for tech.sportskeeda.com

Resource Scan

Scan Details

Site Domain tech.sportskeeda.com
Base Domain sportskeeda.com
Scan Status Ok
Last Scan2026-03-25T02:20:55+00:00
Next Scan 2026-04-08T02:20:55+00:00

Last Scan

Scanned2026-03-25T02:20:55+00:00
URL https://tech.sportskeeda.com/robots.txt
Domain IPs 13.35.202.113, 13.35.202.116, 13.35.202.85, 13.35.202.92
Response IP 13.35.202.92
Found Yes
Hash 0e2e19deb60d5e6291bd00bb2a42310df9cfe5306da9789d0757dcedab7ecb57
SimHash eab0495498df

Groups

mediapartners-google

Rule Path
Disallow

grapeshot

Rule Path
Disallow

peer39_crawler
peer39_crawler/1.0

Rule Path
Disallow

audigentadbot

Rule Path
Disallow

duggmirror

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

perplexitybot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

aliyunsecbot

Rule Path
Disallow /

anthropic-ai

Rule Path
Disallow /

applebot-extended

Rule Path
Disallow /

archive.org_bot

Rule Path
Disallow /

archivebot

Rule Path
Disallow /

arquivo-web-crawler

Rule Path
Disallow /

awariorssbot

Rule Path
Disallow /

awariosmartbot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

claudebot

Rule Path
Disallow /

claude-searchbot

Rule Path
Disallow /

claude-user

Rule Path
Disallow /

claude-web

Rule Path
Disallow /

cohere-ai

Rule Path
Disallow /

cohere-training-data-crawler

Rule Path
Disallow /

dataforseobot

Rule Path
Disallow /

diffbot

Rule Path
Disallow /

duckassistbot

Rule Path
Disallow /

echoboxbot

Rule Path
Disallow /

europarchive.org

Rule Path
Disallow /

facebookbot

Rule Path
Disallow /

friendlycrawler

Rule Path
Disallow /

heritrix

Rule Path
Disallow /

ia_archiver

Rule Path
Disallow /

ia_archiver-web.archive.org

Rule Path
Disallow /

imagesiftbot

Rule Path
Disallow /

jetslide

Rule Path
Disallow /

linkarchiver

Rule Path
Disallow /

magpie-crawler

Rule Path
Disallow /

meta-externalagent
meta-externalagent

Rule Path
Disallow /

meta-externalfetcher
meta-externalfetcher

Rule Path
Disallow /

mycentralaiscraperbot

Rule Path
Disallow /

news-please

Rule Path
Disallow /

newsnow

Rule Path
Disallow /

nicecrawler

Rule Path
Disallow /

omgili

Rule Path
Disallow /

omgilibot

Rule Path
Disallow /

poseidon research crawler

Rule Path
Disallow /

quillbot.com

Rule Path
Disallow /

quora-bot

Rule Path
Disallow /

primalbot

Rule Path
Disallow /

scrapy

Rule Path
Disallow /

seekrbot

Rule Path
Disallow /

seznamhomepagecrawler

Rule Path
Disallow /

taragroup intelligent bot

Rule Path
Disallow /

timpibot

Rule Path
Disallow /

turnitinbot

Rule Path
Disallow /

viennatinybot

Rule Path
Disallow /

web-archive-net.com.bot

Rule Path
Disallow /

yandexbot

Rule Path
Disallow /

youbot

Rule Path
Disallow /

facebookexternalhit

Rule Path
Allow /*?*smid=

googlebot

Rule Path
Allow *key4%3Dskwfb*
Allow *key4%3Dskcolbasketfb*
Allow *key4%3Dskaewfb*
Allow *key4%3Dskbaseballfb*
Allow *key4%3Dskcfb*
Allow *key4%3Dskchefb*
Allow *key4%3Dskf1fb*
Allow *key4%3Dskmaincfb*
Allow *key4%3Dskcfb*
Allow *key4%3Dskfootballfb*
Allow *key4%3Dskgolffb*
Allow *key4%3Dskmmafb*
Allow *key4%3Dsknascarfb*
Allow *key4%3Dsknbamemesfb*
Allow *key4%3Dskpffb*
Allow *key4%3Dsktenfb*
Allow *key4%3Dskufcfb*
Disallow /001/
Disallow /amp/*
Disallow /profile/*
Disallow /football-match-preview/
Disallow /olympics-page-2/
Disallow /apc/
Disallow /cgi-bin
Disallow /wp-admin/
Disallow /wp-includes/
Disallow /readme.html
Disallow /license.txt
Disallow /search/
Disallow /wp-content/plugins/
Disallow /wp-content/cache/
Disallow /wp-content/themes/
Disallow /members
Disallow /archives
Disallow /?s=
Disallow /?p=
Disallow /wp-content/uploads/
Disallow /?attachment_id=
Disallow /?author=
Disallow /uc/
Disallow /quiz/
Disallow /redirect
Disallow /viral-picks/*
Disallow /taxonomy
Disallow /comments/count/*
Disallow /*/load-more
Disallow /*/load_more
Disallow /*/ajax/archive
Disallow /*/ajax/related
Disallow /*/ajax/trending
Disallow /cricket/ajax/teams/related/*
Disallow /admin
Disallow /admin/*
Disallow /*/ajax/archive?page
Disallow /login?after-login*
Disallow /login?*
Disallow /third-party/*
Disallow /version
Disallow *?key1=*
Disallow *%26key1%3D*
Disallow *?key2=*
Disallow *%26key2%3D*
Disallow *?key3=*
Disallow *%26key3%3D*
Disallow *?key4=*
Disallow *%26key4%3D*
Disallow *?givingPlayers=*
Disallow *%26givingPlayers%3D*
Disallow *?gettingPlayers=*
Disallow *%26gettingPlayers%3D*
Disallow *?ref=*
Disallow *%26ref%3D*
Disallow *?article-trending=*
Disallow *%26article-trending%3D*
Disallow *?page=*
Disallow *%26page%3D*
Disallow *?date=*
Disallow *%26date%3D*
Disallow *?comment-popup=*
Disallow *%26comment-popup%3D*
Disallow *?comment-meta-id=*
Disallow *%26comment-meta-id%3D*
Disallow *?key=*
Disallow *%26key%3D*
Disallow *?Key2=*
Disallow *%26Key2%3D*
Disallow *?Key4=*
Disallow *%26Key4%3D*
Disallow *?tab=*
Disallow *%26tab%3D*
Disallow *?related=*
Disallow *%26related%3D*
Disallow *?view=*
Disallow *%26view%3D*
Disallow *?traffic=*
Disallow *%26traffic%3D*
Disallow *?after-login=*
Disallow *%26after-login%3D*
Disallow /edit-profile
Disallow /login
Disallow *?team_stats_event_type=*
Disallow *%26team_stats_event_type%3D*
Disallow *?player_stats_event_type=*
Disallow *%26player_stats_event_type%3D*
Disallow /creator/*

Other Records

Field Value
sitemap https://tech.sportskeeda.com/news-sitemap.xml
sitemap https://tech.sportskeeda.com/sitemap.xml

Comments

  • allow adsense bot to parse no-index content
  • Disallow the AI chat bots to crawl the pages
  • disallow folders
  • These query params appear from the links getting indexed/crawled from the Facebook