codesports.com.au
robots.txt
Robots Exclusion Standard data for codesports.com.au
Resource Scan
Scan Details
| Site Domain | codesports.com.au |
| Base Domain | codesports.com.au |
| Scan Status | Ok |
| Last Scan | 2025-11-28T06:50:10+00:00 |
| Next Scan | 2025-12-12T06:50:10+00:00 |
Last Scan
| Scanned | 2025-11-28T06:50:10+00:00 |
| URL | https://codesports.com.au/robots.txt |
| Redirect | https://www.codesports.com.au/robots.txt |
| Redirect Domain | www.codesports.com.au |
| Redirect Base | codesports.com.au |
| Domain IPs | 23.41.38.116, 2600:1413:5000:1185::372b, 2600:1413:5000:1187::372b |
| Redirect IPs | 23.41.38.116, 2600:1413:5000:1185::372b, 2600:1413:5000:1187::372b |
| Response IP | 23.75.213.185 |
| Found | Yes |
| Hash | 9973c9d6fbd1d7fa85c068a6b5d516261031dfb9d7b39e70e8284c36a34b0876 |
| SimHash | 3946573086bd |
Groups
screamingfrogseospider
comscorecrawler
moatbot
adbeat
integralads.com
expo9
outbrain
semasio.net
weborama-fetcher
yahooadmonitoring
netseer.com
semrushbot
omgili.com
facebookcrawler
facebookexternalhit
facebot
ias_crawler
ias_wombles
iasbot
identrics
meltwaternews
links.streem.com.au
webz-bot
linkedinbot
whatsapp
redditbot
mediapartners-google
google-inspectiontool
adsbot-google
storebot-google
adsbot-google-mobile
adsbot-google-mobile-apps
apis-google
google web light
google-ads-creatives-assistant
google-adwords
google-amphtml
google-cloudvertexbot
google-play-newsstand
google-read-aloud
google-safety
google-site-verification
google-structured-data-testing-tool
googleimageproxy
twitterbot
applebot
applebot-aggregator
applenewsbot
adidxbot
admantx
admantx-usbatch/3.1
ahc
ahc/2.1
ahrefsbot
amazonadbot
arquivo.pt
bidswitchbot/1.0
bnf.fr_bot
botify
checkmy
chrome-lighthouse
cognitiveseo
contentking
criteobot/0.1
datadog-synthetics
deepcrawl
dlvrit
echoboxbot/1.0
flipboard
grapeshotcrawler
verity
hyscore/1.0
ia_archiver
iframely
jetoctopus
labne
leikibot
lumarbot
mj12bot
newrelicpinger
okta
parselybot
peer39_crawler/1.0
photon
pingdom
pinterestbot
proofpoint
proximic
pubmatic
rogerbot
rytebot
schema-validator
semantic-scholar
sentry
serankingbot
similarwebbot
sirdatabot
sistrix
site24x7
sitebulb
siteimprovebot
slack-imgproxy
slackbot
slackbot-linkexpanding
skypeuripreview
smartologybot
snap url preview service
snapchat
socialflow
statuscake
taboolabot
telegrambot
thetradedesk
uptimerobot
yahoomailproxy
yahoo-linkpreview
| Rule | Path |
|---|---|
| Disallow |
googlebot
googlebot-image
googlebot-news
googlebot-video
googleother
googleother-image
googleother-video
bingbot
bingpreview
microsoftpreview
msnbot
duckduckbot
slurp
yahoo
y!j-bot
baiduspider
mojeekbot
petalbot
qwantbot
seznambot
sogou
yandex
yandexbot
yandeximages
yeti
yisouspider
| Rule | Path |
|---|---|
| Disallow | /*/comments-* |
| Disallow | /*/print/ |
| Disallow | /*SIT* |
| Disallow | /*.swf |
| Disallow | /printpage/ |
| Disallow | /404 |
| Disallow | /akam/ |
| Disallow | /sport/superracing/form-guide |
| Disallow | /sport/superracing/form-guide/horse-statistics |
| Disallow | /sport/superracing/form-guide/jockey-statistics |
| Disallow | /sport/superracing/form-guide/trainer-statistics |
| Disallow | /sport/superracing/form-guide/races |
| Disallow | /sport/superracing/form-guide/race-card |
| Disallow | /ipad/stories |
| Disallow | /archive/ |
| Disallow | /business-unify/ |
| Disallow | /classifieds-unify/ |
| Disallow | /entertainment-unify/ |
| Disallow | /help-unify/ |
| Disallow | /home-unify-an/ |
| Disallow | /money-unify/ |
| Disallow | /news-unify/ |
| Disallow | /pagenotfound-unify/ |
| Disallow | /realestate-unify/ |
| Disallow | /sport-unify/ |
| Disallow | /technology-unify/ |
| Disallow | /travel-unify/ |
| Disallow | /nocookies-unify/ |
| Disallow | /shop-unify/ |
| Disallow | /activity-centre/ |
| Disallow | /it-test-only/ |
| Disallow | /enewsletters/ |
*
| Rule | Path |
|---|---|
| Allow | /ads.txt |
| Allow | /app-ads.txt |
| Disallow | / |
Other Records
| Field | Value |
|---|---|
| sitemap | https://www.codesports.com.au/sitemap.xml |
| sitemap | https://www.codesports.com.au/news-sitemap.xml |
Warnings
- 2 invalid lines.
Comments