Log In Sign Up

canliradyodinle.fm
robots.txt

Robots Exclusion Standard data for canliradyodinle.fm

Resource Scan

Scan Details

Site Domain canliradyodinle.fm
Base Domain canliradyodinle.fm
Scan Status Ok
Last Scan2024-09-22T14:10:08+00:00
Next Scan 2024-09-29T14:10:08+00:00

Last Scan

Scanned2024-09-22T14:10:08+00:00
URL https://canliradyodinle.fm/robots.txt
Domain IPs 104.26.4.23, 104.26.5.23, 172.67.75.173, 2606:4700:20::681a:417, 2606:4700:20::681a:517, 2606:4700:20::ac43:4bad
Response IP 172.67.75.173
Found Yes
Hash f0995bbb1efd1f298c403aaf0a01b24eac75685dea035b7392f029a7f7bc37a8
SimHash ee67cc301ca2

Groups

*

Rule Path
Allow /
Allow /wp-admin/admin-ajax.php
Disallow /wp-json/
Disallow /live/
Disallow /wp-admin/
Disallow /wp-content/themes/canliradyodinle/
Disallow /wp-content/themes/mobile/
Disallow /canli-yayin/
Disallow /canliradyolar/
Disallow /*.php*$
Disallow /listener-*$
Disallow /listen-*$
Disallow /*ref%3D*
Disallow /*?ref=
Disallow /*?s=*
Disallow /search/*
Disallow /mini-mod*
Disallow /cdn-cgi/bm/cv/
Disallow /cdn-cgi/challenge-platform/

nuclei
wikido
riddler
petalbot
zoominfobot
go-http-client
node/simplecrawler
cazoodlebot
dotbot/1.0
gigabot
barkrowler
blexbot
magpie-crawler

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap https://www.canliradyodinle.fm/sitemap_index.xml
Back to top

Comments

  • Global rules
  • -----------------
  • Disallow
  • -----------------
  • Prevent crawling CF challenge URLs
  • Sitemap
  • -----------------
  • Ban bots that don't benefit us.
  • --------------------------------
Back to top