Log In Sign Up

canliyayinradyolar.com
robots.txt

Robots Exclusion Standard data for canliyayinradyolar.com

Resource Scan

Scan Details

Site Domain canliyayinradyolar.com
Base Domain canliyayinradyolar.com
Scan Status Ok
Last Scan2024-09-25T02:21:21+00:00
Next Scan 2024-10-02T02:21:21+00:00

Last Scan

Scanned2024-09-25T02:21:21+00:00
URL https://canliyayinradyolar.com/robots.txt
Redirect https://www.canliradyodinle.fm/robots.txt
Redirect Domain www.canliradyodinle.fm
Redirect Base canliradyodinle.fm
Domain IPs 104.21.6.86, 172.67.154.209, 2606:4700:3031::6815:656, 2606:4700:3036::ac43:9ad1
Redirect IPs 104.26.4.23, 104.26.5.23, 172.67.75.173, 2606:4700:20::681a:417, 2606:4700:20::681a:517, 2606:4700:20::ac43:4bad
Response IP 104.26.4.23
Found Yes
Hash f0995bbb1efd1f298c403aaf0a01b24eac75685dea035b7392f029a7f7bc37a8
SimHash ee67cc301ca2

Groups

*

Rule Path
Allow /
Allow /wp-admin/admin-ajax.php
Disallow /wp-json/
Disallow /live/
Disallow /wp-admin/
Disallow /wp-content/themes/canliradyodinle/
Disallow /wp-content/themes/mobile/
Disallow /canli-yayin/
Disallow /canliradyolar/
Disallow /*.php*$
Disallow /listener-*$
Disallow /listen-*$
Disallow /*ref%3D*
Disallow /*?ref=
Disallow /*?s=*
Disallow /search/*
Disallow /mini-mod*
Disallow /cdn-cgi/bm/cv/
Disallow /cdn-cgi/challenge-platform/

nuclei
wikido
riddler
petalbot
zoominfobot
go-http-client
node/simplecrawler
cazoodlebot
dotbot/1.0
gigabot
barkrowler
blexbot
magpie-crawler

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap https://www.canliradyodinle.fm/sitemap_index.xml
Back to top

Comments

  • Global rules
  • -----------------
  • Disallow
  • -----------------
  • Prevent crawling CF challenge URLs
  • Sitemap
  • -----------------
  • Ban bots that don't benefit us.
  • --------------------------------
Back to top