Log In Sign Up

tarihtebugun.org
robots.txt

Robots Exclusion Standard data for tarihtebugun.org

Resource Scan

Scan Details

Site Domain tarihtebugun.org
Base Domain tarihtebugun.org
Scan Status Ok
Last Scan2025-10-27T08:54:31+00:00
Next Scan 2025-11-03T08:54:31+00:00

Last Scan

Scanned2025-10-27T08:54:31+00:00
URL https://www.tarihtebugun.org/robots.txt
Domain IPs 192.99.229.89
Response IP 192.99.229.89
Found Yes
Hash 34d4d1a078de9b6ccf990cb8a471011b484bb6e45a6b2e4541b3206fc1108de2
SimHash 99c5fdd24bf7

Groups

crawler4j

Rule Path
Disallow /

httrack

Rule Path
Disallow /

netcaptor

Rule Path
Disallow /

offline explorer

Rule Path
Disallow /

spiderku/0.9

Rule Path
Disallow /

steeler

Rule Path
Disallow /

webcopier v3.3

Rule Path
Disallow /

webcopier v3.2a

Rule Path
Disallow /

webcopier

Rule Path
Disallow /

webcrawler

Rule Path
Disallow /

web downloader/4.9

Rule Path
Disallow /

web downloader/5.8

Rule Path
Disallow /

webgather 3.0

Rule Path
Disallow /

webstripper/2.56

Rule Path
Disallow /

webzip/3.65

Rule Path
Disallow /

webzip

Rule Path
Disallow /

wget

Rule Path
Disallow /

zao

Rule Path
Disallow /

zeus 2.6

Rule Path
Disallow /

teleport

Rule Path
Disallow /

teleportpro

Rule Path
Disallow /

baiduspider+

Rule Path
Disallow /

*

Rule Path
Allow /
Disallow /?ref=
Disallow /?sort=
Disallow /arama.php?
Disallow /hesap_numaralari.php
Disallow /hesap_numaralari.html
Disallow /tbkey.php?
Disallow /yapim-asamasinda.html
Back to top

Comments

  • User-agent: Googlebot
  • Allow: /bidibidi_design/bidibidi_js*
  • Allow: /bidibidi_design/bidibidi_template*
  • Allow: /bidibidi_design/fot*
  • Allow: /bidibidi_design/bidibidi_template/bidibidi_tarihtebugun/bidibidi_css/*
  • Allow: /bidibidi_design/fot*
  • Disallow: /vendor/
  • Disallow: /css/
  • Disallow: /js/
Back to top