guncel-egitim.org
robots.txt

Robots Exclusion Standard data for guncel-egitim.org

Resource Scan

Scan Details

Site Domain guncel-egitim.org
Base Domain guncel-egitim.org
Scan Status Ok
Last Scan2024-10-31T03:28:02+00:00
Next Scan 2024-11-07T03:28:02+00:00

Last Scan

Scanned2024-10-31T03:28:02+00:00
URL https://guncel-egitim.org/robots.txt
Domain IPs 37.230.106.119
Response IP 37.230.106.119
Found Yes
Hash 3df377273e3de0ebf3cbd047ebaa6b7fff76feaec0cdaed431644ceec8c7e15c
SimHash e564de347593

Groups

*

Rule Path
Disallow /wp-admin
Disallow /cgi-bin/
Disallow /stats/
Disallow /feed/
Disallow /wp-includes/
Disallow /*ref%3D*
Disallow /*?ref=*
Disallow /?ref=*
Allow /wp-includes/*.js
Allow /wp-includes/*.css
Allow /wp-includes/images/
Allow /wp-admin/admin-ajax.php
Disallow *?replytocom

mediapartners-google

Rule Path
Allow /

adsbot-google

Rule Path
Allow /

googlebot-image

Rule Path
Allow /

googlebot-mobile

Rule Path
Allow /

cuil

Rule Path
Allow /

yandexbot

Rule Path
Allow /

slurp

Rule Path
Allow /

yahoo

Rule Path
Allow /

yahoo! slurp

Rule Path
Allow /

ia_archiver-web.archive.org

Rule Path
Disallow /

aipbot

Rule Path
Disallow /

becomebot

Rule Path
Disallow /

psbot

Rule Path
Disallow /

fast

Rule Path
Disallow /

gigabot

Rule Path
Disallow /

voyager

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

backrub/*.*

Rule Path
Disallow /

grub.org

Rule Path
Disallow /

botrighthere

Rule Path
Disallow /

larbin

Rule Path
Disallow /

walhello appie

Rule Path
Disallow /

python-urllib

Rule Path
Disallow /

cherrypicker

Rule Path
Disallow /

emailcollector

Rule Path
Disallow /

webbandit

Rule Path
Disallow /

emailwolf

Rule Path
Disallow /

copyrightcheck

Rule Path
Disallow /

crescent

Rule Path
Disallow /

Other Records

Field Value
sitemap https://www.guncel-egitim.org/sitemap_index.xml