Log In Sign Up

huseyin-uysal.com
robots.txt

Robots Exclusion Standard data for huseyin-uysal.com

Resource Scan

Scan Details

Site Domain huseyin-uysal.com
Base Domain huseyin-uysal.com
Scan Status Ok
Last Scan2024-11-14T17:09:43+00:00
Next Scan 2024-11-21T17:09:43+00:00

Last Scan

Scanned2024-11-14T17:09:43+00:00
URL https://huseyin-uysal.com/robots.txt
Domain IPs 89.252.187.162
Response IP 89.252.187.162
Found Yes
Hash ca455c9ccf2eaab7f7e9e18d711892e802e63b6df4986dfe7ed34fb418951f6b
SimHash 0554c7b10593

Groups

*

Rule Path
Disallow /wp-admin
Disallow /wp-includes
Disallow /sozluk
Allow /tag

mediapartners-google

Rule Path
Allow /

adsbot-google

Rule Path
Allow /

googlebot-image

Rule Path
Allow /

googlebot-mobile

Rule Path
Allow /

yandexbot

Rule Path
Allow /

twitterbot

Rule Path
Allow /

*

Rule Path
Disallow *?refsite=www.n1ads.com&ref=alexa-traffic-rank

yandex

Rule Path
Allow /

cuil

Rule Path
Allow /

slurp

Rule Path
Allow /

yahoo

Rule Path
Allow /

yahoo! slurp

Rule Path
Allow /

ia_archiver-web.archive.org

Rule Path
Disallow /

aipbot

Rule Path
Disallow /

becomebot

Rule Path
Disallow /

psbot

Rule Path
Disallow /

fast

Rule Path
Disallow /

gigabot

Rule Path
Disallow /

voyager

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

backrub/*.*

Rule Path
Disallow /

grub.org

Rule Path
Disallow /

botrighthere

Rule Path
Disallow /

larbin

Rule Path
Disallow /

psbot

Rule Path
Disallow /

walhello appie

Rule Path
Disallow /

python-urllib

Rule Path
Disallow /

cherrypicker

Rule Path
Disallow /

emailcollector

Rule Path
Disallow /

webbandit

Rule Path
Disallow /

emailwolf

Rule Path
Disallow /

copyrightcheck

Rule Path
Disallow /

crescent

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap https://huseyin-uysal.com/sitemap_index.xml
Back to top

Warnings

  • `host` is not a known field.
Back to top