ilk-okulum.com
robots.txt

Robots Exclusion Standard data for ilk-okulum.com

Resource Scan

Scan Details

Site Domain ilk-okulum.com
Base Domain ilk-okulum.com
Scan Status Ok
Last Scan2024-09-22T21:36:30+00:00
Next Scan 2024-09-29T21:36:30+00:00

Last Scan

Scanned2024-09-22T21:36:30+00:00
URL https://ilk-okulum.com/robots.txt
Domain IPs 94.73.170.151
Response IP 94.73.170.151
Found Yes
Hash b7cb9c7b3dc48df3c33fc7bd2ae5f4629415ae7fadf2cb7fb5aed31ce8644149
SimHash c42d580c63d1

Groups

googlebot

Rule Path
Disallow

bizbot04 kirk.overleaf.com

Rule Path
Disallow

happybot (gserver.kw.net)

Rule Path
Disallow

californiabrownspider

Rule Path
Disallow

ei*net/0.1 libwww/0.1

Rule Path
Disallow

ibot/1.0 libwww-perl/0.40

Rule Path
Disallow

merritt/1.0

Rule Path
Disallow

statfetcher/1.0

Rule Path
Disallow

teachersoft/1.0 libwww/2.17

Rule Path
Disallow

www collector

Rule Path
Disallow

processor/0.0alpha libwww-perl/0.20

Rule Path
Disallow

wobot/1.0 from 206.214.202.45

Rule Path
Disallow

libertech-rover

Rule Path
Disallow

whowhere robot

Rule Path
Disallow

iti spider

Rule Path
Disallow

w3index

Rule Path
Disallow

mycnnspider

Rule Path
Disallow

summycrawler

Rule Path
Disallow

ogspider

Rule Path
Disallow

linklooker

Rule Path
Disallow

cyberspyder

Rule Path
Disallow

slowbot

Rule Path
Disallow

heraspider

Rule Path
Disallow

surfbot

Rule Path
Disallow

bizbot003

Rule Path
Disallow

webwalker

Rule Path
Disallow

sandbot

Rule Path
Disallow

enigmabot

Rule Path
Disallow

spyder3.microsys.com

Rule Path
Disallow

*

Rule Path
Disallow