moly.hu
robots.txt

Robots Exclusion Standard data for moly.hu

Resource Scan

Scan Details

Site Domain moly.hu
Base Domain moly.hu
Scan Status Ok
Last Scan2024-05-17T03:26:45+00:00
Next Scan 2024-05-24T03:26:45+00:00

Last Scan

Scanned2024-05-17T03:26:45+00:00
URL https://moly.hu/robots.txt
Domain IPs 91.250.86.152
Response IP 91.250.86.152
Found Yes
Hash 98312ed886935a9d1a2dbe372d9dbd3d3e67499ed85011a26dae5e8fcfbf28a1
SimHash f20c8be08c92

Groups

*

Rule Path
Disallow /
Allow /$
Allow /sitemap*
Allow /belepes$
Allow /ads.txt
Allow /app-ads.txt
Allow /manifest.json
Allow /alkotok/*
Disallow /alkotok/*/
Allow /alkotok/*/eletmu$
Allow /alkotok/*/snitt$
Allow /alkotok/*/wikipedia$
Allow /alkotok/*/wikipedia-angol$
Disallow /alkotok/uj$
Allow /konyvek/*
Disallow /konyvek/*/
Disallow /konyvek/uj$
Disallow /konyvek/osszehasonlitas$
Allow /enciklopedia/*
Disallow /enciklopedia/*/
Disallow /enciklopedia/uj$
Allow /sorozatok/*
Disallow /sorozatok/*/
Disallow /sorozatok/uj$
Allow /kiadok/*
Disallow /kiadok/*/
Disallow /kiadok/uj$
Allow /tagok/*
Disallow /tagok/*/
Disallow /tagok/uj$
Allow /bejegyzesek/*
Disallow /bejegyzesek/*/
Disallow /bejegyzesek/uj$

mediapartners-google

Rule Path
Allow /

googlebot-image

Rule Path
Allow /

grapeshot

Rule Path
Allow /

facebookexternalhit

Rule Path
Allow /

criteobot

Rule Path
Allow /

ahrefsbot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

megaindex

Rule Path
Disallow /

dataforseobot

Rule Path
Disallow /

proximic

Rule Path
Disallow /

yandex

Rule Path
Disallow /

seokicks

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

oszkbot

Rule Path
Disallow /

leikibot

Rule Path
Disallow /

sentibot

Rule Path
Disallow /

qwarrybot

Rule Path
Disallow /

amazonadbot

Rule Path
Disallow /

yandexmobilebot

Rule Path
Disallow /

yandexaccessibilitybot

Rule Path
Disallow /

seekportbot

Rule Path
Disallow /

neticle

Rule Path
Disallow /

trendictionbot

Rule Path
Disallow /

cincraw

Rule Path
Disallow /

seznambot

Rule Path
Disallow /

mail.ru_bot

Rule Path
Disallow /

hyscore

Rule Path
Disallow /

uipbot

Rule Path
Disallow /

amazonbot

Rule Path
Disallow /

bsbot

Rule Path
Disallow /

domainstatsbot

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

coccocbot-web

Rule Path
Disallow /

buck

Rule Path
Disallow /

timpibot

Rule Path
Disallow /

claudebot

Rule Path
Disallow /

Other Records

Field Value
sitemap https://moly.hu/sitemap.xml.gz

Comments

  • Crawl-delay: 1