dejongeturken.com
robots.txt

Robots Exclusion Standard data for dejongeturken.com

Resource Scan

Scan Details

Site Domain dejongeturken.com
Base Domain dejongeturken.com
Scan Status Ok
Last Scan2026-02-05T08:19:12+00:00
Next Scan 2026-02-12T08:19:12+00:00

Last Scan

Scanned2026-02-05T08:19:12+00:00
URL https://dejongeturken.com/robots.txt
Domain IPs 2a02:2350:5:10a:91:b9a1:afc4:371e, 46.30.213.18
Response IP 46.30.213.18
Found Yes
Hash ba358e9e9883a185f03c20ab57f02bc1abb6b90351fe39ba5573459d9a770044
SimHash 481cc0e2e31a

Groups

*

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

ahrefsbot

Rule Path
Disallow /

whatsapp

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

claudebot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

aspiegelbot

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

applebot

Rule Path
Disallow /

amazonbot

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

yandex.com/bots

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

yandexbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

googleother

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

googlebot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

googlebot-image

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

googlebot-video

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10
crawl-delay 10

Warnings

  • 1 invalid line.