liefdevolle.date
robots.txt

Robots Exclusion Standard data for liefdevolle.date

Resource Scan

Scan Details

Site Domain liefdevolle.date
Base Domain liefdevolle.date
Scan Status Ok
Last Scan2025-06-18T05:45:20+00:00
Next Scan 2025-07-18T05:45:20+00:00

Last Scan

Scanned2025-06-18T05:45:20+00:00
URL https://liefdevolle.date/robots.txt
Redirect https://www.liefdevolle.date/robots.txt
Redirect Domain www.liefdevolle.date
Redirect Base liefdevolle.date
Domain IPs 172.104.136.162
Redirect IPs 172.104.136.162
Response IP 172.104.136.162
Found Yes
Hash 6c170e49b71fbb5870c8a53f69061bbc0986711c1d23b097a132caa365c4886d
SimHash 2051d821c091

Groups

*
*

Rule Path
Disallow /api
Disallow /inbox
Disallow /zoeken/*
Disallow /zoeken?*

baiduspider

Rule Path
Disallow /

fasterfox

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

teracent-feed-processing

Rule Path
Disallow /

ingrid

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20

jyxobot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

shopwiki

Rule Path
Disallow /

cazoodlebot

Rule Path
Disallow /

msnbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20

slurp

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20

yahoo! slurp

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20

rogerbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 2

Other Records

Field Value
sitemap https://www.liefdevolle.date/sitemap.xml