blog.geld.nl
robots.txt

Robots Exclusion Standard data for blog.geld.nl

Archived Snapshots

Resource Scan

Scan Details

Site Domain	blog.geld.nl
Base Domain	geld.nl
Scan Status	Ok
Last Scan	2024-10-22T08:44:06+00:00
Next Scan	2024-11-21T08:44:06+00:00

Last Scan

Scanned	2024-10-22T08:44:06+00:00
URL	https://blog.geld.nl/robots.txt
Domain IPs	2001:16e8:0:211::100, 62.148.167.229
Response IP	62.148.167.229
Found	Yes
Hash	ee532a2094c67759977d834df4374aadd085e222597d9e52d2b6e2da1c782376
SimHash	b31c6d408cb0

Groups

*

Rule	Path
Disallow

Rule

Path

Disallow

rogerbot

Rule	Path
Disallow	/

Rule

Path

Disallow

ezooms

Rule	Path
Disallow	/

Rule

Path

Disallow

webcrawler

Rule	Path
Disallow	/

Rule

Path

Disallow

bubing

Rule	Path
Disallow	/

Rule

Path

Disallow

proximic

Rule	Path
Disallow	/

Rule

Path

Disallow

trendictionbot

Rule	Path
Disallow	/

Rule

Path

Disallow

vagabondo

Rule	Path
Disallow	/

Rule

Path

Disallow

bilbo

Rule	Path
Disallow	/

Rule

Path

Disallow

linkdexbot

Rule	Path
Disallow	/

Rule

Path

Disallow

exabot

Rule	Path
Disallow	/

Rule

Path

Disallow

slurp

Rule	Path
Disallow	/

Rule

Path

Disallow

niki-bot

Rule	Path
Disallow	/

Rule

Path

Disallow

compspybot

Rule	Path
Disallow	/

Rule

Path

Disallow

trendictionbot

Rule	Path
Disallow	/

Rule

Path

Disallow

semrushbot

Rule	Path
Disallow	/

Rule

Path

Disallow

beetlebot

Rule	Path
Disallow	/

Rule

Path

Disallow

mail.ru_bot

Rule	Path
Disallow	/

Rule

Path

Disallow

istellabot

Rule	Path
Disallow	/

Rule

Path

Disallow

plukkie

Rule	Path
Disallow	/

Rule

Path

Disallow

melodibot

Rule	Path
Disallow	/

Rule

Path

Disallow

spbot

Rule	Path
Disallow	/

Rule

Path

Disallow

queryseekerspider

Rule	Path
Disallow	/

Rule

Path

Disallow

buddhabot

Rule	Path
Disallow	/

Rule

Path

Disallow

imrbot

Rule	Path
Disallow	/

Rule

Path

Disallow

acoonbot

Rule	Path
Disallow	/

Rule

Path

Disallow

yacybot

Rule	Path
Disallow	/

Rule

Path

Disallow

gufybot

Rule	Path
Disallow	/

Rule

Path

Disallow

nesotebot

Rule	Path
Disallow	/

Rule

Path

Disallow

sogou web spider

Rule	Path
Disallow	/

Rule

Path

Disallow

seokicks-robot

Rule	Path
Disallow	/

Rule

Path

Disallow

solomonobot

Rule	Path
Disallow	/

Rule

Path

Disallow

wbsearchbot

Rule	Path
Disallow	/

Rule

Path

Disallow

noyabot

Rule	Path
Disallow	/

Rule

Path

Disallow

flappybot

Rule	Path
Disallow	/

Rule

Path

Disallow

siteexplorer

Rule	Path
Disallow	/

Rule

Path

Disallow

proximic

Rule	Path
Disallow	/

Rule

Path

Disallow

blexbot

Rule	Path
Disallow	/

Rule

Path

Disallow

crawler4j

Rule	Path
Disallow	/

Rule

Path

Disallow

nutch

Rule	Path
Disallow	/

Rule

Path

Disallow

berlin-fu-cow

Rule	Path
Disallow	/

Rule

Path

Disallow

riddler

Rule	Path
Disallow	/

Rule

Path

Disallow

python-requests

Rule	Path
Disallow	/

Rule

Path

Disallow

python-requests/2.8.1

Rule	Path
Disallow	/

Rule

Path

Disallow

Other Records

Field	Value
sitemap	https://blog.geld.nl/sitemap.xml

Field

Value

sitemap

https://blog.geld.nl/sitemap.xml

Warnings

4 invalid lines.

blog.geld.nlrobots.txt

Resource Scan

Scan Details

Last Scan

Groups

*

rogerbot

ezooms

webcrawler

bubing

proximic

trendictionbot

vagabondo

bilbo

linkdexbot

exabot

slurp

niki-bot

compspybot

trendictionbot

semrushbot

beetlebot

mail.ru_bot

istellabot

plukkie

melodibot

spbot

queryseekerspider

buddhabot

imrbot

acoonbot

yacybot

gufybot

nesotebot

sogou web spider

seokicks-robot

solomonobot

wbsearchbot

noyabot

flappybot

siteexplorer

proximic

blexbot

crawler4j

nutch

berlin-fu-cow

riddler

python-requests

python-requests/2.8.1

Other Records

Warnings

blog.geld.nl
robots.txt