spritmonitor.de
robots.txt

Robots Exclusion Standard data for spritmonitor.de

Resource Scan

Scan Details

Site Domain spritmonitor.de
Base Domain spritmonitor.de
Scan Status Ok
Last Scan2024-07-03T00:45:43+00:00
Next Scan 2024-07-10T00:45:43+00:00

Last Scan

Scanned2024-07-03T00:45:43+00:00
URL https://spritmonitor.de/robots.txt
Redirect https://www.spritmonitor.de/robots.txt
Redirect Domain www.spritmonitor.de
Redirect Base spritmonitor.de
Domain IPs 62.138.184.52
Redirect IPs 2001:1520:1:200::530, 62.138.184.52
Response IP 62.138.184.52
Found Yes
Hash 5e7af97abec2d95e7d4b005f7fb62e85d308e3fd4a3138bdaba8fc69ba00d2a3
SimHash 620e88d0c312

Groups

*

Rule Path
Disallow /de/nachricht.html
Disallow /en/message.html
Disallow /fr/message.html
Disallow /de/mein_account.html
Disallow /en/my_account.html
Disallow /fr/mon_compte.html

fasterfox

Rule Path
Disallow /

mediapartners-google

Rule Path
Disallow

shopwiki

Rule Path
Disallow /

yandex

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

mj12bot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 120

gptbot

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

ias_crawler

Rule Path
Disallow /

mojeekbot

Rule Path
Disallow /