fickpension.com
robots.txt

Robots Exclusion Standard data for fickpension.com

Resource Scan

Scan Details

Site Domain fickpension.com
Base Domain fickpension.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2024-09-26T12:23:14+00:00
Next Scan 2024-11-25T12:23:14+00:00

Last Successful Scan

Scanned2024-07-06T12:09:06+00:00
URL https://fickpension.com/robots.txt
Domain IPs 3.160.196.102, 3.160.196.106, 3.160.196.31, 3.160.196.62
Response IP 3.160.246.31
Found Yes
Hash 04771c5da09e09c98c8fa360d288d3c179a2dc8b33901b36fdad9a0e514b4002
SimHash 0b5ec864abeb

Groups

mj12bot

Rule Path
Disallow /

yandexbot

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

sougou

Rule Path
Disallow /

bingbot

Rule Path
Disallow /es/
Disallow /fr/
Disallow /it/
Disallow /tr/
Disallow /no/
Disallow /nl/

Other Records

Field Value
crawl-delay 90

*

Rule Path
Disallow /es/
Disallow /fr/
Disallow /it/
Disallow /tr/
Disallow /no/
Disallow /nl/

Other Records

Field Value
crawl-delay 30

Comments

  • GUPPYWEB4