fickpension.com
robots.txt

Robots Exclusion Standard data for fickpension.com

Resource Scan

Scan Details

Site Domain fickpension.com
Base Domain fickpension.com
Scan Status Ok
Last Scan2024-06-06T12:08:53+00:00
Next Scan 2024-07-06T12:08:53+00:00

Last Scan

Scanned2024-06-06T12:08:53+00:00
URL https://fickpension.com/robots.txt
Domain IPs 13.226.210.39, 13.226.210.48, 13.226.210.77, 13.226.210.81
Response IP 108.157.60.32
Found Yes
Hash 04771c5da09e09c98c8fa360d288d3c179a2dc8b33901b36fdad9a0e514b4002
SimHash 0b5ec864abeb

Groups

mj12bot

Rule Path
Disallow /

yandexbot

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

sougou

Rule Path
Disallow /

bingbot

Rule Path
Disallow /es/
Disallow /fr/
Disallow /it/
Disallow /tr/
Disallow /no/
Disallow /nl/

Other Records

Field Value
crawl-delay 90

*

Rule Path
Disallow /es/
Disallow /fr/
Disallow /it/
Disallow /tr/
Disallow /no/
Disallow /nl/

Other Records

Field Value
crawl-delay 30

Comments

  • GUPPYWEB4