signeda.lt
robots.txt

Robots Exclusion Standard data for signeda.lt

Resource Scan

Scan Details

Site Domain signeda.lt
Base Domain signeda.lt
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2024-10-08T04:26:27+00:00
Next Scan 2025-01-06T04:26:27+00:00

Last Successful Scan

Scanned2023-08-21T07:12:40+00:00
URL https://signeda.lt/robots.txt
Redirect https://www.signeda.lt/robots.txt
Redirect Domain www.signeda.lt
Redirect Base signeda.lt
Domain IPs 104.26.10.127, 104.26.11.127, 172.67.70.96, 2606:4700:20::681a:a7f, 2606:4700:20::681a:b7f, 2606:4700:20::ac43:4660
Redirect IPs 104.26.10.127, 104.26.11.127, 172.67.70.96, 2606:4700:20::681a:a7f, 2606:4700:20::681a:b7f, 2606:4700:20::ac43:4660
Response IP 104.26.11.127
Found Yes
Hash 112de089957c78873c1fa6cb7c90b64c958ba95fd1bf5fcee609301c9286b146
SimHash a219c90245f4

Groups

velenpublicwebcrawler
crawler_eb_germany_2.0

Rule Path
Disallow /

*

Rule Path
Disallow /dev/*
Disallow /test/*
Disallow /ua/*
Disallow /lt/*
Disallow /en/*
Disallow /de/*
Disallow /md/*
Disallow /lv/*
Disallow /pl/*
Disallow /ru/*
Disallow /nl/*
Disallow /*%26amp%3Bquality
Disallow /*?route=product%2Fsearch
Disallow /account/login
Disallow /account/register
Disallow /admin/
Disallow /system/
Disallow /catalog/pics
Disallow /catalog/model
Disallow /catalog/language
Disallow /catalog/controller
Disallow /csv/
Disallow /*?route=product%2F*

Warnings

  • 1 invalid line.