lib-art.com
robots.txt

Robots Exclusion Standard data for lib-art.com

Resource Scan

Scan Details

Site Domain lib-art.com
Base Domain lib-art.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2025-07-01T01:31:56+00:00
Next Scan 2025-08-30T01:31:56+00:00

Last Successful Scan

Scanned2024-12-18T07:21:03+00:00
URL https://lib-art.com/robots.txt
Domain IPs 104.21.62.131, 172.67.136.42, 2606:4700:3030::ac43:882a, 2606:4700:3034::6815:3e83
Response IP 104.21.62.131
Found Yes
Hash 741514bb3ffe540fb67252bf0ec3606a9d1f67d68898c82c0d97439e792e3169
SimHash 420c9cc2c599

Groups

yandex

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 19

mj12bot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 19

ahrefsbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 19

blexbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 19

semrushbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 19

*

Rule Path
Disallow /cnstats

*

Rule Path
Disallow /tmp

*

Rule Path
Disallow /forum

Other Records

Field Value
sitemap http://www.lib-art.com/sitemap.xml