angloinfo.com
robots.txt

Robots Exclusion Standard data for angloinfo.com

Resource Scan

Scan Details

Site Domain angloinfo.com
Base Domain angloinfo.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2024-09-14T13:33:48+00:00
Next Scan 2024-12-13T13:33:48+00:00

Last Successful Scan

Scanned2022-09-05T16:04:19+00:00
URL https://angloinfo.com/robots.txt
Redirect https://www.angloinfo.com/robots.txt
Redirect Domain www.angloinfo.com
Redirect Base angloinfo.com
Response IP 13.33.33.88
Found Yes
Hash 3785417e0be2b8f5f463d73cd4c9f3217d62b3ca9aabf68f92ff6a14b6f7c691
SimHash 881efc02ea30

Groups

blexbot

Rule Path
Disallow /

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

ia_archiver

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

magpie-crawler

Rule Path
Disallow /

megaindex.ru/2.0

Rule Path
Disallow /

megaindex.ru

Rule Path
Disallow /

megaindex.ru

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

semrushbot-sa

Rule Path
Disallow /

semrushbot-ba

Rule Path
Disallow /

semrushbot-si

Rule Path
Disallow /

semrushbot-swa

Rule Path
Disallow /

semrushbot-ct

Rule Path
Disallow /

semrushbot-bm

Rule Path
Disallow /

semrushbot-seoab

Rule Path
Disallow /

seekport crawler

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

linespider

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

*

Rule Path
Disallow /monaco/events/
Disallow /monaco/discussions/
Disallow /monaco/classifieds/
Disallow /31457341/
Disallow /member/
Disallow /*/*/*/*/like
Disallow /*/*/*/*/report