legistorm.com
robots.txt

Robots Exclusion Standard data for legistorm.com

Resource Scan

Scan Details

Site Domain legistorm.com
Base Domain legistorm.com
Scan Status Ok
Last Scan2024-05-22T07:44:59+00:00
Next Scan 2024-06-21T07:44:59+00:00

Last Scan

Scanned2024-05-22T07:44:59+00:00
URL https://legistorm.com/robots.txt
Redirect https://www.legistorm.com/robots.txt
Redirect Domain www.legistorm.com
Redirect Base legistorm.com
Domain IPs 104.26.4.208, 104.26.5.208, 172.67.69.49, 2606:4700:20::681a:4d0, 2606:4700:20::681a:5d0, 2606:4700:20::ac43:4531
Redirect IPs 104.26.4.208, 104.26.5.208, 172.67.69.49, 2606:4700:20::681a:4d0, 2606:4700:20::681a:5d0, 2606:4700:20::ac43:4531
Response IP 104.26.4.208
Found Yes
Hash 4f28396ff640152e55e2f0f46110f58a65717cfa490cab73b9d4df225578ff40
SimHash 6837da50a25a

Groups

slurp

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

msnbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 2

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

*

Rule Path
Disallow /search/*
Disallow /person_contact_list/*
Disallow /pfd/checkPfd/id/*
Disallow /getIdentifier/*
Disallow /hierarchy/*
Disallow /get_last_offices/*
Disallow /download_filing_pdf/*
Disallow /memberpdf/*
Disallow /hearingDownload/*
Disallow /getKML/*

teoma

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

psbot

Rule Path
Disallow /

gaisbot

Rule Path
Disallow /

e-societyrobot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

twengabot

Rule Path
Disallow /

yandex

Rule Path
Disallow /

netseer

Rule Path
Disallow /

blekkobot

Rule Path
Disallow /

proximic

Rule Path
Disallow /

yacybot

Rule Path
Disallow /

www.integromedb.org/crawler

Rule Path
Disallow /

scoutjet

Rule Path
Disallow /

googlebot-image

Rule Path
Disallow /