lcia.org
robots.txt

Robots Exclusion Standard data for lcia.org

Resource Scan

Scan Details

Site Domain lcia.org
Base Domain lcia.org
Scan Status Ok
Last Scan2024-09-17T01:52:22+00:00
Next Scan 2024-10-17T01:52:22+00:00

Last Scan

Scanned2024-09-17T01:52:22+00:00
URL https://lcia.org/robots.txt
Domain IPs 104.22.68.137, 104.22.69.137, 172.67.11.36, 2606:4700:10::6816:4489, 2606:4700:10::6816:4589, 2606:4700:10::ac43:b24
Response IP 104.22.68.137
Found Yes
Hash 41966db78632d100ac3d99cda8653f127ebf332746e96bc68a6590ade4ac3c33
SimHash 2140d9a78674

Groups

*

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 600

googlebot
googlebot-image
mediapartners-google
msnbot
msnbot-media
slurp
yahoo-blogs
yahoo-mmcrawler

Rule Path
Disallow /article/preview.aspx
Disallow /access/
Disallow /aspnet_client/
Disallow /bin/
Disallow /content/preview.aspx
Disallow /dashboard/
Disallow /data/
Disallow /email/
Disallow /events/bookings/
Disallow /events/preview.aspx
Disallow /events/success.aspx
Disallow /faq/preview.apsx
Disallow /form/
Disallow /forum/preview.aspx
Disallow /media/download.aspx
Disallow /membership/conformation.aspx
Disallow /membership/paymentreturn.aspx
Disallow /membership/preview.aspx
Disallow /membership/subscribe.aspx
Disallow /membership/subscribedetails.aspx
Disallow /membership/success.aspx
Disallow /poll/preview.aspx
Disallow /poll/success.aspx
Disallow /shop/checkout.aspx
Disallow /shop/paymentreturn.aspx
Disallow /shop/transactions.aspx
Disallow /user/
Disallow /usercontrols/
Disallow /utility/

Other Records

Field Value
crawl-delay 600

Comments

  • but allow only important bots
  • Directories