www2.montgomerycountymd.gov
robots.txt

Robots Exclusion Standard data for www2.montgomerycountymd.gov

Resource Scan

Scan Details

Site Domain www2.montgomerycountymd.gov
Base Domain montgomerycountymd.gov
Scan Status Ok
Last Scan2024-05-31T23:40:05+00:00
Next Scan 2024-06-30T23:40:05+00:00

Last Scan

Scanned2024-05-31T23:40:05+00:00
URL https://www2.montgomerycountymd.gov/robots.txt
Domain IPs 104.18.30.168, 104.18.31.168, 2606:4700::6812:1ea8, 2606:4700::6812:1fa8
Response IP 104.18.30.168
Found Yes
Hash 8ea3831a9aa334ad0844d3acd6b464377e0d6105433232b459f512f63d14e667
SimHash 5c179331d751

Groups

googlebot

Rule Path
Disallow /mcgportalapps/SearchPhoneBook.aspx

slurp

Rule Path
Disallow /mcgportalapps/SearchPhoneBook.aspx

msnbot

Rule Path
Disallow /mcgportalapps/SearchPhoneBook.aspx

sogou

Rule Path
Disallow /

sogou spider

Rule Path
Disallow /

sogou web spider/4.0(+http://www.sogou.com/docs/help/webmasters.htm

Product Comment
sogou web spider/4.0(+http://www.sogou.com/docs/help/webmasters.htm 07)
Rule Path
Disallow /
Disallow /mcgportalapps/SearchPhoneBook.aspx

magpie-crawler

Rule Path
Disallow /

ut-dorkbot

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

neevabot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

seekrbot

Rule Path
Disallow /

yandexbot

Rule Path
Disallow /

seznambot

Rule Path
Disallow /

bitsightbot

Rule Path
Disallow /

mauibot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

Comments

  • //* Added the following on 20230103 *//