liquery.com
robots.txt

Robots Exclusion Standard data for liquery.com

Resource Scan

Scan Details

Site Domain liquery.com
Base Domain liquery.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a server error.
Last Scan2025-03-15T13:11:21+00:00
Next Scan 2025-05-14T13:11:21+00:00

Last Successful Scan

Scanned2025-01-15T13:08:55+00:00
URL https://liquery.com/robots.txt
Redirect https://www.liquery.com/robots.txt
Redirect Domain www.liquery.com
Redirect Base liquery.com
Domain IPs 104.21.34.215, 172.67.165.150, 2606:4700:3035::ac43:a596, 2606:4700:3036::6815:22d7
Redirect IPs 104.21.34.215, 172.67.165.150, 2606:4700:3035::ac43:a596, 2606:4700:3036::6815:22d7
Response IP 104.21.34.215
Found Yes
Hash f947922d2d5b6b422b666c7ea4b46e41fa57f5be3bb6523e96d04ba4171ce348
SimHash 3e8c6b494f1b

Groups

*

Rule Path
Disallow /cgi-bin/WebFunctions.pm
Disallow /cgi-bin/gethtml.pl
Disallow /cgi-bin/lists.pl
Disallow /cgi-bin/nohup.out
Disallow /cgi-bin/counter.pl
Disallow /cgi-bin/index.pl
Disallow /cgi-bin/login.pl
Disallow /cgi-bin/parsejson.pl
Disallow /cgi-bin/search.pl
Disallow /cgi-bin/createpages.pl
Disallow /cgi-bin/inventory.pl
Disallow /cgi-bin/logout.pl
Disallow /cgi-bin/populateitems.pl
Disallow /cgi-bin/updateinventory.pl
Disallow /cgi-bin/generatesitemap.pl
Disallow /cgi-bin/lcboitem.pl
Disallow /cgi-bin/nav.pl
Disallow /cgi-bin/pricehistory.pl
Disallow /cgi-bin/updateitems.pl

Other Records

Field Value
sitemap http://www.liquery.com/sitemap.xml.gz

Comments

  • robots.txt for http://www.liquery.com/
  • Disallow: /cgi-bin/ #to prevent bots from creating session
  • selectively disallow
  • Disallow: /cgi-bin/product.pl