Log In Sign Up

liquidweb.com
robots.txt

Robots Exclusion Standard data for liquidweb.com

Resource Scan

Scan Details

Site Domain liquidweb.com
Base Domain liquidweb.com
Scan Status Ok
Last Scan2024-05-01T06:32:06+00:00
Next Scan 2024-05-31T06:32:06+00:00

Last Scan

Scanned2024-05-01T06:32:06+00:00
URL https://liquidweb.com/robots.txt
Redirect https://www.liquidweb.com/robots.txt
Redirect Domain www.liquidweb.com
Redirect Base liquidweb.com
Domain IPs 108.138.7.116, 108.138.7.36, 108.138.7.44, 108.138.7.93, 2600:9000:2490:1000:1f:a0ac:7fc0:93a1, 2600:9000:2490:3400:1f:a0ac:7fc0:93a1, 2600:9000:2490:5000:1f:a0ac:7fc0:93a1, 2600:9000:2490:800:1f:a0ac:7fc0:93a1, 2600:9000:2490:aa00:1f:a0ac:7fc0:93a1, 2600:9000:2490:ac00:1f:a0ac:7fc0:93a1, 2600:9000:2490:f200:1f:a0ac:7fc0:93a1, 2600:9000:2490:fa00:1f:a0ac:7fc0:93a1
Redirect IPs 108.157.254.15, 108.157.254.22, 108.157.254.49, 108.157.254.69, 2600:9000:2753:1c00:6:f48d:3e00:93a1, 2600:9000:2753:3400:6:f48d:3e00:93a1, 2600:9000:2753:7800:6:f48d:3e00:93a1, 2600:9000:2753:8800:6:f48d:3e00:93a1, 2600:9000:2753:9000:6:f48d:3e00:93a1, 2600:9000:2753:b400:6:f48d:3e00:93a1, 2600:9000:2753:f000:6:f48d:3e00:93a1, 2600:9000:2753:f600:6:f48d:3e00:93a1
Response IP 108.157.254.22
Found Yes
Hash 8ea802f418c9d4b19abfa47ad4e6218dba791b02143b15432612bb0393d77c3e
SimHash 7874b6404bb2

Groups

*

Rule Path Comment
Disallow /wp-admin/ -
Disallow /xmlrpc.php -
Disallow /sso/ -
Disallow /delegate-www.html -
Disallow /bingsiteauth.xml -
Disallow /rigor/ -
Disallow *.pdf$ Block pdf files from all bots.
Disallow /affiliate/home/ -
Disallow /kb/wp-admin/ -
Disallow /kb/xmlrpc.php -

irlbot

Rule Path
Disallow /

nerdybot

Rule Path
Disallow /

trendictionbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

duggmirror

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap https://www.liquidweb.com/sitemap.xml
Back to top

Comments

  • robots.txt for https://www.liquidweb.com/
  • 10 March 2020
  • Greetings human user.
  • This is robot territory.
  • To join our human team please visit www.liquidweb.com/careers/
  • Misc pages
  • PDF Files
  • Affiliate portal
  • KB pages
  • Texas A&M research bot
  • NerdyBot for Nerdy Data dotcom
  • Trendiction Bot - market research
  • MJ12Bot
  • Dugg content scraper
Back to top