Log In Sign Up

demarque.com
robots.txt

Robots Exclusion Standard data for demarque.com

Resource Scan

Scan Details

Site Domain demarque.com
Base Domain demarque.com
Scan Status Ok
Last Scan2024-09-20T05:58:28+00:00
Next Scan 2024-10-20T05:58:28+00:00

Last Scan

Scanned2024-09-20T05:58:28+00:00
URL https://demarque.com/robots.txt
Redirect https://www.demarque.com/robots.txt
Redirect Domain www.demarque.com
Redirect Base demarque.com
Domain IPs 159.203.16.84
Redirect IPs 4.205.72.224
Response IP 4.205.72.224
Found Yes
Hash 9d35d280765b665f57e0cd69123e429c94d8bebf405a82ab33e12c4f5337e800
SimHash 3875d351c5d5

Groups

applebot

Rule Path
Disallow /

bingbot

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

googlebot

Rule Path
Allow /

*

No rules defined. All paths allowed.

Back to top

Comments

  • Applebot is the web crawler for Apple.
  • https://www.apple.com/go/applebot
  • Bing search engine robot.
  • https://www.bing.com/bingbot.htm
  • Petal search engine Robot (HUAWEI).
  • https://aspiegel.com/petalbot
  • SEMrushBot is the search bot software used by SEMrush.
  • https://www.semrush.com/bot.html
  • Googlebot is the generic name for Google's web crawler.
  • https://support.google.com/webmasters/answer/182072
  • All other robots are disallowed from crawling our servers.
Back to top