net32.com
robots.txt

Robots Exclusion Standard data for net32.com

Resource Scan

Scan Details

Site Domain net32.com
Base Domain net32.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2024-08-05T10:28:52+00:00
Next Scan 2024-11-03T10:28:52+00:00

Last Successful Scan

Scanned2023-09-18T10:27:09+00:00
URL https://net32.com/robots.txt
Redirect https://www.net32.com/robots.txt
Redirect Domain www.net32.com
Redirect Base net32.com
Domain IPs 104.18.8.138, 104.18.9.138, 2606:4700::6812:88a, 2606:4700::6812:98a
Redirect IPs 104.18.8.138, 104.18.9.138, 2606:4700::6812:88a, 2606:4700::6812:98a
Response IP 104.18.9.138
Found Yes
Hash 083180fd915233102938bd2fe822a1b21ce94bc7ebcae96048a16507c28ce41e
SimHash 30f0dc6caf4b

Groups

*

Rule Path
Disallow /admin

*

Rule Path
Disallow /oms

*

Rule Path
Disallow /rest

*

Rule Path
Disallow /media/dental/en_US/user-guide

*

Rule Path
Disallow /promo/buy-get/*/*$

*

Rule Path
Disallow /ec/*/reviews$

yandexbot

Rule Path
Disallow /

mail.ru_bot

Rule Path
Disallow /

sputnik

Rule Path
Disallow /

arachni

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

easouspider

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

sogou web spider

Rule Path
Disallow /

zumbot

Rule Path
Disallow /

exabot

Rule Path
Disallow /

mojeekbot

Rule Path
Disallow /

sitesucker

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

semrushbot-sa

Rule Path
Disallow

Comments

  • Disallows indexing of the PDPs behind the buy/get pages.
  • Disallows indexing of the PDP's review pages.
  • RUSSIA
  • RUSSIA
  • RUSSIA
  • RUSSIA
  • UKRAINE
  • CHINA
  • CHINA
  • S. KOREA
  • S. KOREA
  • FRANCE
  • UK
  • VORACIOUS PEST
  • WEBMEUP CRAWLER
  • SEMRUSH
  • (Allowed)