bizcommunity.com
robots.txt

Robots Exclusion Standard data for bizcommunity.com

Resource Scan

Scan Details

Site Domain bizcommunity.com
Base Domain bizcommunity.com
Scan Status Ok
Last Scan2024-11-12T02:36:21+00:00
Next Scan 2024-11-19T02:36:21+00:00

Last Scan

Scanned2024-11-12T02:36:21+00:00
URL https://bizcommunity.com/robots.txt
Redirect https://www.bizcommunity.com/robots.txt
Redirect Domain www.bizcommunity.com
Redirect Base bizcommunity.com
Domain IPs 197.242.159.218
Redirect IPs 197.242.159.218
Response IP 197.242.159.218
Found Yes
Hash c0ba617d47bdaf22c3e51f3b976c3545ed4ec026c854d1e13fd8232a34c6090a
SimHash 883863313ad3

Groups

*

Rule Path
Disallow /Adm
Disallow /adm
Disallow /Ad/
Disallow /ad/
Disallow /Cf
Disallow /cf
Disallow /De
Disallow /de
Disallow /Feed
Disallow /feed
Disallow /Lo
Disallow /lo
Disallow /Help
Disallow /help
Disallow /Man
Disallow /man
Disallow /My
Disallow /my
Disallow /Red
Disallow /red
Disallow /Req
Disallow /req
Disallow /Secu
Disallow /secu
Disallow /Web
Disallow /web
Disallow /Trap.

googlebot-news

Rule Path
Disallow /job/nelspruit/sales/sales-consultantagent-687765a

ia_archiver

Rule Path
Disallow /

ia_archiver/1.6

Rule Path
Disallow /

girafa

Rule Path
Disallow /

msiecrawler

Rule Path
Disallow /

magentanews

Rule Path
Disallow /

magentanews/6.0

Rule Path
Disallow /

Comments

  • robots.txt must be lowercase, content is case sensitive
  • Old links
  • Disallow: /Job
  • Disallow: /job