Log In Sign Up

officespace.com
robots.txt

Robots Exclusion Standard data for officespace.com

Resource Scan

Scan Details

Site Domain officespace.com
Base Domain officespace.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2024-10-07T12:40:14+00:00
Next Scan 2025-01-05T12:40:14+00:00

Last Successful Scan

Scanned2024-06-10T12:39:02+00:00
URL https://officespace.com/robots.txt
Redirect https://www.officespace.com/robots.txt
Redirect Domain www.officespace.com
Redirect Base officespace.com
Domain IPs 54.204.41.220
Redirect IPs 54.204.41.220
Response IP 54.204.41.220
Found Yes
Hash 396d384f8fa747272f0a58f05872825a54df1b965d84e2a7c04fa12c4803e8c6
SimHash 82607ccf8450

Groups

*

Rule Path
Disallow /api/osnew/v3/batchstats
Disallow /comments
Disallow /favorites
Disallow /find$
Disallow /find?
Disallow /find/
Disallow /leads
Disallow /map/
Disallow /map?
Disallow /map$
Disallow /messages
Disallow /phone
Disallow /search/map/
Disallow /search/map?
Disallow /search/map$
Disallow /stats/error
Disallow /stats/log
Disallow /uploads
Disallow /users/sign_in
Disallow /users/sign_out
Disallow /users/sign_up
Disallow /users/auth
Disallow /users/password
Disallow /api/*

sogou spider

Rule Path
Disallow /

turnitinbot

Rule Path
Disallow /

yandex

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

domain re-animator bot

Rule Path
Disallow /

spbot

Rule Path
Disallow /

crazywebcrawler-spider

Rule Path
Disallow /

yahoo pipes 1.0

Rule Path
Disallow /

sogou web spider

Rule Path
Disallow /

cliqzbot

Rule Path
Disallow /

gigabot

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap https://www.officespace.com/system/os-daily-sitemap.xml.gz
sitemap https://www.officespace.com/system/os-sitemap.xml.gz
Back to top

Comments

  • See http://www.robotstxt.org/wc/norobots.html for documentation on how to use the robots.txt file
  • To ban all spiders from the entire site uncomment the next two lines:
  • User-Agent: *
  • Disallow: /
  • Disallow: /manage
  • Disallow: /users/edit
  • 2024-04-08 Disabled some below for OS-157/SEO work
  • Should be reenabled later or just handle with nginx rules
  • Disallow: /admin
  • Disallow: /pages/add_availability
  • Disallow: /listingcontact
  • Disallow: /api/listings/heatmap
  • Disallow: /building/agent_contact
  • Disallow: /building/tenants
  • Disallow: /inquiry/inquiry_form
  • Disallow: /api/listings
  • Disallow: /zip/*
  • Disallow: /special-purpose
  • http://openlinkprofiler.org/bot
  • http://www.crazywebcrawler.com/
  • http://pipes.yahoo.com/pipes/
  • http://www.sogou.com/docs/help/webmasters.htm
  • https://cliqz.com/en/cliqzbot
  • http://www.gigablast.com/
Back to top