Log In Sign Up

ibm.com
robots.txt

Robots Exclusion Standard data for ibm.com

Resource Scan

Scan Details

Site Domain ibm.com
Base Domain ibm.com
Scan Status Ok
Last Scan2024-05-31T13:51:07+00:00
Next Scan 2024-06-14T13:51:07+00:00

Last Scan

Scanned2024-05-31T13:51:07+00:00
URL https://ibm.com/robots.txt
Redirect https://www.ibm.com/robots.txt
Redirect Domain www.ibm.com
Redirect Base ibm.com
Domain IPs 23.15.98.127, 2600:1413:b000:586::3831, 2600:1413:b000:59c::3831
Redirect IPs 104.69.36.203, 2600:1413:b000:38e::1e89, 2600:1413:b000:397::1e89
Response IP 104.69.36.203
Found Yes
Hash d5d9cf8c64ebabd80de568aef6fab26086f01cb3c5a42fcb9d7c62a9797b319e
SimHash ad1c1851ae50

Groups

*

Rule Path
Disallow /account/registration
Disallow /account/mypro
Disallow /account/myint
Disallow /Admin
Disallow /cgi-
Disallow /contact/employees/servlets
Disallow /data/
Disallow /db2s
Disallow /developerworks/*-pdf.pdf$
Disallow /developerworks/forums/servlet
Disallow /developerworks/forums/abuse
Disallow /developerworks/forums/post
Disallow /docs/api
Disallow /fcgi-
Disallow /fscripts
Disallow /image
Disallow /mashupmaker
Disallow /mysupport/s/forumshome
Disallow /mysupport/s/forumsproduct
Disallow /mysupport/s/forumsquestion
Disallow /partnerworld/portal
Disallow /portal/home
Disallow /products/finder
Disallow /profiles
Disallow /scripts
Disallow /Scripts
Disallow /search/?
Disallow /Search
Disallow /smartmarket/*/productListInitial.action?
Disallow /standards
Disallow /support/knowledgecenter/api
Disallow /support/knowledgecenter/v1
Disallow /survey
Disallow /testing
Disallow /tmp
Disallow /vrm/bin/sendThisPage
Disallow /webmaster
Disallow /zx
Disallow /zz
Disallow /*.ssi$
Disallow /account/myibm/InterestsEdit.do
Disallow /wcs
Disallow /wcsstore
Disallow /webapp
Disallow /web/portal/software/websphere
Disallow /common/austin-summit
Disallow /link
Disallow /links
Disallow /web/portal/commerce
Disallow /industries/clients
Disallow /standards
Disallow /test
Disallow /test-kick-tires
Disallow /test-v1-page-template
Disallow /product-detail
Disallow /watson/assets/pdfs/reg
Disallow /zprod-ease
Disallow /zProd-ease
Disallow /zshowcase
Disallow /widgets-admin
Disallow /w3

fast corporate crawler

Rule Path
Disallow /account/registration
Disallow /account/mypro
Disallow /account/myint
Disallow /Admin
Disallow /cgi-
Disallow /common
Disallow /contact/employees/servlets
Disallow /data/
Disallow /db2s
Disallow /developerworks
Disallow /fcgi-
Disallow /fscripts
Disallow /homepage
Disallow /i/
Disallow /image
Disallow /investor
Disallow /link
Disallow /mashupmaker
Disallow /portal/home
Disallow /products/finder
Disallow /profiles
Disallow /SametimeWebApp
Disallow /scripts
Disallow /Scripts
Disallow /search/?
Disallow /Search
Disallow /survey
Disallow /tmp
Disallow /vrm/bin/sendThisPage
Disallow /webmaster
Disallow /zx
Disallow /zz

magpie-crawler

Rule Path
Disallow /connections/

magpie-crawler/1.1

Rule Path
Disallow /connections/

magpierss

Rule Path
Disallow /connections/

psnsearch

Rule Path
Disallow /developerworks/wikis/login.action

httrack

Rule Path
Disallow /

acrolinx-bot

Rule Path
Disallow /account/registration
Disallow /account/mypro
Disallow /account/myint
Disallow /Admin
Disallow /cgi-
Disallow /contact/employees/servlets
Disallow /data/
Disallow /db2s
Disallow /developerworks/*-pdf.pdf$
Disallow /developerworks/forums/servlet
Disallow /developerworks/forums/abuse
Disallow /developerworks/forums/post
Disallow /docs/api
Disallow /fcgi-
Disallow /fscripts
Disallow /image
Disallow /mashupmaker
Disallow /mysupport/s/forumshome
Disallow /mysupport/s/forumsproduct
Disallow /mysupport/s/forumsquestion
Disallow /partnerworld/portal
Disallow /portal/home
Disallow /products/finder
Disallow /profiles
Disallow /scripts
Disallow /Scripts
Disallow /search/?
Disallow /Search
Disallow /smartmarket/*/productListInitial.action?
Disallow /standards
Disallow /support/knowledgecenter/api
Disallow /support/knowledgecenter/v1
Disallow /survey
Disallow /testing
Disallow /tmp
Disallow /vrm/bin/sendThisPage
Disallow /webmaster
Disallow /zx
Disallow /zz
Disallow /*.ssi$
Disallow /account/myibm/InterestsEdit.do
Disallow /wcs
Disallow /wcsstore
Disallow /webapp
Disallow /web/portal/software/websphere
Disallow /common/austin-summit
Disallow /link
Disallow /links
Disallow /web/portal/commerce
Disallow /industries/clients
Disallow /standards
Disallow /test
Disallow /test-kick-tires
Disallow /test-v1-page-template
Disallow /product-detail
Disallow /watson/assets/pdfs/reg
Disallow /zprod-ease
Disallow /zProd-ease
Disallow /zshowcase
Back to top

Other Records

Field Value
sitemap https://www.ibm.com/homepage_sitemap.xml
sitemap https://www.ibm.com/downloads/cas/sitemap/sitemap.xml
sitemap https://www.ibm.com/common/ssi/start/sitemap.xml
sitemap https://www.ibm.com/content/dam/adobe-cms/IBM_Adobe_Sitemap.xml
sitemap https://www.ibm.com/mysupport/s/sitemap.xml
sitemap https://www.ibm.com/docs/sitemap.xml
sitemap https://www.ibm.com/blog/sitemap_index.xml
Back to top

Comments

  • $Id: robots.txt,v 1.101 2023/03/29 05:00:57 sumartow Exp $
  • This is a file retrieved by webwalkers a.k.a. spiders that
  • conform to a defacto standard.
  • See <URL:http://www.robotstxt.org/wc/exclusion.html#robotstxt>
  • Comments to the webmaster should be posted at <URL:http://www.ibm.com/contact>
  • Format is:
  • User-agent: <name of spider>
  • Disallow: <nothing> | <path>
  • ------------------------------------------------------------------------------
  • Disallow: /homepage
  • Disallow: /internal
  • Added for IN4173782 on 7Aug2013
  • Added for IN4177562 on 8Aug2013
  • Added for IN4177562 on 8Aug2013
  • Added to block site mirroring
  • Disallow: //
  • Disallow: /homepage
  • Disallow: /internal
  • Added for EI-2179 on 17Apr2020
  • Added by JLiao for SD EI-2359,EI-2360 on 23Jun2020
  • Added for EI-2216 on 06May2020
Back to top

Warnings

  • 3 invalid lines.
Back to top