Log In Sign Up

urbanladder.com
robots.txt

Robots Exclusion Standard data for urbanladder.com

Resource Scan

Scan Details

Site Domain urbanladder.com
Base Domain urbanladder.com
Scan Status Ok
Last Scan2024-09-22T17:47:29+00:00
Next Scan 2024-10-06T17:47:29+00:00

Last Scan

Scanned2024-09-22T17:47:29+00:00
URL https://urbanladder.com/robots.txt
Redirect https://www.urbanladder.com:443/robots.txt
Redirect Domain www.urbanladder.com
Redirect Base urbanladder.com
Domain IPs 3.6.4.72, 65.0.159.51
Redirect IPs 96.17.96.23, 96.17.96.6
Response IP 23.32.29.96
Found Yes
Hash 6fdf5c39d01883e5a392de410d320c3125a97c2f648fb0fe7fad705317bfe958
SimHash f0a5698f6fd0

Groups

*

Rule Path
Disallow */checkout*
Disallow */cart*
Disallow */orders*
Disallow */countries*
Disallow */line_items*
Disallow */password_resets*
Disallow */states*
Disallow */user_sessions*
Disallow */user_registrations*
Disallow */users*
Disallow */account*
Disallow */assets/icofont-preview.html*
Disallow */products/search*
Disallow */products/search*
Disallow */login*
Disallow */auth/google_oauth2*
Disallow */auth/facebook*
Disallow */resetpassword*
Disallow */signup*
Disallow */lcvtnc*
Disallow */ulhappy2014*
Disallow */ul101*
Disallow */ulhappyloop*
Disallow */privacy-policy*
Disallow */shipping-delivery*
Disallow */security-payments*
Disallow */mobile_app_install_static_page*
Disallow */services/emis_applicable*
Disallow */rate-experience*
Disallow */icici-home-loan*
Disallow */sku*
Disallow */*?___SID=U*
Disallow /*?keywords=*
Disallow */locations/sofa-set-in-*
Disallow */locations/single-beds-in-*
Disallow */locations/double-beds-in-*
Disallow */locations/fabric-sofas-in-*
Disallow */locations/l-shaped-sofas-in-*
Disallow */locations/wooden-sofas-in-*
Disallow */locations/leatherette-sofas-in-*
Disallow */locations/bed-in-*
Disallow */locations/living-room-furniture-in-*
Disallow */locations/mattresses-in-*
Disallow */locations/dining-table-set-in-*
Disallow */modular-wardrobes-change*
Allow *.js
Allow *.css

*

Rule Path
Disallow */search*
Back to top

Other Records

Field Value
sitemap https://www.urbanladder.com/sitemaps/sitemap.xml.gz
sitemap https://www.urbanladder.com/sitemaps/sitemap1.xml.gz
sitemap https://www.urbanladder.com/sitemaps/sitemap2.xml.gz
sitemap https://www.urbanladder.com/sitemaps/sitemap3.xml.gz
sitemap https://www.urbanladder.com/sitemaps/sitemapImagesIndex.xml.gz
Back to top

Comments

  • See http://www.robotstxt.org/wc/norobots.html for documentation on how to use the robots.txt file
  • To ban all spiders from the entire site uncomment the next two lines:
  • User-agent: *
  • Disallow: /
  • Blocking SKU urls - 3rd June 2021
  • Blocking keyword URL's
  • Blocking Location Pages of 3 Category - 8th Feb 2021
  • URL parameters blocking for SEO
  • Tweak for SRC Only
  • Commented on Jan 7, 2020
  • Disallow: /*?src*
  • Tweak for SEARCH
Back to top