Log In Sign Up

designlab.com
robots.txt

Robots Exclusion Standard data for designlab.com

Resource Scan

Scan Details

Site Domain designlab.com
Base Domain designlab.com
Scan Status Ok
Last Scan2025-09-28T22:49:53+00:00
Next Scan 2025-10-28T22:49:53+00:00

Last Scan

Scanned2025-09-28T22:49:53+00:00
URL https://designlab.com/robots.txt
Domain IPs 13.203.125.58, 13.233.175.166, 3.109.243.18
Response IP 3.109.243.18
Found Yes
Hash 5bd9d20169f7b162848f1c8ed0040469f34268ab3e72aa728617c49c996bf91b
SimHash f854d0150cb1

Groups

twitterbot

Rule Path
Disallow

facebookexternalhit

Rule Path
Disallow

*

Rule Path
Disallow /login/
Disallow /profile/
Disallow /academy/
Disallow /course/
Disallow /privacy/
Disallow /terms/
Disallow /mentor-dpa/
Disallow /code-of-conduct/
Disallow /certificates/
Disallow /blog-category/*/*_page%3D*%26*_page%3D*
Disallow /ui-ux-design-events-and-workshops?*_page=*

facebookexternalhit

Rule Path
Allow /
Back to top

Other Records

Field Value
sitemap https://designlab.com/sitemap.xml
Back to top

Comments

  • Don't allow extra, redundant blog category pagination
  • e.g. allow /blog-category/software-tools?4d16de4d_page=3
  • don't allow /blog-category/software-tools?4d16de4d_page=3&b48ec7e4_page=3
  • Allow Facebook to load meta data and previews from URLs that are normally blocked. Like certificates.
Back to top