Log In Sign Up

obgist.com
robots.txt

Robots Exclusion Standard data for obgist.com

Resource Scan

Scan Details

Site Domain obgist.com
Base Domain obgist.com
Scan Status Ok
Last Scan2025-10-19T21:07:19+00:00
Next Scan 2025-10-26T21:07:19+00:00

Last Scan

Scanned2025-10-19T21:07:19+00:00
URL https://obgist.com/robots.txt
Redirect https://www.obgist.com/robots.txt
Redirect Domain www.obgist.com
Redirect Base obgist.com
Domain IPs 104.21.48.136, 172.67.152.38, 2606:4700:3035::ac43:9826, 2606:4700:3036::6815:3088
Redirect IPs 104.21.48.136, 172.67.152.38, 2606:4700:3035::ac43:9826, 2606:4700:3036::6815:3088
Response IP 172.67.152.38
Found Yes
Hash a01eaa4400197517543f54d0b0ad296b162fea44d5795c82e7ae2dca82d57d78
SimHash 60400381a494

Groups

*

Rule Path
Disallow /comments/feeds/
Disallow /*?showComment=*
Disallow /*/comments/default/*
Disallow /search/
Disallow /search?*
Disallow /search?*updated-max=*
Disallow /search/label/*?
Disallow /*?updated-max=*
Allow /search/label/
Allow /feeds/posts/default
Allow /feeds/posts/summary
Back to top

Other Records

Field Value
sitemap https://www.obgist.com/sitemap.xml
sitemap https://www.obgist.com/sitemap-pages.xml
Back to top

Comments

  • Obgist Copyrighted
  • Block Comment Feeds
  • Block search result pagination
  • Allow label searches (for SEO)
  • Allow feeds and homepage
  • Preferred domain for sitemaps
Back to top