patternobsession.com
robots.txt

Robots Exclusion Standard data for patternobsession.com

Resource Scan

Scan Details

Site Domain patternobsession.com
Base Domain patternobsession.com
Scan Status Failed
Failure ReasonScan timed out.
Last Scan2026-01-30T04:48:55+00:00
Next Scan 2026-03-01T04:48:55+00:00

Last Successful Scan

Scanned2026-01-01T04:47:18+00:00
URL https://patternobsession.com/robots.txt
Domain IPs 141.136.33.244, 2a02:4780:a:1960:0:356c:c7d8:3
Response IP 141.136.33.244
Found Yes
Hash 767630b31e7aecb5d08462514a1bdc4a85d0e924a8ee25bf0ff44908ef2bb885
SimHash 2732db50e6b0

Groups

*

Rule Path
Allow /
Disallow /admin/
Disallow /api.php
Disallow /private/
Disallow /tmp/
Disallow /cache/
Disallow /config/
Allow /PrivacyPolicy.php
Allow /TermsofService.php
Allow /index.php
Allow /allArticles.php
Allow /ourStory.php
Allow /support.php
Allow /sitemap.xml
Allow /sitemap.php
Allow /hari_mkheyra.php
Allow /optimize.php
Allow /pattern_api.php
Allow /*.css$
Allow /*.js$
Allow /*.txt$
Allow /*.jpg$
Allow /*.jpeg$
Allow /*.png$
Allow /*.gif$
Allow /*.svg$
Allow /*.webp$

Other Records

Field Value
crawl-delay 1

mediapartners-google

Rule Path
Allow /

adsbot-google

Rule Path
Allow /

ahrefsbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

Other Records

Field Value
sitemap https://patternobsession.com/sitemap.xml

Comments

  • Pattern Obsession - Robots.txt
  • Updated: 2026-01-01
  • Allow all crawlers by default
  • Disallow specific directories
  • Explicitly allow important pages
  • Allow CSS and other assets
  • Sitemap location
  • Crawl delay (1 second between requests)
  • Allow Mediavine crawler explicitly
  • Block aggressive crawlers