community.thebackend.io
robots.txt

Robots Exclusion Standard data for community.thebackend.io

Resource Scan

Scan Details

Site Domain community.thebackend.io
Base Domain thebackend.io
Scan Status Ok
Last Scan2025-10-25T14:37:23+00:00
Next Scan 2025-11-24T14:37:23+00:00

Last Scan

Scanned2025-10-25T14:37:23+00:00
URL https://community.thebackend.io/robots.txt
Domain IPs 13.125.167.211, 3.37.105.98
Response IP 3.37.105.98
Found Yes
Hash 551099caaf35fa4ee7b1cad6fad153dec341762a9779f992ea310999ab19d114
SimHash a89d1d857670

Groups

mauibot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

seo spider

Rule Path
Disallow /

*

Rule Path
Disallow /admin/
Disallow /auth/
Disallow /assets/browser-update*.js
Disallow /email/
Disallow /session
Disallow /user-api-key
Disallow /*?api_key*
Disallow /*?*api_key*
Disallow /badges
Disallow /u/
Disallow /my
Disallow /search
Disallow /tag/*/l
Disallow /g
Disallow /t/*/*.rss
Disallow /c/*.rss
Disallow /t/topic/3432
Disallow /t/topic/3746
Disallow /t/topic/4220
Disallow /t/topic/593

googlebot

Rule Path
Disallow /admin/
Disallow /auth/
Disallow /assets/browser-update*.js
Disallow /email/
Disallow /session
Disallow /user-api-key
Disallow /*?api_key*
Disallow /*?*api_key*

Comments

  • See http://www.robotstxt.org/robotstxt.html for documentation on how to use the robots.txt file