bjnephrology.org
robots.txt

Robots Exclusion Standard data for bjnephrology.org

Resource Scan

Scan Details

Site Domain bjnephrology.org
Base Domain bjnephrology.org
Scan Status Ok
Last Scan2025-12-08T14:09:37+00:00
Next Scan 2026-01-07T14:09:37+00:00

Last Scan

Scanned2025-12-08T14:09:37+00:00
URL https://bjnephrology.org/robots.txt
Domain IPs 162.241.63.20
Response IP 162.241.63.20
Found Yes
Hash f29c8b845eb403a82427d00db9d4f8aa848aa7cbbe60a21c600c1fe30a06e6a5
SimHash 6338cc52e0b9

Groups

*

Rule Path
Disallow /wp-admin/
Disallow /wp-includes/
Disallow /wp-content/plugins/
Disallow /wp-login.php
Disallow /xmlrpc.php
Disallow /feed/
Disallow /?s=
Disallow /search
Allow /wp-admin/admin-ajax.php

gptbot
chatgpt-user
ccbot
claudebot
google-extended
alphaai
anthropic-ai
applebot-extended
bytespider
claude-web
claudebot
diffbot
facebookbot
gptbot
imagesiftbot
omgili
omgilibot
openai
seekport crawler
seekbot
semrushbot/2~bl
semrushbot
semrushbot-sa
baiduspider
baiduspider/2.0
ahrefsbot
mj12bot
dotbot
perplexitybot
barkrowler
dataforseobot
coccocbot-web

Rule Path
Disallow /

bingbot
msnbot
msnbot-media

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

Other Records

Field Value
sitemap https://bjnephrology.org/sitemap_index.xml

Comments

  • ===== Block AI & scraping bots =====