dsj.org
robots.txt

Robots Exclusion Standard data for dsj.org

Resource Scan

Scan Details

Site Domain dsj.org
Base Domain dsj.org
Scan Status Ok
Last Scan2025-10-17T06:51:36+00:00
Next Scan 2025-11-16T06:51:36+00:00

Last Scan

Scanned2025-10-17T06:51:36+00:00
URL https://dsj.org/robots.txt
Redirect https://www.dsj.org/robots.txt
Redirect Domain www.dsj.org
Redirect Base dsj.org
Domain IPs 141.193.213.10, 141.193.213.11
Redirect IPs 104.196.252.123
Response IP 104.196.252.123
Found Yes
Hash 81089a17e2cfdde7552d2dccf0f71a7f9dbff56ac38388ae5f0054e05e45598b
SimHash 6a1f54e8fa92

Groups

*

Rule Path
Disallow /wp-login.php
Disallow /wp-admin/

Other Records

Field Value
crawl-delay 600

gptbot

Rule Path
Disallow /

amazonbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

uptimerobot

Rule Path
Disallow /

turnitin

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

freshpingbot

Rule Path
Disallow /

googlebot

Rule Path
Allow /

bingbot

Rule Path
Disallow /

jooblebot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

facebookexternalhit

Rule Path
Disallow /

seekportbot

Rule Path
Disallow /