kafshe.org
robots.txt

Robots Exclusion Standard data for kafshe.org

Resource Scan

Scan Details

Site Domain kafshe.org
Base Domain kafshe.org
Scan Status Failed
Failure StageFetching resource.
Failure ReasonCouldn't connect to server.
Last Scan2025-08-09T16:42:07+00:00
Next Scan 2025-09-08T16:42:07+00:00

Last Successful Scan

Scanned2025-06-18T02:05:24+00:00
URL https://kafshe.org/robots.txt
Redirect https://kafshu.ir/robots.txt
Redirect Domain kafshu.ir
Redirect Base kafshu.ir
Domain IPs 104.21.84.54, 172.67.186.149, 2606:4700:3033::ac43:ba95, 2606:4700:3036::6815:5436
Redirect IPs 104.21.80.196, 172.67.187.26, 2606:4700:3033::6815:50c4, 2606:4700:3037::ac43:bb1a
Response IP 104.21.80.196
Found Yes
Hash 5e31751ecb1bce148cd31263af6b3dfd0bcab8f42ded6139e913da27e6b9d8e8
SimHash 135662222d1b

Groups

*

Rule Path
Disallow /wp-admin/
Allow /wp-admin/admin-ajax.php
Allow /wp-admin/load-scripts.php
Allow /wp-admin/load-styles.php
Allow /wp-content/uploads/
Allow /brand/

semrushbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

sogou

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

exabot

Rule Path
Disallow /

archive.org_bot

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

seznambot

Rule Path
Disallow /

ezooms

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

megaindex

Rule Path
Disallow /

cliqzbot

Rule Path
Disallow /

seekport crawler

Rule Path
Disallow /

sistrix

Rule Path
Disallow /

Other Records

Field Value
sitemap https://kafshe.org/sitemap_index.xml