menewsha.com
robots.txt

Robots Exclusion Standard data for menewsha.com

Resource Scan

Scan Details

Site Domain menewsha.com
Base Domain menewsha.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2026-02-05T01:54:56+00:00
Next Scan 2026-05-06T01:54:56+00:00

Last Successful Scan

Scanned2025-06-16T00:10:39+00:00
URL https://menewsha.com/robots.txt
Redirect https://www.menewsha.com/robots.txt
Redirect Domain www.menewsha.com
Redirect Base menewsha.com
Domain IPs 104.21.20.34, 172.67.191.74
Redirect IPs 104.21.20.34, 172.67.191.74
Response IP 172.67.191.74
Found Yes
Hash fbdff459e84ecd33e8245cda77a0852003fe13b2069fc09bf7c5af32cd9cf51e
SimHash 9206c162eb12

Groups

semrushbot

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

baiduspider
yisouspider
petalbot

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

sogou web spider

Rule Path
Disallow /

sogou inst spider

Rule Path
Disallow /

*

Rule Path
Disallow /forum/members/
Disallow /forum/search.php
Allow /

amazonbot

Rule Path
Disallow /

meta-externalagent

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

Warnings

  • 1 invalid line.