writeonconforums.org
robots.txt

Robots Exclusion Standard data for writeonconforums.org

Resource Scan

Scan Details

Site Domain writeonconforums.org
Base Domain writeonconforums.org
Scan Status Failed
Failure StageFetching resource.
Failure ReasonCouldn't connect to server.
Last Scan2024-09-09T17:59:07+00:00
Next Scan 2024-12-08T17:59:07+00:00

Last Successful Scan

Scanned2023-10-22T20:22:23+00:00
URL https://writeonconforums.org/robots.txt
Domain IPs 104.18.241.94, 104.18.242.94
Response IP 104.18.242.94
Found Yes
Hash c3c6216c27040f0aa7580445d746d7db737e4ff291b78a896b2231acc8984abb
SimHash 4a1cdc43e690

Groups

mj12bot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

semrushbot-sa

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

snappreviewbot

Rule Path
Disallow /

netseer

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

buyerbase

Rule Path
Disallow /

buyerbase user agent

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

msnbot

Rule Path
Disallow /login/
Disallow /post/
Disallow /message/
Disallow /search/
Disallow /calendar/
Disallow /conversation/

Other Records

Field Value
crawl-delay 30

bingbot

Rule Path
Disallow /login/
Disallow /post/
Disallow /message/
Disallow /search/
Disallow /calendar/
Disallow /conversation/

Other Records

Field Value
crawl-delay 60

*

Rule Path
Disallow /login/
Disallow /post/
Disallow /message/
Disallow /search/
Disallow /calendar/
Disallow /conversation/
Disallow /threads/recent
Disallow /posts/recent