Log In Sign Up

neocodex.us
robots.txt

Robots Exclusion Standard data for neocodex.us

Resource Scan

Scan Details

Site Domain neocodex.us
Base Domain neocodex.us
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2025-03-13T11:42:00+00:00
Next Scan 2025-06-11T11:42:00+00:00

Last Successful Scan

Scanned2024-10-22T11:40:19+00:00
URL https://neocodex.us/robots.txt
Domain IPs 104.21.46.24, 172.67.222.160, 2606:4700:3031::ac43:dea0, 2606:4700:3036::6815:2e18
Response IP 104.21.46.24
Found Yes
Hash 5a932b11cbd88bd9bafdd5f4d192adb528cbc1cca5102661dfa2bd2035cd073a
SimHash d232dc90d238

Groups

mediapartners-google

Rule Path
Disallow

magpie-crawler

Rule Path
Disallow /

msnbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 2

ahrefsbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

yandex

No rules defined. All paths allowed.

Other Records

Field Value Comment
crawl-delay 5 specifies a 2 second timeout

semrushbot-sa

No rules defined. All paths allowed.

Other Records

Field Value Comment
crawl-delay 5 specifies a 2 second timeout

*

Rule Path
Disallow /api/
Disallow /forum/ircChat/
Disallow /forum/admin/
Disallow /forum/cache/
Disallow /forum/converge_local/
Disallow /forum/hooks/
Disallow /forum/ips_kernel/
Disallow /forum/retail/
Disallow /forum/public/style_captcha/
Disallow /forum/index.php?app=core&module=task
Disallow /forum/index.php?app=core&module=usercp&tab=forums&area=forumsubs
Disallow /forum/index.php?app=core&module=usercp&tab=forums&area=watch&watch=topic
Disallow /forum/index.php?app=forums&module=extras&section=forward
Disallow /forum/index.php?app=members&module=messaging
Disallow /forum/index.php?app=members&module=chat
Disallow /forum/index.php?app=members&module=search
Disallow /forum/index.php?app=members&module=search&do=active
Disallow /forum/index.php?&unlockUserAgent=1
Disallow /*app%3Dcore%26module%3Dusercp
Disallow /*app%3Dcore%26amp%3Bmodule%3Dusercp
Disallow /*app%3Dmembers%26module%3Dmessaging
Disallow /*%26p%3D
Disallow /*%26pid%3D
Disallow /*%26hl%3D
Disallow /*%26start%3D
Disallow /*view__getnewpost$
Disallow /*view__getlastpost$
Disallow /*view__old$
Disallow /*view__new$
Disallow /*view__getfirst$
Disallow /*view__getprevious$
Disallow /*view__getnext$
Disallow /*view__getlast$
Disallow /*%26view%3Dgetnewpost$
Disallow /*%26view%3Dgetlastpost$
Disallow /*%26view%3Dold$
Disallow /*%26view%3Dnew$
Disallow /*%26view%3Dgetfirst$
Disallow /*%26view%3Dgetprevious$
Disallow /*%26view%3Dgetnext$
Disallow /*%26view%3Dgetlast$
Disallow /*?s=
Disallow /*%26s%3D
Disallow /forum/index.php?app=core&module=global&section=login&do=deleteCookies
Disallow /forum/index.php?app=core&module=global&section=rss
Disallow /forum/index.php?app=forums&module=extras&section=rating
Disallow /forum/index.php?app=forums&module=forums&section=markasread
Disallow /forum/index.php?app=forums&module=moderate&section=moderate
Disallow /forum/index.php?module=post&section=post&do=postreply
Disallow /forum/index.php?app=itemdb&module=charts&section=price
Disallow /forum/index.php?app=members&module=profile&section=dname
Disallow /forum/index.php?app=members&module=profile&section=friends
Disallow /forum/index.php?app=members&module=profile&section=status
Disallow /forum/index.php?app=members&module=messaging
Disallow /forum/index.php?app=core&module=search
Disallow /forum/index.php?app=forums&module=extras
Back to top