helifreak.com
robots.txt

Robots Exclusion Standard data for helifreak.com

Resource Scan

Scan Details

Site Domain helifreak.com
Base Domain helifreak.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2024-05-29T12:56:51+00:00
Next Scan 2024-06-05T12:56:51+00:00

Last Successful Scan

Scanned2023-09-22T13:58:19+00:00
URL https://helifreak.com/robots.txt
Redirect https://www.helifreak.com/robots.txt
Redirect Domain www.helifreak.com
Redirect Base helifreak.com
Domain IPs 104.16.248.51, 104.16.249.51
Redirect IPs 104.17.97.5, 104.17.98.5, 2606:4700::6811:6105, 2606:4700::6811:6205
Response IP 104.17.98.5
Found Yes
Hash adb36c1b963d536d5f477a137318bbbade07bac0cc0b186d30f30370bd0f783a
SimHash 26c6c97903c7

Groups

boardtracker

Rule Path
Disallow /

gigabot

Rule Path
Disallow /

twiceler

Rule Path
Disallow /

slurp

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 2

msnbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 2

*

Rule Path
Disallow /admincp/
Disallow /announcement.php
Disallow /calendar.php
Disallow /cron.php
Disallow /editpost.php
Disallow /faq.php
Disallow /group.php
Disallow /inlinemod.php
Disallow /joinrequests.php
Disallow /login.php
Disallow /member.php
Disallow /memberlist.php
Disallow /misc.php
Disallow /modcp/
Disallow /moderator.php
Disallow /newreply.php
Disallow /newthread.php
Disallow /online.php
Disallow /payments.php
Disallow /printthread.php
Disallow /private.php
Disallow /profile.php
Disallow /register.php
Disallow /report.php
Disallow /search.php
Disallow /sendmessage.php
Disallow /showgroups.php
Disallow /showpost.php
Disallow /subscription.php
Disallow /subscriptions.php
Disallow /threadrate.php
Disallow /usercp.php
Disallow /usernote.php
Disallow /vbgooglemapme.php
Disallow /blog.php
Disallow /advert/
Disallow /album.php
Disallow /mobiquo/
Disallow /*sort%3D*
Disallow /*order%3D*
Disallow /*prefixid%3D*