Log In Sign Up

megahit.org
robots.txt

Robots Exclusion Standard data for megahit.org

Resource Scan

Scan Details

Site Domain megahit.org
Base Domain megahit.org
Scan Status Ok
Last Scan2024-11-08T13:31:54+00:00
Next Scan 2024-11-15T13:31:54+00:00

Last Scan

Scanned2024-11-08T13:31:54+00:00
URL https://megahit.org/robots.txt
Domain IPs 104.21.56.213, 172.67.155.244, 2606:4700:3031::6815:38d5, 2606:4700:3036::ac43:9bf4
Response IP 172.67.155.244
Found Yes
Hash c15cd85911b374291659968823d6e9bf11db4d193edbf5c4c21c8b61fbe4d4b7
SimHash 510df471c511

Groups

*

Rule Path
Disallow /engine/go.php
Disallow /user/
Disallow /newposts/
Disallow /statistics.html
Disallow /*subaction%3Duserinfo
Disallow /*subaction%3Dnewposts
Disallow /*do%3Dlastcomments
Disallow /*do%3Dfeedback
Disallow /*do%3Dregister
Disallow /*do%3Dlostpassword
Disallow /*do%3Daddnews
Disallow /*do%3Dstats
Disallow /*do%3Dpm
Disallow /*do%3Dsearch
Disallow /*do%3Ddownload
Disallow /*do%3Dgo
Disallow /download/

mj12bot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

ia_archiver

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

applebot

Rule Path
Disallow /

amazonbot

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap https://megahit.org/sitemap.xml
Back to top

Warnings

  • `host` is not a known field.
Back to top