Log In Sign Up

megahookups.net
robots.txt

Robots Exclusion Standard data for megahookups.net

Resource Scan

Scan Details

Site Domain megahookups.net
Base Domain megahookups.net
Scan Status Ok
Last Scan5/19/2025, 11:51:13 AM
Next Scan 6/18/2025, 11:51:13 AM

Last Scan

Scanned5/19/2025, 11:51:13 AM
URL https://megahookups.net/robots.txt
Redirect https://www.megahookups.net/robots.txt
Redirect Domain www.megahookups.net
Redirect Base megahookups.net
Domain IPs 104.21.80.211, 172.67.153.212, 2606:4700:3036::6815:50d3, 2606:4700:3037::ac43:99d4
Redirect IPs 104.21.80.211, 172.67.153.212, 2606:4700:3036::6815:50d3, 2606:4700:3037::ac43:99d4
Response IP 104.21.80.211
Found Yes
Hash 2820e0b1aa206202844d5b2ef50222e6c666a821e669068f0b3bef8a7db5547d
SimHash b007d03b89a9

Groups

*

Rule Path
Disallow /?s=dmca
Disallow /?s=terms
Disallow /?s=2257
Disallow /?p=join
Disallow /members
Disallow /webmasters
Disallow /dmca
Disallow /terms
Disallow /2257
Disallow /join
Disallow /tags/
Disallow /tag/
Disallow /user/
Disallow /users/
Disallow /category/
Disallow /rating/
Disallow /video/
Disallow /feed/
Disallow /page/

wget
scrapy
simplepie
go-resty
cfnetwork
facebookexternalhit
wget
python-requests
sogou web spider
checkmarknetwork
domaincrawler
weborama-fetcher
libwww-perl
zmeu
crawler4j
curl
go-http-client
ahrefs
mj12bot
seokicks
domainsbot
komodiabot
dotbot
nimbostratus
smtbot
ccbot
semrush
xforce-security
pdrlabs.net
security.ipip.net
mail.ru
yandex
baidu
baiduboxapp
m.baidu.com
baiduspider

Rule Path
Disallow /

*

Rule Path
Disallow /js/
Disallow /ajax/
Disallow /portal
Back to top

Comments

  • __________ ___. __ __ __
  • \______ \ ____\_ |__ _____/ |_ ______ _/ |____ ____/ |_
  • | _// _ \| __ \ / _ \ __\/ ___/ \ __\ \/ /\ __\
  • | | ( <_> ) \_\ ( <_> ) | \___ \ | | > < | |
  • |____|_ /\____/|___ /\____/|__| /____ > /\ |__| /__/\_ \ |__|
  • \/ \/ \/ \/ \/
Back to top