glamgeek.com
robots.txt

Robots Exclusion Standard data for glamgeek.com

Resource Scan

Scan Details

Site Domain glamgeek.com
Base Domain glamgeek.com
Scan Status Ok
Last Scan2024-09-27T06:29:34+00:00
Next Scan 2024-10-04T06:29:34+00:00

Last Scan

Scanned2024-09-27T06:29:34+00:00
URL https://glamgeek.com/robots.txt
Redirect https://www.glamgeek.com/robots.txt
Redirect Domain www.glamgeek.com
Redirect Base glamgeek.com
Domain IPs 104.21.63.201, 172.67.150.13, 2606:4700:3030::ac43:960d, 2606:4700:3037::6815:3fc9
Redirect IPs 104.21.63.201, 172.67.150.13, 2606:4700:3030::ac43:960d, 2606:4700:3037::6815:3fc9
Response IP 104.21.63.201
Found Yes
Hash 794e79b13417ea7674490d1180dcb311a923b9f04594cc7d5baa4f16434ccda6
SimHash 881e4f42c65a

Groups

*

Rule Path
Disallow /blog/*
Disallow /buy$
Disallow /*/buy$
Disallow /*/buy?
Disallow /go$
Disallow /*/go$
Disallow /*/go?
Disallow /search*
Disallow /user/*

a6-indexer

Rule Path
Disallow /

alphaseobot

Rule Path
Disallow /

alphaseobot-sa

Rule Path
Disallow /

aspiegelbot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

blackboard safeassign

Rule Path
Disallow /

blexbot/1.0

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

crawler4j

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

gigabot

Rule Path
Disallow /

liebaofast

Rule Path
Disallow /

mauibot

Rule Path
Disallow /

mauibot (crawler.feedback+wc@gmail.com)

Rule Path
Disallow /

megaindex.ru/2.0

Rule Path
Disallow /

mqqbrowser

Rule Path
Disallow /

nimbostratus-bot/v1.3.2

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

seekport crawler

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

semrushbot-sa

Rule Path
Disallow /

seznambot

Rule Path
Disallow /

sputnikbot/2.3

Rule Path
Disallow /

the knowledge ai

Rule Path
Disallow /

turnitinbot

Rule Path
Disallow /

ucbrowser

Rule Path
Disallow /

yacybot

Rule Path
Disallow /

yeti

Rule Path
Disallow /

yisouspider

Rule Path
Disallow /

Comments

  • robots.txt
  • Block bots