Log In Sign Up

goldkueste24.ch
robots.txt

Robots Exclusion Standard data for goldkueste24.ch

Resource Scan

Scan Details

Site Domain goldkueste24.ch
Base Domain goldkueste24.ch
Scan Status Ok
Last Scan2024-09-21T15:30:55+00:00
Next Scan 2024-09-28T15:30:55+00:00

Last Scan

Scanned2024-09-21T15:30:55+00:00
URL https://goldkueste24.ch/robots.txt
Domain IPs 52.212.52.84, 54.247.69.169, 63.32.161.232
Response IP 54.247.69.169
Found Yes
Hash 28cb47eaf9ee5162152306bb996b119049708870a7ae03a0c84f4ccb0c55dcc9
SimHash 7b76507aff73

Groups

ccbot

Rule Path
Disallow /

flamingo_searchengine

Rule Path
Disallow /

exabot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

genieo

Rule Path
Disallow /

crystalsemantics

Rule Path
Disallow /

grapeshot

Rule Path
Disallow /

megaindex.ru/2.0

Rule Path
Disallow /

blp_bbot/0.1

Rule Path
Disallow /

slurp

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 1

gptbot

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow /

psbot
petalbot
mail.ru_bot
megaindex
yisouspider
bytespider
sogou web spider
sogou inst spider
proximic
admantx
seekport crawler
blexbot
mj12bot

Rule Path
Disallow /
Back to top

Comments

  • Special Areas of the Page
  • Special File Endings:
  • Disallow commercial bots
Back to top

Warnings

  • 3 invalid lines.
Back to top