Log In Sign Up

web-hosting-top.com
robots.txt

Robots Exclusion Standard data for web-hosting-top.com

Resource Scan

Scan Details

Site Domain web-hosting-top.com
Base Domain web-hosting-top.com
Scan Status Ok
Last Scan2025-06-16T08:43:15+00:00
Next Scan 2025-06-30T08:43:15+00:00

Last Scan

Scanned2025-06-16T08:43:15+00:00
URL https://web-hosting-top.com/robots.txt
Domain IPs 198.71.63.205
Response IP 198.71.63.205
Found Yes
Hash b43db327087604c3ce55cec3926bf452347bb57d9bca84b14ccfb2da359a8f22
SimHash 0a1a5590c880

Groups

petalbot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

semrushbot-sa

Rule Path
Disallow /

semrushbot-ba

Rule Path
Disallow /

semrushbot-si

Rule Path
Disallow /

semrushbot-swa

Rule Path
Disallow /

semrushbot-ct

Rule Path
Disallow /

semrushbot-bm

Rule Path
Disallow /

semrushbot-seoab

Rule Path
Disallow /

speedy

Rule Path
Disallow /

turnitinbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

sproose

Rule Path
Disallow /

becomebot

Rule Path
Disallow /

archive.org_bot

Rule Path
Disallow /

ia_archiver

Rule Path
Disallow /

gigabot

Rule Path
Disallow /

discobot

Rule Path
Disallow /

charlotte

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

exabot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

gurujibot

Rule Path
Disallow /

yacybot

Rule Path
Disallow /

googlebot

Rule Path
Disallow /*/type/visit$

msnbot

Rule Path
Disallow /*/type/visit$

slurp

Rule Path
Disallow /*/type/visit$
Back to top

Warnings

  • 2 invalid lines.
Back to top