Log In Sign Up

boost-my.website
robots.txt

Robots Exclusion Standard data for boost-my.website

Resource Scan

Scan Details

Site Domain boost-my.website
Base Domain boost-my.website
Scan Status Ok
Last Scan2025-09-02T01:53:28+00:00
Next Scan 2025-10-02T01:53:28+00:00

Last Scan

Scanned2025-09-02T01:53:28+00:00
URL https://boost-my.website/robots.txt
Domain IPs 185.151.30.171, 2a07:7800::171
Response IP 185.151.30.171
Found Yes
Hash 94b7890452af788c03db8c3a6efa9eec28412b03e3cf7d79cd8f8b4a421621bc
SimHash 721cdf04684b

Groups

googlebot

Rule Path
Allow /

googlebot-image

Rule Path
Allow /

mediapartners-google

Rule Path
Allow /

google-inspectiontool

Rule Path
Allow /

bingbot

Rule Path
Allow /

yandexbot

Rule Path
Allow /

applebot

Rule Path
Allow /

linkedinbot

Rule Path
Allow /

twitterbot

Rule Path
Allow /

pinterestbot

Rule Path
Allow /

facebookexternalhit

Rule Path
Allow /

gptbot

Rule Path
Allow /

duckduckbot

Rule Path
Allow /

baiduspider

Rule Path
Allow /

sogou spider

Rule Path
Allow /

slurp

Rule Path
Allow /

ccbot

Rule Path
Allow /

yeti

Rule Path
Allow /

ahrefsbot

Rule Path
Allow /

semrushbot

Rule Path
Allow /

semrushbot-si

Rule Path
Allow /

semrushbot-ba

Rule Path
Allow /

semrushbot-swa

Rule Path
Allow /

semrushbot-ct

Rule Path
Allow /

semrushbot-coub

Rule Path
Allow /

splitsignalbot

Rule Path
Allow /

rogerbot

Rule Path
Allow /

dotbot

Rule Path
Allow /

vegi bot

Rule Path
Allow /

seostats

Rule Path
Allow /

mauibot

Rule Path
Allow /

techseo360

Rule Path
Allow /

mj12bot

Rule Path
Allow /

similarweb

Rule Path
Allow /

screaming frog seo spider

Rule Path
Allow /

lumar

Rule Path
Allow /

cognitiveseo

Rule Path
Allow /

oncrawl

Rule Path
Allow /

blexbot

Rule Path
Allow /

megaindex.ru

Rule Path
Allow /

siteauditbot

Rule Path
Allow /

sitebulb

Rule Path
Allow /

botify

Rule Path
Allow /

jetoctopus

Rule Path
Allow /

netpeak spider

Rule Path
Allow /

contentking

Rule Path
Allow /

exabot

Rule Path
Allow /

swiftbot

Rule Path
Allow /

uptimerobot

Rule Path
Allow /

import.io

Rule Path
Allow /

webhose.io

Rule Path
Allow /

dexi.io

Rule Path
Allow /

zyte

Rule Path
Allow /

outwit

Rule Path
Allow /

getleft

Rule Path
Allow /

httrack

Rule Path
Allow /

cyotek webcopy

Rule Path
Allow /

helium scraper

Rule Path
Allow /

sequentum

Rule Path
Allow /

webharvy

Rule Path
Allow /

visual scraper

Rule Path
Allow /

parsehub

Rule Path
Allow /

octoparse

Rule Path
Allow /

*

Rule Path
Disallow /

amazonbot

Rule Path
Disallow /

*

Rule Path
Disallow /wp-admin/
Allow /wp-admin/admin-ajax.php
Back to top

Warnings

  • 2 invalid lines.
Back to top