astroradio.com
robots.txt

Robots Exclusion Standard data for astroradio.com

Resource Scan

Scan Details

Site Domain astroradio.com
Base Domain astroradio.com
Scan Status Ok
Last Scan2024-09-17T04:46:28+00:00
Next Scan 2024-10-17T04:46:28+00:00

Last Scan

Scanned2024-09-17T04:46:28+00:00
URL https://www.astroradio.com/robots.txt
Domain IPs 82.223.120.226
Response IP 82.223.120.226
Found Yes
Hash 0d9a876c765125d615c2b106edd45f1df12075b69b54c632a7d7eaf38f1582ef
SimHash f34e514aeeb5

Groups

googlebot-image

Rule Path
Disallow /wp-includes/
Allow /wp-content/uploads/

mediapartners-google*

Rule Path
Disallow

semrushbot

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

duggmirror

Rule Path
Disallow /

noxtrumbot

Rule Path
Disallow /

msiecrawler

Rule Path
Disallow /

webcopier

Rule Path
Disallow /

httrack

Rule Path
Disallow /

doc

Rule Path
Disallow /

zao

Rule Path
Disallow /

sitecheck.internetseer.com

Rule Path
Disallow /

zealbot

Rule Path
Disallow /

sitesnagger

Rule Path
Disallow /

webstripper

Rule Path
Disallow /

webcopier

Rule Path
Disallow /

fetch

Rule Path
Disallow /

teleport

Rule Path
Disallow /

teleportpro

Rule Path
Disallow /

webzip

Rule Path
Disallow /

linko

Rule Path
Disallow /

xenu

Rule Path
Disallow /

larbin

Rule Path
Disallow /

zyborg

Rule Path
Disallow /

wget

Rule Path
Disallow /

grub-client

Rule Path
Disallow /

k2spider

Rule Path
Disallow /

google-hoteladsverifier

Rule Path
Disallow /

npbot

Rule Path
Disallow /

webreaper

Rule Path
Disallow /

berlin-fu-cow

Rule Path
Disallow /

jikespider

Rule Path
Disallow /

semager

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

orangebot

Rule Path
Disallow /

screaming frog seo spider

Rule Path
Disallow /

megaindex.ru/2.0

Rule Path
Disallow /

megaindex.ru

Rule Path
Disallow /

megaindex.ru

Rule Path
Disallow /

seokicks-robot

Rule Path
Disallow /

ltx71

Rule Path
Disallow /

sputnikbot

Rule Path
Disallow /

spbot

Rule Path
Disallow /

seoscanners.net

Rule Path
Disallow /

deusu

Rule Path
Disallow /

meds-online24.com

Rule Path
Disallow /

seoscanners

Rule Path
Disallow /

pingdom.com

Rule Path
Disallow /

prtgcloudbot

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

bubing

Rule Path
Disallow /

lcc

Rule Path
Disallow /

mauibot

Rule Path
Disallow /

mauibot+

Rule Path
Disallow /

sogou web spider

Rule Path
Disallow /

seekport crawler

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

barkrowler

Rule Path
Disallow /

microsoft.url.control

Rule Path
Disallow /

libwww

Rule Path
Disallow /

seznambot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

adidxbot

Rule Path
Disallow /

the knowledge ai

Rule Path
Disallow /

netestate ne crawler

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

amazonbot

Rule Path
Disallow /

seekportbot

Rule Path
Disallow /

*

Rule Path
Disallow /*add-to-cart%3D*
Disallow /*?*
Disallow /?s=
Disallow /carrito/
Disallow /confirmar1/
Disallow /en/cart/
Disallow /en/checkout/
Disallow /ca/cistella-de-la-compra/
Disallow /ca/confirmar-compra/
Disallow /trackback/
Disallow /wp-admin/
Disallow /feed/
Disallow */feed/
Disallow */trackback/
Disallow /*.sql$
Disallow /*.tgz$
Disallow /*.gz$
Disallow /*.tar$
Disallow /*.svn$
Disallow /staticblocks/
Disallow /en/staticblocks/
Disallow /ca/staticblocks/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/

Warnings

  • 1 invalid line.