etq.com
robots.txt

Robots Exclusion Standard data for etq.com

Resource Scan

Scan Details

Site Domain etq.com
Base Domain etq.com
Scan Status Ok
Last Scan2024-06-12T12:27:48+00:00
Next Scan 2024-07-12T12:27:48+00:00

Last Scan

Scanned2024-06-12T12:27:48+00:00
URL https://etq.com/robots.txt
Redirect https://www.etq.com/robots.txt
Redirect Domain www.etq.com
Redirect Base etq.com
Domain IPs 3.165.82.112, 3.165.82.27, 3.165.82.82, 3.165.82.83
Redirect IPs 104.21.90.193, 172.67.160.56, 2606:4700:3032::ac43:a038, 2606:4700:3037::6815:5ac1
Response IP 104.21.90.193
Found Yes
Hash 87352bc1b76e12bbbc5c438afbcf076ac959b433f2ee44aabd681bb9ce43597e
SimHash 64359bb050f3

Groups

*

Rule Path
Allow /wp/wp-admin/admin-ajax.php
Disallow /wp/wp-admin/
Disallow /wp/wp-includes/
Disallow /error-404

googlebot

Rule Path
Allow /

googlebot-image

Rule Path
Allow /app/uploads/

mediapartners-google

Rule Path
Allow /

adsbot-google

Rule Path
Allow /

googlebot-mobile

Rule Path
Allow /

bingbot

Rule Path
Allow /

msnbot

Rule Path
Allow /

msnbot-media

Rule Path
Allow /app/uploads/

applebot

Rule Path
Allow /

yandexbot

Rule Path
Allow /

yandeximages

Rule Path
Allow /app/uploads/

slurp

Rule Path
Allow /

baiduspider

Rule Path
Allow /

baiduspider-image

Rule Path
Allow /app/uploads/

duckduckbot

Rule Path
Allow /

qwantify

Rule Path
Allow /

dotbot

Rule Path
Disallow /

giftghostbot

Rule Path
Disallow /

seznam

Rule Path
Disallow /

paperlibot

Rule Path
Disallow /

genieo

Rule Path
Disallow /

dataprovider/6.101

Rule Path
Disallow /

dataprovidersiteexplorer

Rule Path
Disallow /

dazoobot/1.0

Rule Path
Disallow /

diffbot

Rule Path
Disallow /

domainstatsbot/1.0

Rule Path
Disallow /

dotbot/1.1

Rule Path
Disallow /

dubaiindex

Rule Path
Disallow /

ecommercebot

Rule Path
Disallow /

expertsearchspider

Rule Path
Disallow /

feedbin

Rule Path
Disallow /

fetch/2.0a

Rule Path
Disallow /

ffbot/1.0

Rule Path
Disallow /

focusbot/1.1

Rule Path
Disallow /

huaweisymantecspider

Rule Path
Disallow /

huaweisymantecspider/1.0

Rule Path
Disallow /

jobdiggerspider

Rule Path
Disallow /

lemurwebcrawler

Rule Path
Disallow /

lipperheylinkexplorer

Rule Path
Disallow /

lssrocketcrawler/1.0

Rule Path
Disallow /

lyt.srv1.5

Rule Path
Disallow /

miadev/0.0.1

Rule Path
Disallow /

najdi.si/3.1

Rule Path
Disallow /

bountiibot

Rule Path
Disallow /

experibot_v1

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 5

Other Records

Field Value
sitemap https://www.etq.com/sitemap_index.xml

Comments

  • Direct link to Sitemap.xml
  • General Crawl Directives
  • Allow All Known Bots
  • Disallow Spam Bots
  • Define Crawl Delay