Log In Sign Up

covercraft.com
robots.txt

Robots Exclusion Standard data for covercraft.com

Resource Scan

Scan Details

Site Domain covercraft.com
Base Domain covercraft.com
Scan Status Ok
Last Scan2024-09-17T04:03:14+00:00
Next Scan 2024-10-17T04:03:14+00:00

Last Scan

Scanned2024-09-17T04:03:14+00:00
URL https://covercraft.com/robots.txt
Redirect https://www.covercraft.com/robots.txt
Redirect Domain www.covercraft.com
Redirect Base covercraft.com
Domain IPs 4.255.64.38
Redirect IPs 104.18.18.131, 104.18.19.131, 2606:4700::6812:1283, 2606:4700::6812:1383
Response IP 104.18.19.131
Found Yes
Hash 2fdc55d532979c13e749d863ff6b3204ecdac524a71eda8bec1f7dbfbef530e3
SimHash d0605f54bf82

Groups

blexbot

Rule Path
Disallow /

yandex

Rule Path
Disallow /

semrushbot-sa

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

fatbot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

domainappender

Rule Path
Disallow /

coccoc

Rule Path
Disallow /

image.coccoc

Rule Path
Disallow /

easouspider

Rule Path
Disallow /

linkdexbot

Rule Path
Disallow /

mail.ru_bot

Rule Path
Disallow /

netseer

Rule Path
Disallow /

proximic

Rule Path
Disallow /

searchmetricsbot

Rule Path
Disallow /

smtbot

Rule Path
Disallow /

seznambot

Rule Path
Disallow /

omgilibot

Rule Path
Disallow /

sogou web spider

Rule Path
Disallow /

pimonster

Rule Path
Disallow /

magpie-crawler

Rule Path
Disallow /

scrapy

Rule Path
Disallow /

expanse

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

mauibot

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

aspiegelbot

Rule Path
Disallow /

internet-measurement

Rule Path
Disallow /

dataforseobot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

barkrowler

Rule Path
Disallow /

wp_is_mobile

Rule Path
Disallow /

python-requests

Rule Path
Disallow /

java

Rule Path
Disallow /

mozlila

Rule Path
Disallow /

censysinspect

Rule Path
Disallow /

dataprovider

Rule Path
Disallow /

foregenix

Rule Path
Disallow /

ioncrawl

Rule Path
Disallow /

*

Rule Path
Allow /
Disallow /style-guide
Disallow /my-account/*
Disallow /cart
Disallow /checkout/*
Disallow /c/*?query=*
Back to top

Other Records

Field Value
sitemap https://www.covercraft.com/sitemap.xml
Back to top

Comments

  • bad user agents - copied from BCD
  • allow all other bots
  • internal stuff
Back to top

Warnings

  • 4 invalid lines.
Back to top