Log In Sign Up

jewelryandfindings.com
robots.txt

Robots Exclusion Standard data for jewelryandfindings.com

Resource Scan

Scan Details

Site Domain jewelryandfindings.com
Base Domain jewelryandfindings.com
Scan Status Ok
Last Scan2024-09-20T20:02:51+00:00
Next Scan 2024-10-20T20:02:51+00:00

Last Scan

Scanned2024-09-20T20:02:51+00:00
URL https://jewelryandfindings.com/robots.txt
Redirect https://www.jewelryandfindings.com/robots.txt
Redirect Domain www.jewelryandfindings.com
Redirect Base jewelryandfindings.com
Domain IPs 104.26.10.244, 104.26.11.244, 172.67.69.47, 2606:4700:20::681a:af4, 2606:4700:20::681a:bf4, 2606:4700:20::ac43:452f
Redirect IPs 104.26.10.244, 104.26.11.244, 172.67.69.47, 2606:4700:20::681a:af4, 2606:4700:20::681a:bf4, 2606:4700:20::ac43:452f
Response IP 172.67.69.47
Found Yes
Hash fc8a48704a94d078d060ad2ab66dd39390d3c82c2d6cda026fa4b3ff59a37e5e
SimHash d271d77cc630

Groups

*

Rule Path
Disallow /buyer/flash/
Disallow /common/flash/
Disallow /MyAccount/
Disallow /Customer/
Disallow /Order/
Disallow /Checkout/
Disallow /TicketCenter/
Disallow /Coupon/
Disallow /Cash/
Disallow /ShoppingCart-ShoppingCart-1

ahrefsbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

aspseek

Rule Path
Disallow /

axmo

Rule Path
Disallow /

booch

Rule Path
Disallow /

dts agent

Rule Path
Disallow /

downloader

Rule Path
Disallow /

emailcollector

Rule Path
Disallow /

emailsiphon

Rule Path
Disallow /

emailwolf

Rule Path
Disallow /

expired domain sleuth

Rule Path
Disallow /

gaisbot

Rule Path
Disallow /

grub

Rule Path
Disallow /

hughcrawler

Rule Path
Disallow /

iaea.org

Rule Path
Disallow /

lcabotaccept

Rule Path
Disallow /

iconsurf

Rule Path
Disallow /

iltrovatore-setaccio

Rule Path
Disallow /

indy library

Rule Path
Disallow /

iupui

Rule Path
Disallow /

kittiecentral

Rule Path
Disallow /

larbin

Rule Path
Disallow /

lwp-trivial

Rule Path
Disallow /

metatagrobot

Rule Path
Disallow /

missigua locator

Rule Path
Disallow /

netresearchserver

Rule Path
Disallow /

nextgensearch

Rule Path
Disallow /

npbot

Rule Path
Disallow /

nutch

Rule Path
Disallow /

objectssearch

Rule Path
Disallow /

oracle ultra search

Rule Path
Disallow /

peerbot

Rule Path
Disallow /

pictureofinternet

Rule Path
Disallow /

plantynet

Rule Path
Disallow /

quepasacreep

Rule Path
Disallow /

scspider

Rule Path
Disallow /

soft411

Rule Path
Disallow /

spider.acont.de

Rule Path
Disallow /

sqworm

Rule Path
Disallow /

ssm agent

Rule Path
Disallow /

tamu

Rule Path
Disallow /

theusefulbot

Rule Path
Disallow /

turnitinbot

Rule Path
Disallow /

tutorial crawler

Rule Path
Disallow /

tutorgig

Rule Path
Disallow /

webcopier

Rule Path
Disallow /

webzip

Rule Path
Disallow /

zipppbot

Rule Path
Disallow /

xenu

Rule Path
Disallow /

wotbox

Rule Path
Disallow /

wget

Rule Path
Disallow /

naverbot

Rule Path
Disallow /

mozdex

Rule Path
Disallow /

sosospider

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

youdaobot

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

toutiaospider

Rule Path
Disallow /

yisouspider

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

ahrefssiteaudit

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

ahrefsbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

mj12bot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

pinterest

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 1

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 1
Back to top

Comments

  • <URL:http://www.robotstxt.org/wc/exclusion.html#robotstxt>
  • Format is:
  • User-agent: <name of spider>
  • Disallow: <nothing> | <path>
  • -----------------------------------------------------------------------------
Back to top

Warnings

  • 7 invalid lines.
Back to top