Log In Sign Up

travelcarsnz.com
robots.txt

Robots Exclusion Standard data for travelcarsnz.com

Resource Scan

Scan Details

Site Domain travelcarsnz.com
Base Domain travelcarsnz.com
Scan Status Ok
Last Scan2024-11-12T14:46:35+00:00
Next Scan 2024-11-19T14:46:35+00:00

Last Scan

Scanned2024-11-12T14:46:35+00:00
URL https://travelcarsnz.com/robots.txt
Domain IPs 147.79.122.219, 2a02:4780:39:c24f:2b48:fb6f:2b48:6f45
Response IP 147.79.122.219
Found Yes
Hash c90a83401266b10242859fc0aa30ef90f0cc603c7a4939c2c6e2e47eeb1f0d8a
SimHash 309edad1d6fa

Groups

googleweblight

Rule Path
Allow /

googlebot-image

Rule Path
Allow /

adsbot-google-mobile

Rule Path
Allow /

adsbot-google

Rule Path
Allow /

mediapartners-google

Rule Path
Disallow /wp-admin/

etaospider

Rule Path
Disallow /

shopwiki

Rule Path
Disallow /

tineye-bot

Rule Path
Disallow /

fasterfox

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

sproose

Rule Path
Disallow /

shopwiki

Rule Path
Disallow /

shim-crawler

Rule Path
Disallow /

steeler

Rule Path
Disallow /

exabot

Rule Path
Disallow /

ng

Rule Path
Disallow /

fatbot

Rule Path
Disallow /

voyager

Rule Path
Disallow /

gigabot

Rule Path
Disallow /

redcarpet

Rule Path
Disallow /

cazoodle

Rule Path
Disallow /

answerbus

Rule Path
Disallow /

teoma

Rule Path
Disallow /

becomebot

Rule Path
Disallow /

auditbot

Rule Path
Disallow /

page_verifier

Rule Path
Disallow /

visbot

Rule Path
Disallow /

sproose

Rule Path
Disallow /

multicrawler

Rule Path
Disallow /

hiddenmarket

Rule Path
Disallow /

furlbot

Rule Path
Disallow /

minirank

Rule Path
Disallow /

worqmada

Rule Path
Disallow /

ocelli

Rule Path
Disallow /

accoona-ai-agent

Rule Path
Disallow /

bookdog

Rule Path
Disallow /

kretrieve

Rule Path
Disallow /

snappy

Rule Path
Disallow /

webpix

Rule Path
Disallow /

pyquery

Rule Path
Disallow /

depspid

Rule Path
Disallow /

netresearchserver

Rule Path
Disallow /

xspider

Rule Path
Disallow /

easydl

Rule Path
Disallow /

sygolbot

Rule Path
Disallow /

charlotte

Rule Path
Disallow /

smbot

Rule Path
Disallow /

easydl

Rule Path
Disallow /

panscient.com

Rule Path
Disallow /

gurujibot

Rule Path
Disallow /

snapbot

Rule Path
Disallow /

nwspider

Rule Path
Disallow /

dragonfly

Rule Path
Disallow /

jambot

Rule Path
Disallow /

kbeebot

Rule Path
Disallow /

digger

Rule Path
Disallow /

icu

Rule Path
Disallow /

gurujibot

Rule Path
Disallow /

blaiz-bee

Rule Path
Disallow /

gurujibot

Rule Path
Disallow /

webaroobot

Rule Path
Disallow /

rufusbot

Rule Path
Disallow /

favorstarbot

Rule Path
Disallow /

speedy

Rule Path
Disallow /

cfetch

Rule Path
Disallow /

pagebullbot

Rule Path
Disallow /

pulsebot

Rule Path
Disallow /

metatagsdir

Rule Path
Disallow /

yodaobot

Rule Path
Disallow /

twiceler

Rule Path
Disallow /

orangespider

Rule Path
Disallow /

vadixbot

Rule Path
Disallow /

netwhatcrawler

Rule Path
Disallow /

nimblecrawler

Rule Path
Disallow /

generic

Rule Path
Disallow /

sirketcebot

Rule Path
Disallow /

summizebot

Rule Path
Disallow /

gigamega

Rule Path
Disallow /

synoobot

Rule Path
Disallow /

litefinder

Rule Path
Disallow /

geonabot

Rule Path
Disallow /

spock

Rule Path
Disallow /

mojeekbot

Rule Path
Disallow /

sitecheck.internetseer.com

Rule Path
Disallow /

zealbot

Rule Path
Disallow /

msiecrawler

Rule Path
Disallow /

sitesnagger

Rule Path
Disallow /

webstripper

Rule Path
Disallow /

webcopier

Rule Path
Disallow /

fetch

Rule Path
Disallow /

offline explorer

Rule Path
Disallow /

teleport

Rule Path
Disallow /

teleportpro

Rule Path
Disallow /

webzip

Rule Path
Disallow /

linko

Rule Path
Disallow /

httrack

Rule Path
Disallow /

microsoft.url.control

Rule Path
Disallow /

xenu

Rule Path
Disallow /

larbin

Rule Path
Disallow /

libwww

Rule Path
Disallow /

zyborg

Rule Path
Disallow /

download ninja

Rule Path
Disallow /

xintellibot

Rule Path
Disallow /

trendictionbot

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

google-extended

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap https://travelcarsnz.com/sitemap.xml
Back to top

Comments

  • Allow for Google Bots
  • Some other bots are known to be trouble
Back to top