Log In Sign Up

priceme.co.nz
robots.txt

Robots Exclusion Standard data for priceme.co.nz

Resource Scan

Scan Details

Site Domain priceme.co.nz
Base Domain priceme.co.nz
Scan Status Ok
Last Scan2024-11-11T02:28:22+00:00
Next Scan 2024-11-18T02:28:22+00:00

Last Scan

Scanned2024-11-11T02:28:22+00:00
URL https://priceme.co.nz/robots.txt
Redirect https://www.priceme.co.nz/robots.txt
Redirect Domain www.priceme.co.nz
Redirect Base priceme.co.nz
Domain IPs 172.66.40.64, 172.66.43.192, 2606:4700:3108::ac42:2840, 2606:4700:3108::ac42:2bc0
Redirect IPs 172.66.40.64, 172.66.43.192, 2606:4700:3108::ac42:2840, 2606:4700:3108::ac42:2bc0
Response IP 172.66.43.192
Found Yes
Hash 5008dd5ba344d4f7e8fbf834efdc78203eba1d9ad550d781ec711cde61cc8978
SimHash e81c50dbce95

Groups

*

Rule Path
Disallow /*/*%7D%7D$
Disallow /*.axd$
Disallow /shops-nearby
Disallow /mycart*
Disallow /privacypolicy
Disallow /Feedback?*
Disallow /blog/wp-login.php*
Disallow /RetailerReview/
Disallow /.axd$
Disallow /JudgementLogin
Disallow /JudgementLogin*
Disallow /GetCatalogfav
Disallow /GetProductfav
Disallow /GetProductList
Disallow /consumer/feed
Disallow /consumer/wp-admin/
Disallow /money/termdepositspostback.aspx*
Disallow /money/savingsaccountspostback.aspx*
Disallow /Compare.aspx*

blexbot

Rule Path
Disallow /

applebot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

seznambot

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

teemer

Rule Path
Disallow /

netseer

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

uptimerobot

Rule Path
Disallow /

domaincrawler

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

eurobot

Rule Path
Disallow /

msiecrawler

Rule Path
Disallow /

webreaper

Rule Path
Disallow /

wget

Rule Path
Disallow /

omniexplorer_bot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

ia_archiver

Rule Path
Disallow /

onet.pl

Rule Path
Disallow /

yandex

Rule Path
Disallow /

gigabot

Rule Path
Disallow /

shopwiki

Rule Path
Disallow /

psbot

Rule Path
Disallow /

ia_archiver

Rule Path
Disallow /

ichiro

Rule Path
Disallow /

cyberpatrol

Rule Path
Disallow /

houxoucrawler

Rule Path
Disallow /

twiceler

Rule Path
Disallow /

turnitinbot

Rule Path
Disallow /

spbot

Rule Path
Disallow /

almaden

Rule Path
Disallow /

aspseek

Rule Path
Disallow /

dumbbot

Rule Path
Disallow /

generic

Rule Path
Disallow /

grub-client

Rule Path
Disallow /

nexabot

Rule Path
Disallow /

npbot

Rule Path
Disallow /

owr_crawler

Rule Path
Disallow /

rabaz

Rule Path
Disallow /

rpt-httpclient

Rule Path
Disallow /

scoutabout

Rule Path
Disallow /

semanticdiscovery

Rule Path
Disallow /

cherry picker

Rule Path
Disallow /

crescent internet toolpak

Rule Path
Disallow /

disco pump

Rule Path
Disallow /

szukacz

Rule Path
Disallow /

teleport

Rule Path
Disallow /

telesoft

Rule Path
Disallow /

ultraseek

Rule Path
Disallow /

vspider

Rule Path
Disallow /

webcopier

Rule Path
Disallow /

webemailextractor

Rule Path
Disallow /

webster pro

Rule Path
Disallow /

webstripper

Rule Path
Disallow /

israbot

Rule Path
Disallow

orthogaffe

Rule Path
Disallow

ubicrawler

Rule Path
Disallow /

doc

Rule Path
Disallow /

zao

Rule Path
Disallow /

internetseer.com

Rule Path
Disallow /

sitecheck.internetseer.com

Rule Path
Disallow /

zealbot

Rule Path
Disallow /

sitesnagger

Rule Path
Disallow /

webzip

Rule Path
Disallow /

linko

Rule Path
Disallow /

httrack

Rule Path
Disallow /

microsoft.url.control

Rule Path
Disallow /

xenu

Rule Path
Disallow /

larbin

Rule Path
Disallow /

libwww

Rule Path
Disallow /

zyborg

Rule Path
Disallow /

download ninja

Rule Path
Disallow /

grub

Rule Path
Disallow /

k2spider

Rule Path
Disallow /

emailsiphon

Rule Path
Disallow /

htdig

Rule Path
Disallow /

lnspiderguy

Rule Path
Disallow /

netprospector

Rule Path
Disallow /

perman surfer

Rule Path
Disallow /

fetch

Rule Path
Disallow /

offline explorer

Rule Path
Disallow /

teleportpro

Rule Path
Disallow /

yahooseeker

Rule Path
Disallow /

becomebot

Rule Path
Disallow /

e-societyrobot

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

amazonbot

Rule Path
Disallow /

facebookbot

Rule Path
Disallow /

anthropic-ai

Rule Path
Disallow /

claude-web

Rule Path
Disallow /

cohere-ai

Rule Path
Disallow /

omgilibot

Rule Path
Disallow /

omgili

Rule Path
Disallow /

perplexitybot

Rule Path
Disallow /

youbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

diffbot

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap https://www.priceme.co.nz/sitemap/SiteMap.xml
Back to top

Comments

  • file for https://www.priceme.co.nz
  • disallow rules for AI crawlers (ChatGPT, Claude etc.)
Back to top