Log In Sign Up

foodme.co.nz
robots.txt

Robots Exclusion Standard data for foodme.co.nz

Resource Scan

Scan Details

Site Domain foodme.co.nz
Base Domain foodme.co.nz
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2024-08-14T01:55:23+00:00
Next Scan 2024-11-12T01:55:23+00:00

Last Successful Scan

Scanned2024-04-17T01:11:40+00:00
URL https://foodme.co.nz/robots.txt
Domain IPs 104.21.47.244, 172.67.174.143, 2606:4700:3032::6815:2ff4, 2606:4700:3033::ac43:ae8f
Response IP 172.67.174.143
Found Yes
Hash 084063f6a01f328fa13024a89487e4d46deedb9a482b96520a3abf1f6f0eeeca
SimHash aa9c508ac6bf

Groups

*

Rule Path
Disallow /*/*%7D%7D$
Disallow /*.axd$
Disallow /shops-nearby
Disallow /mycart*
Disallow /privacypolicy
Disallow /Feedback?*
Disallow /blog/wp-login.php*
Disallow /RetailerReview/
Disallow /.axd$
Disallow /JudgementLogin
Disallow /JudgementLogin*
Disallow /GetCatalogfav
Disallow /GetProductfav
Disallow /GetProductList
Disallow /consumer/feed
Disallow /consumer/wp-admin/
Disallow /money/termdepositspostback.aspx*
Disallow /money/savingsaccountspostback.aspx*
Disallow /Compare.aspx*

blexbot

Rule Path
Disallow /

applebot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

seznambot

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

teemer

Rule Path
Disallow /

netseer

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

uptimerobot

Rule Path
Disallow /

domaincrawler

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

eurobot

Rule Path
Disallow /

msiecrawler

Rule Path
Disallow /

webreaper

Rule Path
Disallow /

wget

Rule Path
Disallow /

omniexplorer_bot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

ia_archiver

Rule Path
Disallow /

onet.pl

Rule Path
Disallow /

yandex

Rule Path
Disallow /

gigabot

Rule Path
Disallow /

shopwiki

Rule Path
Disallow /

psbot

Rule Path
Disallow /

ia_archiver

Rule Path
Disallow /

ichiro

Rule Path
Disallow /

cyberpatrol

Rule Path
Disallow /

houxoucrawler

Rule Path
Disallow /

twiceler

Rule Path
Disallow /

turnitinbot

Rule Path
Disallow /

spbot

Rule Path
Disallow /

almaden

Rule Path
Disallow /

aspseek

Rule Path
Disallow /

dumbbot

Rule Path
Disallow /

generic

Rule Path
Disallow /

grub-client

Rule Path
Disallow /

nexabot

Rule Path
Disallow /

npbot

Rule Path
Disallow /

owr_crawler

Rule Path
Disallow /

rabaz

Rule Path
Disallow /

rpt-httpclient

Rule Path
Disallow /

scoutabout

Rule Path
Disallow /

semanticdiscovery

Rule Path
Disallow /

cherry picker

Rule Path
Disallow /

crescent internet toolpak

Rule Path
Disallow /

disco pump

Rule Path
Disallow /

szukacz

Rule Path
Disallow /

teleport

Rule Path
Disallow /

telesoft

Rule Path
Disallow /

ultraseek

Rule Path
Disallow /

vspider

Rule Path
Disallow /

webcopier

Rule Path
Disallow /

webemailextractor

Rule Path
Disallow /

webster pro

Rule Path
Disallow /

webstripper

Rule Path
Disallow /

israbot

Rule Path
Disallow

orthogaffe

Rule Path
Disallow

ubicrawler

Rule Path
Disallow /

doc

Rule Path
Disallow /

zao

Rule Path
Disallow /

internetseer.com

Rule Path
Disallow /

sitecheck.internetseer.com

Rule Path
Disallow /

zealbot

Rule Path
Disallow /

sitesnagger

Rule Path
Disallow /

webzip

Rule Path
Disallow /

linko

Rule Path
Disallow /

httrack

Rule Path
Disallow /

microsoft.url.control

Rule Path
Disallow /

xenu

Rule Path
Disallow /

larbin

Rule Path
Disallow /

libwww

Rule Path
Disallow /

zyborg

Rule Path
Disallow /

download ninja

Rule Path
Disallow /

grub

Rule Path
Disallow /

k2spider

Rule Path
Disallow /

emailsiphon

Rule Path
Disallow /

htdig

Rule Path
Disallow /

lnspiderguy

Rule Path
Disallow /

netprospector

Rule Path
Disallow /

perman surfer

Rule Path
Disallow /

fetch

Rule Path
Disallow /

offline explorer

Rule Path
Disallow /

teleportpro

Rule Path
Disallow /

yahooseeker

Rule Path
Disallow /

becomebot

Rule Path
Disallow /

e-societyrobot

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap https://foodme.co.nz/sitemap/SiteMap.xml
Back to top

Comments

  • file for https://foodme.co.nz
Back to top