nz.locale.online
robots.txt

Robots Exclusion Standard data for nz.locale.online

Resource Scan

Scan Details

Site Domain nz.locale.online
Base Domain locale.online
Scan Status Ok
Last Scan2024-05-17T20:49:53+00:00
Next Scan 2024-06-16T20:49:53+00:00

Last Scan

Scanned2024-05-17T20:49:53+00:00
URL https://nz.locale.online/robots.txt
Domain IPs 104.26.10.76, 104.26.11.76, 172.67.73.30, 2606:4700:20::681a:a4c, 2606:4700:20::681a:b4c, 2606:4700:20::ac43:491e
Response IP 172.67.73.30
Found Yes
Hash c96c0efbac62721ad35e0ea298f1cd3ffa721dcd0bab6140a99594cf7f9a905c
SimHash 6e4e4a9072bb

Groups

*

Rule Path
Disallow /ajax.php
Disallow /error.html
Disallow /*/add-review.html
Disallow /*/send-message.html

mediapartners-google

Rule Path
Disallow

Other Records

Field Value
crawl-delay 1

googlebot

Rule Path
Disallow

Other Records

Field Value
crawl-delay 1

adsbot-google

Rule Path
Disallow

Other Records

Field Value
crawl-delay 1

googlebot-image

Rule Path
Disallow

Other Records

Field Value
crawl-delay 5

googlebot-mobile

Rule Path
Disallow

Other Records

Field Value
crawl-delay 1

googleother

Rule Path
Disallow

Other Records

Field Value
crawl-delay 5

msnbot

Rule Path
Disallow

Other Records

Field Value
crawl-delay 20

bingbot

Rule Path
Disallow

Other Records

Field Value
crawl-delay 5

slurp

Rule Path
Disallow

Other Records

Field Value
crawl-delay 360

yahoo! slurp

Rule Path
Disallow

Other Records

Field Value
crawl-delay 360

yandex

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

wasalive-bot

Rule Path
Disallow

Other Records

Field Value
crawl-delay 180

kulturarw3

Rule Path
Disallow

Other Records

Field Value
crawl-delay 180

facebookexternalhit

Rule Path
Disallow

Other Records

Field Value
crawl-delay 8

ahrefsbot

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 360

mj12bot

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 360

blexbot

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 360

proximic

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 360

seokicks-robot

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 360

admantx

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 360

dotbot

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 360

wiederfreibot

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 360

cliqzbot

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 360

grapeshotcrawler

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 360

seznambot

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 360

coccocbot

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 100