perfectdiary.com
robots.txt

Robots Exclusion Standard data for perfectdiary.com

Resource Scan

Scan Details

Site Domain perfectdiary.com
Base Domain perfectdiary.com
Scan Status Ok
Last Scan2024-10-01T08:50:08+00:00
Next Scan 2024-10-15T08:50:08+00:00

Last Scan

Scanned2024-10-01T08:50:08+00:00
URL https://perfectdiary.com/robots.txt
Redirect https://www.perfectdiary.com/robots.txt
Redirect Domain www.perfectdiary.com
Redirect Base perfectdiary.com
Domain IPs 23.227.38.64
Redirect IPs 23.227.38.74, 2620:127:f00f:e::
Response IP 23.227.38.74
Found Yes
Hash b802545286eeb313a055a8a0bb32a7403208eb23b624d95cabda5a34442b355a
SimHash f571dffe9618

Groups

*

Rule Path
Disallow /admin
Disallow /cart
Disallow /orders
Disallow /checkouts/
Disallow /checkout
Disallow /36991107117/checkouts
Disallow /36991107117/orders
Disallow /carts
Disallow /account
Disallow /collections/*sort_by*
Disallow /*/collections/*sort_by*
Disallow /collections/*%2B*
Disallow /collections/*%2B*
Disallow /collections/*%2B*
Disallow /*/collections/*%2B*
Disallow /*/collections/*%2B*
Disallow /*/collections/*%2B*
Disallow /blogs/*%2B*
Disallow /blogs/*%2B*
Disallow /blogs/*%2B*
Disallow /*/blogs/*%2B*
Disallow /*/blogs/*%2B*
Disallow /*/blogs/*%2B*
Disallow /*?*oseid=*
Disallow /*preview_theme_id*
Disallow /*preview_script_id*
Disallow /*/*?*ls=*&ls=*
Disallow /*/*?*ls%3D*%3Fls%3D*
Disallow /*/*?*ls%3D*%3Fls%3D*
Disallow /search
Disallow /apple-app-site-association
Disallow /collections/*/products/*
Disallow /index.php?c=category&id=*
Disallow /password
Disallow /apps/sap/t/*
Disallow /recommendations/products?*
Disallow /collections/types?q=*
Disallow /collections/vendors?q*
Disallow /web-pixels-manager*

adsbot-google

Rule Path
Disallow /checkouts/
Disallow /checkout
Disallow /carts
Disallow /orders
Disallow /36991107117/checkouts
Disallow /36991107117/orders
Disallow /*?*oseid=*
Disallow /*preview_theme_id*
Disallow /*preview_script_id*

ahrefsbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

ahrefssiteaudit

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

mj12bot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

pinterest

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 1

baiduspider

Rule Path
Disallow /

baiduspider-ads

Rule Path
Disallow /

almaden

Rule Path
Disallow /

aspseek

Rule Path
Disallow /

axmo

Rule Path
Disallow /

booch

Rule Path
Disallow /

dts agent

Rule Path
Disallow /

downloader

Rule Path
Disallow /

emailcollector

Rule Path
Disallow /

emailsiphon

Rule Path
Disallow /

emailwolf

Rule Path
Disallow /

expired domain sleuth

Rule Path
Disallow /

franklin locator

Rule Path
Disallow /

gaisbot

Rule Path
Disallow /

grub

Rule Path
Disallow /

hughcrawler

Rule Path
Disallow /

iaea.org

Rule Path
Disallow /

lcabotaccept

Rule Path
Disallow /

iconsurf

Rule Path
Disallow /

iltrovatore-setaccio

Rule Path
Disallow /

indy library

Rule Path
Disallow /

iupui

Rule Path
Disallow /

kittiecentral

Rule Path
Disallow /

iaea.org

Rule Path
Disallow /

larbin

Rule Path
Disallow /

lwp-trivial

Rule Path
Disallow /

metatagrobot

Rule Path
Disallow /

missigua locator

Rule Path
Disallow /

netresearchserver

Rule Path
Disallow /

nextgensearch

Rule Path
Disallow /

npbot

Rule Path
Disallow /

nutch

Rule Path
Disallow /

objectssearch

Rule Path
Disallow /

oracle ultra search

Rule Path
Disallow /

peerbot

Rule Path
Disallow /

pictureofinternet

Rule Path
Disallow /

plantynet

Rule Path
Disallow /

quepasacreep

Rule Path
Disallow /

scspider

Rule Path
Disallow /

soft411

Rule Path
Disallow /

spider.acont.de

Rule Path
Disallow /

sqworm

Rule Path
Disallow /

ssm agent

Rule Path
Disallow /

tamu

Rule Path
Disallow /

theusefulbot

Rule Path
Disallow /

turnitinbot

Rule Path
Disallow /

tutorial crawler

Rule Path
Disallow /

tutorgig

Rule Path
Disallow /

webcopier

Rule Path
Disallow /

webzip

Rule Path
Disallow /

zipppbot

Rule Path
Disallow /

wotbox

Rule Path
Disallow /

wget

Rule Path
Disallow /

mozdex

Rule Path
Disallow /

zoominfobot

Rule Path
Disallow /

spyglassbot

Rule Path
Disallow /

seznambot

Rule Path
Disallow /

nimbostratus-bot

Rule Path
Disallow /

digext

Rule Path
Disallow /

cincraw

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

Other Records

Field Value
sitemap https://www.perfectdiary.com/sitemap.xml

Comments

  • Google adsbot ignores robots.txt unless specifically named!