Log In Sign Up

famillypet.com
robots.txt

Robots Exclusion Standard data for famillypet.com

Resource Scan

Scan Details

Site Domain famillypet.com
Base Domain famillypet.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a server error.
Last Scan2024-10-05T12:43:40+00:00
Next Scan 2024-10-19T12:43:40+00:00

Last Successful Scan

Scanned2024-09-20T10:46:19+00:00
URL https://famillypet.com/robots.txt
Domain IPs 2406:da18:9d0:143f:2124:4e9c:36a9:d9de, 52.221.42.138
Response IP 52.221.42.138
Found Yes
Hash fec84f1787bead1efd31aac4874667e51199de5362c960404e0813ac8bfc367e
SimHash 521cf542e822

Groups

xovibot

Rule Path
Disallow /

wotbox

Rule Path
Disallow /

wbsearchbot

Rule Path
Disallow /

unisterbot

Rule Path
Disallow /

turnitinbot

Rule Path
Disallow /

splitsignalbot

Rule Path
Disallow /

spbot

Rule Path
Disallow /

sogou spider

Rule Path
Disallow /

smtbot

Rule Path
Disallow /

siteexplorer

Rule Path
Disallow /

siteauditbot

Rule Path
Disallow /

sistrix

Rule Path
Disallow /

seznambot

Rule Path
Disallow /

seokicks-robot

Rule Path
Disallow /

sentibot

Rule Path
Disallow /

semrushbot-swa

Rule Path
Disallow /

semrushbot-si

Rule Path
Disallow /

semrushbot-ct

Rule Path
Disallow /

semrushbot-coub

Rule Path
Disallow /

semrushbot-ba

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

searchmetricsbot

Rule Path
Disallow /

screenerbot

Rule Path
Disallow /

screaming frog seo spider

Rule Path
Disallow /

rogerbot

Rule Path
Disallow /

publiclibraryarchive.org

Rule Path
Disallow /

panscient.com

Rule Path
Disallow /

oncrawl

Rule Path
Disallow /

nutch

Rule Path
Disallow /

netestate ne crawler

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /wp-admin/
Disallow /

Other Records

Field Value
crawl-delay 10

mixbot

Rule Path
Disallow /

memorybot

Rule Path
Disallow /

megaindex.ru/2.0

Rule Path
Disallow /

megaindex.ru

Rule Path
Disallow /

meanpathbot

Rule Path
Disallow /

mail.ru

Rule Path
Disallow /

lipperhey

Rule Path
Disallow /

linkpadbot

Rule Path
Disallow /

istellabot

Rule Path
Disallow /

heritrix

Rule Path
Disallow /

googlebot

Rule Path
Disallow /wp-admin/

feedbooster

Rule Path
Disallow /

ezooms

Rule Path
Disallow /

exb language crawler

Rule Path
Disallow /

easouspider

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

cognitiveseo

Rule Path
Disallow /

bubing

Rule Path
Disallow /

bpimagewalker/2.0

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

advbot

Rule Path
Disallow /

aboundexbot

Rule Path
Disallow /

abonti

Rule Path
Disallow /

*

Rule Path
Disallow /wp-content/uploads/wpo/wpo-plugins-tables-list.json
Disallow /wp-admin/
Allow /wp-admin/admin-ajax.php

Other Records

Field Value
crawl-delay 10
Back to top

Other Records

Field Value
sitemap https://famillypet.com/sitemap_index.xml
Back to top