stories.saucyrecipes.com
robots.txt
Robots Exclusion Standard data for stories.saucyrecipes.com
Resource Scan
Scan Details
| Site Domain | stories.saucyrecipes.com |
| Base Domain | saucyrecipes.com |
| Scan Status | Failed |
| Failure Stage | Fetching resource. |
| Failure Reason | Server returned a client error. |
| Last Scan | 2024-06-03T06:54:23+00:00 |
| Next Scan | 2024-09-01T06:54:23+00:00 |
Last Successful Scan
| Scanned | 2023-08-09T06:10:31+00:00 |
| URL | https://stories.saucyrecipes.com/robots.txt |
| Domain IPs | 104.21.73.89, 172.67.189.84, 2606:4700:3034::6815:4959, 2606:4700:3035::ac43:bd54 |
| Response IP | 172.67.189.84 |
| Found | Yes |
| Hash | df50d7de0111970578f430467bef55a02cedac387b877986c12e58c63a599577 |
| SimHash | 7814b920f9a2 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /blog/static/ |
| Disallow | /blog/login.php |
| Disallow | /*/*sort%3D* |
| Disallow | /*/*/*sort%3D* |
| Disallow | /*/*/*/*sort%3D* |
| Disallow | /process.php |
aboundex
add catalog
adnormcrawler
ahrefsbot
ahrefsbot.feeds
aihitbot
archive.org_bot
babya discoverer
botify
barkrowler
bbbot
blexbot
botw spider
bubing
butterfly/1.0
catchbot
ccbot
cigiema spider
compspybot
contextad bot 1.0
covenant eyes
crowsnest
css certificate spider
cyveillance
datanyze
dataprovider/6.92
daumoa 4.0
discobot
discoverybot
exabot
ezooms/1.0
feedburner
feedfetcher-google
findlinks
findxbot
fuzz faster u fool
flightdeckreportsbot
goodbot
geedobot
genieo/1.0
gigablastopensource
grapeshot
hubspot webcrawler
indy library
js-kit url resolver
ip-web-crawler
istellabot
konqueror
kraken
linkdexbot/2.0
lipperhey link explorer
meanpathbot
megaindex.com
megaindex.ru
merchantcentricbot
metauri
mj12bot
moreoverbot
ncbot
nerdybot
netcraftsurveyagent
netseer crawler
nominet.org
panscient.com
paperlibot
peerindex
percolatecrawler
proximic
queryseekerspider
ravencrawler
relateiq crawler
rogerbot
searchmetrics
scoutjet
scoutjet
scrapy/2.7.1
scrapy
screenerbot crawler
semalt crawler
semantifire1/0.20
semrushbot
seokicks-robot
seznambot
shodan.io
showyoubot
sistrix crawler
siteexplorer
skimbot/1.0
socialbm_bot
sogou web spider
solomonobot
sosospider
spbot
strawberryj.am
surveybot
tweetedtimes
tweetmemebot/3.0
twitterbot/1.0
umbot-ln/1.0
underscorebot
unwindfetchor
uptimerobot/2.0
urlappendbot
voltron
wbsearchbot
wotbox
xovibot
yahoo pipes 1.0
yrspider
zmeu
zscaler
petalbot
trovitbot
| Rule | Path |
|---|---|
| Disallow | / |
Other Records
| Field | Value |
|---|---|
| sitemap | https://www.zfc.com/subsitemap.xml |
| sitemap | https://www.zfc.com/devsitemap.xml |
| sitemap | https://www.zfc.com/citysitemap.xml |
| sitemap | https://www.zfc.com/schoolsitemap.xml |
| sitemap | https://www.zfc.com/listingsitemap-SEFMIAMI.xml |
| sitemap | https://www.zfc.com/listingsitemap-FTL.xml |
| sitemap | https://www.zfc.com/agentsitemap.xml |
| sitemap | https://www.zfc.com/listingsitemap-Brevard-MLS.xml |
| sitemap | https://www.zfc.com/listingsitemap-NSBBOR.xml |
| sitemap | https://www.zfc.com/listingsitemap-mfrmls.xml |
| sitemap | https://www.zfc.com/listingsitemap-mfrmls2.xml |
Warnings
- 2 invalid lines.