mamasguiderecipes.com
robots.txt

Robots Exclusion Standard data for mamasguiderecipes.com

Archived Snapshots

Resource Scan

Scan Details

Site Domain	mamasguiderecipes.com
Base Domain	mamasguiderecipes.com
Scan Status	Ok
Last Scan	2024-09-28T16:42:17+00:00
Next Scan	2024-10-05T16:42:17+00:00

Last Scan

Scanned	2024-09-28T16:42:17+00:00
URL	https://mamasguiderecipes.com/robots.txt
Domain IPs	192.124.249.18
Response IP	192.124.249.18
Found	Yes
Hash	f8ef370f4d6d56b83db7ec8b8442acfdf66018a06aabd58f9be11578080982e7
SimHash	a18c48e26c93

Groups

ahrefsbot

Rule	Path
Disallow	/

Rule

Path

Disallow

blexbot

Rule	Path
Disallow	/

Rule

Path

Disallow

chrome privacy preserving prefetch proxy

Rule	Path
Disallow	/

Rule

Path

Disallow

clickagy intelligence bot v2

Rule	Path
Disallow	/

Rule

Path

Disallow

criteobot/0.1 (+https://www.criteo.com/criteo-crawler/)

Rule	Path
Disallow	/

Rule

Path

Disallow

dalvik/2.1.0

Rule	Path
Disallow	/

Rule

Path

Disallow

dalvik/2.1.0 linux; u; android 11; sm-a505u build/rp1a.200720.012

Rule	Path
Disallow	/

Rule

Path

Disallow

dalvik/2.1.0 linux; u; android 9.0; zte ba520 build/mra58k

Rule	Path
Disallow	/

Rule

Path

Disallow

grapeshotcrawler

Rule	Path
Disallow	/

Rule

Path

Disallow

mj12bot

Rule	Path
Disallow	/

Rule

Path

Disallow

maxpointcrawler/nutch-1.17 valassis.crawler at valassis dot com

Rule	Path
Disallow	/

Rule

Path

Disallow

photon/1.0

Rule	Path
Disallow	/

Rule

Path

Disallow

semrushbot

Rule	Path
Disallow	/

Rule

Path

Disallow

yahoo

Rule	Path
Disallow	/

Rule

Path

Disallow

yandexbot

Rule	Path
Disallow	/

Rule

Path

Disallow

bingbot

Rule	Path
Disallow	/

Rule

Path

Disallow

bingbot

Rule	Path
Disallow	/

Rule

Path

Disallow

compatible; dataforseobot/1.0; https://dataforseo.com/dataforseo-bot

Rule	Path
Disallow	/

Rule

Path

Disallow

https://awario.com/bots.html

Rule	Path
Disallow	/

Rule

Path

Disallow

https://babbar.tech/crawler

Rule	Path
Disallow	/

Rule

Path

Disallow

https://integralads.com/about-ias/

Rule	Path
Disallow	/

Rule

Path

Disallow

https://opensiteexplorer.org/dotbot

Rule	Path
Disallow	/

Rule

Path

Disallow

https://www.comscore.com/web-crawler

Rule	Path
Disallow	/

Rule

Path

Disallow

https://www.criteo.com/criteo-crawler/

Rule	Path
Disallow	/

Rule

Path

Disallow

https://www.mojeek.com/bot.html

Rule	Path
Disallow	/

Rule

Path

Disallow

https://www.thetradedesk.com/general/ttd-content

Rule	Path
Disallow	/

Rule

Path

Disallow

https://zhanzhang.toutiao.com/

Rule	Path
Disallow	/

Rule

Path

Disallow

proximic

Rule	Path
Disallow	/

Rule

Path

Disallow

python-requests/2.24.0

Rule	Path
Disallow	/

Rule

Path

Disallow

*

Rule	Path
Disallow	/wp-admin/network/
Disallow	/wp-content/plugins/
Disallow	/wp-login

Rule

Path

Disallow

/wp-admin/network/

Disallow

/wp-content/plugins/

Disallow

/wp-login

Other Records

Field	Value
crawl-delay	10.0

Field

Value

crawl-delay

10.0

Warnings

2 invalid lines.

mamasguiderecipes.comrobots.txt

Resource Scan

Scan Details

Last Scan

Groups

ahrefsbot

blexbot

chrome privacy preserving prefetch proxy

clickagy intelligence bot v2

criteobot/0.1 (+https://www.criteo.com/criteo-crawler/)

dalvik/2.1.0

dalvik/2.1.0 linux; u; android 11; sm-a505u build/rp1a.200720.012

dalvik/2.1.0 linux; u; android 9.0; zte ba520 build/mra58k

grapeshotcrawler

mj12bot

maxpointcrawler/nutch-1.17 valassis.crawler at valassis dot com

photon/1.0

semrushbot

yahoo

yandexbot

bingbot

bingbot

compatible; dataforseobot/1.0; https://dataforseo.com/dataforseo-bot

https://awario.com/bots.html

https://babbar.tech/crawler

https://integralads.com/about-ias/

https://opensiteexplorer.org/dotbot

https://www.comscore.com/web-crawler

https://www.criteo.com/criteo-crawler/

https://www.mojeek.com/bot.html

https://www.thetradedesk.com/general/ttd-content

https://zhanzhang.toutiao.com/

proximic

python-requests/2.24.0

*

Other Records

Warnings

mamasguiderecipes.com
robots.txt