findpricetoday.com
robots.txt

Robots Exclusion Standard data for findpricetoday.com

Archived Snapshots

Resource Scan

Scan Details

Site Domain	findpricetoday.com
Base Domain	findpricetoday.com
Scan Status	Ok
Last Scan	2024-09-28T23:23:09+00:00
Next Scan	2024-10-05T23:23:09+00:00

Last Scan

Scanned	2024-09-28T23:23:09+00:00
URL	https://findpricetoday.com/robots.txt
Domain IPs	104.21.4.38, 172.67.131.159, 2606:4700:3036::6815:426, 2606:4700:3036::ac43:839f
Response IP	172.67.131.159
Found	Yes
Hash	a54f44b34bc153b3856e664bca1418f22765acc7ef86ff9c3d3f5c20ba40105f
SimHash	e29c6e72cc1a

Groups

ahrefsbot

Rule	Path
Disallow	/

Rule

Path

Disallow

alphabot

Rule	Path
Disallow	/

Rule

Path

Disallow

baiduspider

Rule	Path
Disallow	/

Rule

Path

Disallow

baiduspider-render

Rule	Path
Disallow	/

Rule

Path

Disallow

buck

Rule	Path
Disallow	/

Rule

Path

Disallow

changedetection

Rule	Path
Disallow	/

Rule

Path

Disallow

cliqzbot

Rule	Path
Disallow	/

Rule

Path

Disallow

exabot

Rule	Path
Disallow	/

Rule

Path

Disallow

flamingo_searchengine

Rule	Path
Disallow	/

Rule

Path

Disallow

grobbot

Rule	Path
Disallow	/

Rule

Path

Disallow

jobboersebot

Rule	Path
Disallow	/

Rule

Path

Disallow

jobs.de-robot

Rule	Path
Disallow	/

Rule

Path

Disallow

linkdexbot

Rule	Path
Disallow	/

Rule

Path

Disallow

mail.ru_bot

Rule	Path
Disallow	/

Rule

Path

Disallow

mediatoolkitbot

Rule	Path
Disallow	/

Rule

Path

Disallow

megaindex.ru/2.0

Rule	Path
Disallow	/

Rule

Path

Disallow

mega-index

Rule	Path
Disallow	/

Rule

Path

Disallow

megaindex.ru

Rule	Path
Disallow	/

Rule

Path

Disallow

mj12bot

Rule	Path
Disallow	/

Rule

Path

Disallow

mojeekbot

Rule	Path
Disallow	/

Rule

Path

Disallow

pinterestbot

Rule	Path
Disallow	/

Rule

Path

Disallow

rogerbot

Rule	Path
Disallow	/

Rule

Path

Disallow

safednsbot

Rule	Path
Disallow	/

Rule

Path

Disallow

semrushbot

Rule	Path
Disallow	/

Rule

Path

Disallow

semrushbot/2~bl

Rule	Path
Disallow	/

Rule

Path

Disallow

semrushbot-ba

Rule	Path
Disallow	/

Rule

Path

Disallow

semrushbot-bm

Rule	Path
Disallow	/

Rule

Path

Disallow

semrushbot-sa

Rule	Path
Disallow	/

Rule

Path

Disallow

seokicks-robot

Rule	Path
Disallow	/

Rule

Path

Disallow

seznambot

Rule	Path
Disallow	/

Rule

Path

Disallow

smtbot

Rule	Path
Disallow	/

Rule

Path

Disallow

spbot

Rule	Path
Disallow	/

Rule

Path

Disallow

trendictionbot

Rule	Path
Disallow	/

Rule

Path

Disallow

tweetmemebot

Rule	Path
Disallow	/

Rule

Path

Disallow

vebidoobot

Rule	Path
Disallow	/

Rule

Path

Disallow

wonderbot

Rule	Path
Disallow	/

Rule

Path

Disallow

wotbox

Rule	Path
Disallow	/

Rule

Path

Disallow

yacybot

Rule	Path
Disallow	/

Rule

Path

Disallow

yak

Rule	Path
Disallow	/

Rule

Path

Disallow

yandexantivirus

Rule	Path
Disallow	/

Rule

Path

Disallow

yandexbot/3.0

Rule	Path
Disallow	/

Rule

Path

Disallow

yandeximages

Rule	Path
Disallow	/

Rule

Path

Disallow

yandexmobilebot

Rule	Path
Disallow	/

Rule

Path

Disallow

yeti

Rule	Path
Disallow	/

Rule

Path

Disallow

Other Records

Field	Value
sitemap	https://findpricetoday.com/sitemap_index.xml

Field

Value

sitemap

https://findpricetoday.com/sitemap_index.xml

findpricetoday.comrobots.txt

Resource Scan

Scan Details

Last Scan

Groups

ahrefsbot

alphabot

baiduspider

baiduspider-render

buck

changedetection

cliqzbot

exabot

flamingo_searchengine

grobbot

jobboersebot

jobs.de-robot

linkdexbot

mail.ru_bot

mediatoolkitbot

megaindex.ru/2.0

mega-index

megaindex.ru

mj12bot

mojeekbot

pinterestbot

rogerbot

safednsbot

semrushbot

semrushbot/2~bl

semrushbot-ba

semrushbot-bm

semrushbot-sa

seokicks-robot

seznambot

smtbot

spbot

trendictionbot

tweetmemebot

vebidoobot

wonderbot

wotbox

yacybot

yak

yandexantivirus

yandexbot/3.0

yandeximages

yandexmobilebot

yeti

Other Records

findpricetoday.com
robots.txt