Log In Sign Up

tomatorate.com
robots.txt

Robots Exclusion Standard data for tomatorate.com

Resource Scan

Scan Details

Site Domain tomatorate.com
Base Domain tomatorate.com
Scan Status Ok
Last Scan2024-10-25T22:17:49+00:00
Next Scan 2024-11-24T22:17:49+00:00

Last Scan

Scanned2024-10-25T22:17:49+00:00
URL https://tomatorate.com/robots.txt
Redirect https://www.tomatorate.com/robots.txt
Redirect Domain www.tomatorate.com
Redirect Base tomatorate.com
Domain IPs 104.26.4.193, 104.26.5.193, 172.67.69.84, 2606:4700:20::681a:4c1, 2606:4700:20::681a:5c1, 2606:4700:20::ac43:4554
Redirect IPs 104.26.4.193, 104.26.5.193, 172.67.69.84, 2606:4700:20::681a:4c1, 2606:4700:20::681a:5c1, 2606:4700:20::ac43:4554
Response IP 172.67.69.84
Found Yes
Hash 0307cff6d94b631bc234e1e023c8b58e90a8725d255ccc135867f053bf2a9e66
SimHash b8149d114674

Groups

petalbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

adcore

Rule Path
Disallow /

ipvanish

Rule Path
Disallow /

requestedoffers.clu

Rule Path
Disallow /

notarius-russia.ru

Rule Path
Disallow /

bitminer.it

Rule Path
Disallow /

secretsharinghub.com

Rule Path
Disallow /

ibsconsultant.com

Rule Path
Disallow /

generositycool.com

Rule Path
Disallow /

ioflood.com

Rule Path
Disallow /

shootorder.com

Rule Path
Disallow /

visualpleasures.net

Rule Path
Disallow /

ninja-crawler

Rule Path
Disallow /

audit

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

opensiteexplorer.org

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

go-resty

Rule Path
Disallow /

crownpeak

Rule Path
Disallow /

clearscope.io

Rule Path
Disallow /

seopowersuite

Rule Path
Disallow /

webceo

Rule Path
Disallow /

cognitiveseo

Rule Path
Disallow /

netpeaksoftware

Rule Path
Disallow /

majestic.com

Rule Path
Disallow /

agencyanalytics

Rule Path
Disallow /

netpeaksoftware

Rule Path
Disallow /

brightedge

Rule Path
Disallow /

raventools

Rule Path
Disallow /

conductor.com

Rule Path
Disallow /

bingbot

Rule Path
Disallow /

seranking

Rule Path
Disallow /

ispionage

Rule Path
Disallow /

similarweb

Rule Path
Disallow /

buzzsumo

Rule Path
Disallow /

sproutsocial

Rule Path
Disallow /

siteimprove.com

Rule Path
Disallow /

ahrefs.com

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

webmeup.com

Rule Path
Disallow /

webmeup-crawler.com

Rule Path
Disallow /

*

Rule Path
Allow /
Back to top

Other Records

Field Value
sitemap https://www.tomatorate.com/sitemap.xml
Back to top

Comments

  • robots.txt
  • This file is to prevent the crawling and indexing of certain parts
  • of your site by web crawlers and spiders run by sites like Yahoo!
  • and Google. By telling these "robots" where not to go on your site,
  • you save bandwidth and server resources.
  • This file will be ignored unless it is at the root of your host:
  • Used: http://example.com/robots.txt
  • Ignored: http://example.com/site/robots.txt
  • For more information about the robots.txt standard, see:
  • http://www.robotstxt.org/robotstxt.html
Back to top

Warnings

  • 2 invalid lines.
Back to top