Log In Sign Up

maisfontes.com
robots.txt

Robots Exclusion Standard data for maisfontes.com

Resource Scan

Scan Details

Site Domain maisfontes.com
Base Domain maisfontes.com
Scan Status Ok
Last Scan2024-11-08T12:09:48+00:00
Next Scan 2024-11-15T12:09:48+00:00

Last Scan

Scanned2024-11-08T12:09:48+00:00
URL https://maisfontes.com/robots.txt
Domain IPs 2406:da18:9d0:143f:2124:4e9c:36a9:d9de, 52.221.42.138
Response IP 52.221.42.138
Found Yes
Hash b03bd81617b11d1a74b45c74c9c833dc148678d8a462acbbe77f98738c7d4f85
SimHash 2c5e5111ff4d

Groups

*

Rule Path
Disallow /*.go$
Disallow /uploads/images/temp/*
Disallow /download/
Disallow /email/
Disallow /admin/
Disallow /fonts/
Disallow /tempfiles/
Disallow /humix/
Allow /

Other Records

Field Value
crawl-delay 1

yandex
yandexbot/3.0

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 2

petalbot

Rule Path
Disallow /

ias_crawler
ias_wombles

Rule Path
Disallow /

dataforseobot

Rule Path
Disallow /

velenpublicwebcrawler

Rule Path
Disallow /

criteobot/0.1

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

grapeshotcrawler

Rule Path
Disallow /

grapeshot

Rule Path
Disallow /

demandbasepublisheranalyzer

Rule Path
Disallow /

amazonbot

Rule Path
Disallow /

amazonbot/0.1

Rule Path
Disallow /

webprosbot

Rule Path
Disallow /

fmmalithi-x

Rule Path
Disallow /

demandbasepublisheranalyzer

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

zoominfobot

Rule Path
Disallow /

barkrowler

Rule Path
Disallow /

datanyze

Rule Path
Disallow /

yisouspider

Rule Path
Disallow /

adscanner

Rule Path
Disallow /

zgrab

Rule Path
Disallow /

sogou spider

Rule Path
Disallow /

blekkobot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

coccocbot-web

Rule Path
Disallow /

qwarrybot

Rule Path
Disallow /

gumgum
gumgum-bot

Rule Path
Disallow /

seekport crawler

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

mj12bot
mj12bot/v1.4.8

Rule Path
Disallow /

adsbot
adsbot/3.1

Rule Path
Disallow /

dotbot

Rule Path
Disallow /
Back to top

Comments

  • --------
  • Yandex Parms
  • --------
  • Bing
  • --------
  • if need to allow the access in MaisFontes, contact us: support@maisfontes.com
  • --------
  • https://www.admantx.com/service-fetcher.html
  • User-agent: ias-ir
  • User-agent: ias-sg
  • User-agent: ias-or
  • User-agent: ias-va
  • User-agent: ias-jp
  • User-agent: ias-sg
  • User-agent: ADmantX
  • Disallow: /
  • Petal from petalbot@huawei.com
  • https://integralads.com/ias-privacy-data-management/policies/site-indexing-policy/
  • User-agent: Genieo
  • Disallow: /
  • User-agent: Genieo/1.0
  • Disallow: /
  • User-Agent: The Knowledge AI
  • Disallow: /
  • User-agent: Seobility
  • Disallow: /
  • http://help.coccoc.com/searchengine
  • http://www.qwarry.com/bot.html
  • verity-support@gumgum.com
  • http://seekport.com
  • https://commoncrawl.org/big-picture/frequently-asked-questions/
  • http://mj12bot.com/
  • https://seostar.co/robot/
  • https://moz.com/help/moz-procedures/crawlers/dotbot
  • https://www.comscore.com/web-crawler
  • User-agent: proximic
  • Disallow: /
  • http://megaindex.com/crawler
  • User-agent: megaindex.ru
  • User-agent: megaindex.com
  • Disallow: /
  • Ezoic
  • User-agent: EzLynx/0.1
  • Disallow: /
Back to top

Warnings

  • `clean-param` is not a known field.
Back to top