itfenix.com
robots.txt

Robots Exclusion Standard data for itfenix.com

Resource Scan

Scan Details

Site Domain itfenix.com
Base Domain itfenix.com
Scan Status Ok
Last Scan2025-12-23T00:09:25+00:00
Next Scan 2026-01-22T00:09:25+00:00

Last Scan

Scanned2025-12-23T00:09:25+00:00
URL https://itfenix.com/robots.txt
Domain IPs 173.254.28.198
Response IP 173.254.28.198
Found Yes
Hash 1eeb3933cd7151a7d6e6a809458a223988d653658c2eed6f3a2a1c8fcba05e1a
SimHash 68165e65dc9d

Groups

*

Rule Path
Disallow /admin308xi1mmqijyhgscpqf/
Disallow /cgi-bin/
Disallow /include/
Disallow /includes/
Disallow /layout/
Disallow /noindex/
Disallow /perfil/vendedor/
Disallow /perfil/comprador/
Disallow /*/sid/
Disallow /*?sid=
Disallow /*%26sid%3D
Disallow /*?cur=
Disallow /*%26cur
Disallow /*?utm_source=
Disallow /*%26utm_source%3D
Disallow /*?ref=
Disallow /*%26ref%3D
Allow /

googlebot

Rule Path
Allow /

bingbot

Rule Path
Allow /

slurp

Rule Path
Allow /

duckduckbot

Rule Path
Allow /
Allow /css/
Allow /js/
Allow /fonts/
Allow /ads/preferences/
Allow /pagead/

baiduspider

Rule Path
Disallow /

yandex

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 10

semrushbot

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 10

dotbot

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 10

blexbot

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 10

petalbot

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 10

sogou

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 10

exabot

Rule Path
Disallow /

seznambot

Rule Path
Disallow /

coccocbot

Rule Path
Disallow /

aspiegelbot

Rule Path
Disallow /

cliqzbot

Rule Path
Disallow /

voilabot

Rule Path
Disallow /

wotbox

Rule Path
Disallow /

uptimerobot

Rule Path
Disallow /

barkrowler

Rule Path
Disallow /

mauibot

Rule Path
Disallow /

velenpublicwebcrawler

Rule Path
Disallow /

megaindex

Rule Path
Disallow /

seekportbot

Rule Path
Disallow /

webdatastatsbot

Rule Path
Disallow /

updownbot

Rule Path
Disallow /

scrapy

Rule Path
Disallow /

surveybot

Rule Path
Disallow /

zoominfobot

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

dataforseobot

Rule Path
Disallow /

mail.ru_bot

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

claudebot

Rule Path
Disallow /

twitterbot

Rule Path
Allow /

facebookexternalhit

Rule Path
Allow /

linkedinbot

Rule Path
Allow /

applebot

Rule Path
Allow /

pinterestbot

Rule Path
Allow /

telegrambot

Rule Path
Allow /

Other Records

Field Value
sitemap https://www.itfenix.com/sitemapcom.xml
sitemap https://www.itfenix.com/estore/1_index_sitemap.xml
sitemap https://www.it-fenix.mx/sitemap.xml

Comments

  • Permitir recursos necesarios
  • Permitir Google Ads y scripts útiles
  • Sitemaps
  • Bloqueo de bots no deseados
  • Permitir bots útiles de redes sociales