hevo.io
robots.txt

Robots Exclusion Standard data for hevo.io

Resource Scan

Scan Details

Site Domain hevo.io
Base Domain hevo.io
Scan Status Ok
Last Scan2024-11-17T20:55:28+00:00
Next Scan 2024-12-01T20:55:28+00:00

Last Scan

Scanned2024-11-17T20:55:28+00:00
URL https://hevo.io/robots.txt
Redirect https://hevodata.com/robots.txt
Redirect Domain hevodata.com
Redirect Base hevodata.com
Domain IPs 3.0.196.27, 54.251.62.204
Redirect IPs 13.35.238.3, 13.35.238.36, 13.35.238.60, 13.35.238.76
Response IP 13.35.238.76
Found Yes
Hash 1e5e8965270fbb2d59c876a6976d7efeece11c0b6ade99dfe5520d645f525727
SimHash 9845ca73c965

Groups

*

Rule Path
Allow /
Disallow /wp-admin/
Disallow /login/
Disallow /learn/tag/*
Disallow /page/*
Disallow /learn/?p=
Disallow /learn/*/?q=
Disallow /learn/*/?nonamp=
Disallow /blog/?p=
Disallow /blog/*/?q=
Disallow /blog/*/?nonamp=
Disallow /blog/tag/*
Disallow /plugins*
Disallow /learn/*/feed/
Disallow /blog/*/feed/
Disallow /learnv2/*
Disallow /blogv2/*
Disallow /auth/*
Disallow /hevo-utility/*
Disallow /.json
Disallow /news*
Disallow /*.pdf$
Disallow /GSP/*
Disallow /landing/
Disallow /whitelabel/
Disallow /SiteUnderMaintaince.htm
Disallow /*com_comment*
Disallow /cache/
Disallow /cli/
Disallow /components/
Disallow /includes/
Disallow /installation/
Disallow /language/
Disallow /libraries/
Disallow /logs/
Disallow /media/
Disallow /modules/
Disallow /chat/

teleport

Rule Path
Disallow /

teleportpro

Rule Path
Disallow /

emailcollector

Rule Path
Disallow /

emailsiphon

Rule Path
Disallow /

webbandit

Rule Path
Disallow /

webzip

Rule Path
Disallow /

webreaper

Rule Path
Disallow /

webstripper

Rule Path
Disallow /

web downloader

Rule Path
Disallow /

webcopier

Rule Path
Disallow /

offline explorer pro

Rule Path
Disallow /

httrack website copier

Rule Path
Disallow /

offline commander

Rule Path
Disallow /

leech

Rule Path
Disallow /

websnake

Rule Path
Disallow /

blackwidow

Rule Path
Disallow /

http weazel

Rule Path
Disallow /
Disallow /archived/*
Disallow /amp/*

twitterbot

Rule Path
Disallow *

sogou web spider

Rule Path
Disallow /

sogou inst spider

Rule Path
Disallow /

sogou spider2

Rule Path
Disallow /

germcrawler

Rule Path
Disallow /

sogou

Rule Path
Disallow /

Other Records

Field Value
sitemap https://hevodata.com/sitemap.xml
sitemap https://hevodata.com/learn/sitemap_index.xml
sitemap https://hevodata.com/blog/sitemap_index.xml
sitemap https://docs.hevodata.com/sitemap.xml

Comments

  • Global rules for all user-agents
  • Disallow specific URLs related to the "blog" and "learn" section
  • disallow rules for django backend API
  • Other specific disallow rules
  • Add other specific disallow rules here if needed
  • Specific rules for certain user-agents
  • Sitemap locations