covidhealth.com
robots.txt

Robots Exclusion Standard data for covidhealth.com

Resource Scan

Scan Details

Site Domain covidhealth.com
Base Domain covidhealth.com
Scan Status Ok
Last Scan2024-10-04T12:24:01+00:00
Next Scan 2024-10-11T12:24:01+00:00

Last Scan

Scanned2024-10-04T12:24:01+00:00
URL https://covidhealth.com/robots.txt
Redirect https://www.covidhealth.com/robots.txt
Redirect Domain www.covidhealth.com
Redirect Base covidhealth.com
Domain IPs 3.94.28.44, 44.209.222.27
Redirect IPs 3.94.28.44, 44.209.222.27
Response IP 44.209.222.27
Found Yes
Hash 3b4a9747ffe88bd99943fda693a61f8806e79c18a13c14fb825b50cafec84f03
SimHash 7b4f5851ccb0

Groups

*

Rule Path
Disallow */get-pdf

Other Records

Field Value
crawl-delay 12

mj12bot

Rule Path
Disallow */get-pdf

Other Records

Field Value
crawl-delay 20

awariorssbot
awariosmartbot
blexbot
dataforseobot
dotbot
panscient.com
petalbot
turnitinbot

Rule Path
Disallow */get-pdf

Other Records

Field Value
crawl-delay 16

ahrefsbot
ccbot
claudebot
daum
friendlycrawler
gptbot
hawaiibot
linkedinbot
megaindex.com
megaindex.ru
netestate
netestate%20ne%20crawler
panscient.com
seekportbot
seokicks
seokicks-robot
serpstatbot
treato-bot

Rule Path
Disallow */get-pdf

Other Records

Field Value
crawl-delay 14

bingbot
slurp

Rule Path
Disallow */get-pdf

Other Records

Field Value
crawl-delay 8

Other Records

Field Value
sitemap https://www.covidhealth.com/sitemap.xml

Comments

  • automation managed
  • block get-pdf
  • default 10 sec delay
  • 20 sec delay
  • 16 sec delay
  • 14 sec delay
  • 8 sec delay