covidhealth.com
robots.txt

Robots Exclusion Standard data for covidhealth.com

Resource Scan

Scan Details

Site Domain covidhealth.com
Base Domain covidhealth.com
Scan Status Ok
Last Scan2024-11-08T12:54:36+00:00
Next Scan 2024-11-15T12:54:36+00:00

Last Scan

Scanned2024-11-08T12:54:36+00:00
URL https://covidhealth.com/robots.txt
Redirect https://www.covidhealth.com/robots.txt
Redirect Domain www.covidhealth.com
Redirect Base covidhealth.com
Domain IPs 3.230.84.86, 52.45.87.85
Redirect IPs 3.230.84.86, 52.45.87.85
Response IP 52.45.87.85
Found Yes
Hash 5bd1aa4976d4530392d3a21610549d1ba7134e06d37ff1e1583cdc8b55daae5f
SimHash 7b4f5851ccb0

Groups

*

Rule Path
Disallow */get-pdf

Other Records

Field Value
crawl-delay 12

mj12bot
seekportbot

Rule Path
Disallow */get-pdf

Other Records

Field Value
crawl-delay 20

awariorssbot
awariosmartbot
blexbot
dataforseobot
dotbot
panscient.com
petalbot
turnitinbot

Rule Path
Disallow */get-pdf

Other Records

Field Value
crawl-delay 16

ahrefsbot
ccbot
claudebot
daum
friendlycrawler
gptbot
hawaiibot
linkedinbot
megaindex.com
megaindex.ru
netestate
netestate%20ne%20crawler
panscient.com
seokicks
seokicks-robot
serpstatbot
treato-bot

Rule Path
Disallow */get-pdf

Other Records

Field Value
crawl-delay 14

bingbot
slurp

Rule Path
Disallow */get-pdf

Other Records

Field Value
crawl-delay 8

Other Records

Field Value
sitemap https://www.covidhealth.com/sitemap.xml

Comments

  • automation managed
  • block get-pdf
  • default 10 sec delay
  • 20 sec delay
  • 16 sec delay
  • 14 sec delay
  • 8 sec delay