realhealthmag.com
robots.txt

Robots Exclusion Standard data for realhealthmag.com

Resource Scan

Scan Details

Site Domain realhealthmag.com
Base Domain realhealthmag.com
Scan Status Ok
Last Scan2024-11-10T19:47:23+00:00
Next Scan 2024-11-17T19:47:23+00:00

Last Scan

Scanned2024-11-10T19:47:23+00:00
URL https://realhealthmag.com/robots.txt
Redirect https://www.realhealthmag.com/robots.txt
Redirect Domain www.realhealthmag.com
Redirect Base realhealthmag.com
Domain IPs 3.215.112.53, 52.45.87.85
Redirect IPs 3.215.112.53, 52.45.87.85
Response IP 3.215.112.53
Found Yes
Hash ce6b5b0da593f7f0f4f258a01ef3a69d6550c696e6de6d130dc2b15be618597b
SimHash 7b4f7851cc90

Groups

*

Rule Path
Disallow */get-pdf

Other Records

Field Value
crawl-delay 12

mj12bot
seekportbot

Rule Path
Disallow */get-pdf

Other Records

Field Value
crawl-delay 20

awariorssbot
awariosmartbot
blexbot
dataforseobot
dotbot
panscient.com
petalbot
turnitinbot

Rule Path
Disallow */get-pdf

Other Records

Field Value
crawl-delay 16

ahrefsbot
ccbot
claudebot
daum
friendlycrawler
gptbot
hawaiibot
linkedinbot
megaindex.com
megaindex.ru
netestate
netestate%20ne%20crawler
panscient.com
seokicks
seokicks-robot
serpstatbot
treato-bot

Rule Path
Disallow */get-pdf

Other Records

Field Value
crawl-delay 14

bingbot
slurp

Rule Path
Disallow */get-pdf

Other Records

Field Value
crawl-delay 8

Other Records

Field Value
sitemap https://www.realhealthmag.com/sitemap.xml

Comments

  • automation managed
  • block get-pdf
  • default 10 sec delay
  • 20 sec delay
  • 16 sec delay
  • 14 sec delay
  • 8 sec delay