intelligentinsurer.com
robots.txt

Robots Exclusion Standard data for intelligentinsurer.com

Resource Scan

Scan Details

Site Domain intelligentinsurer.com
Base Domain intelligentinsurer.com
Scan Status Ok
Last Scan2024-09-05T13:02:01+00:00
Next Scan 2024-10-05T13:02:01+00:00

Last Scan

Scanned2024-09-05T13:02:01+00:00
URL https://intelligentinsurer.com/robots.txt
Redirect https://www.intelligentinsurer.com/robots.txt
Redirect Domain www.intelligentinsurer.com
Redirect Base intelligentinsurer.com
Domain IPs 13.226.2.112, 13.226.2.115, 13.226.2.76, 13.226.2.97, 2600:9000:2022:200:1:88f1:3ec0:93a1, 2600:9000:2022:2a00:1:88f1:3ec0:93a1, 2600:9000:2022:a000:1:88f1:3ec0:93a1, 2600:9000:2022:a400:1:88f1:3ec0:93a1, 2600:9000:2022:c00:1:88f1:3ec0:93a1, 2600:9000:2022:ca00:1:88f1:3ec0:93a1, 2600:9000:2022:e600:1:88f1:3ec0:93a1, 2600:9000:2022:e800:1:88f1:3ec0:93a1
Redirect IPs 13.226.2.112, 13.226.2.37, 13.226.2.44, 13.226.2.62, 2600:9000:2795:5400:13:b75b:e580:93a1, 2600:9000:2795:5a00:13:b75b:e580:93a1, 2600:9000:2795:7000:13:b75b:e580:93a1, 2600:9000:2795:a00:13:b75b:e580:93a1, 2600:9000:2795:a200:13:b75b:e580:93a1, 2600:9000:2795:de00:13:b75b:e580:93a1, 2600:9000:2795:e000:13:b75b:e580:93a1, 2600:9000:2795:f800:13:b75b:e580:93a1
Response IP 52.85.49.19
Found Yes
Hash f24a230d414bce8b5e99b9c864294b009dadcc0f2c7931fcff9d21afcf469e21
SimHash 285cdc80b213

Groups

*

Rule Path
Disallow /search?

baiduspider

Rule Path
Disallow /

baiduspider-video

Rule Path
Disallow /

baiduspider-image

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

yandex

Rule Path
Disallow /

megaindex.ru/2.0

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

blekkobot

Rule Path
Disallow /

paperlibot

Rule Path
Disallow /

searchmetricsbot

Rule Path
Disallow /

twengabot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

sistrix crawler

Rule Path
Disallow /

psbot

Rule Path
Disallow /

tineye

Rule Path
Disallow /

Other Records

Field Value
sitemap https://www.intelligentinsurer.com/sitemap.xml

Warnings

  • 2 invalid lines.