insurance-forums.net
robots.txt

Robots Exclusion Standard data for insurance-forums.net

Resource Scan

Scan Details

Site Domain insurance-forums.net
Base Domain insurance-forums.net
Scan Status Ok
Last Scan2025-05-09T22:06:29+00:00
Next Scan 2025-05-16T22:06:29+00:00

Last Scan

Scanned2025-05-09T22:06:29+00:00
URL https://insurance-forums.net/robots.txt
Redirect https://www.insurance-forums.com/robots.txt
Redirect Domain www.insurance-forums.com
Redirect Base insurance-forums.com
Domain IPs 104.21.112.1, 104.21.16.1, 104.21.32.1, 104.21.48.1, 104.21.64.1, 104.21.80.1, 104.21.96.1, 2606:4700:3030::6815:1001, 2606:4700:3030::6815:2001, 2606:4700:3030::6815:3001, 2606:4700:3030::6815:4001, 2606:4700:3030::6815:5001, 2606:4700:3030::6815:6001, 2606:4700:3030::6815:7001
Redirect IPs 104.25.147.118, 104.25.148.118, 172.67.97.34, 2606:4700:20::6819:9376, 2606:4700:20::6819:9476, 2606:4700:20::ac43:6122
Response IP 172.67.97.34
Found Yes
Hash 03a917165dcc57027efd26b1cfa7878135e4fc677845e3359be899e6a7e8a062
SimHash 9e61621046b3

Groups

*

Rule Path
Disallow /community/conversations/
Disallow /community/account/
Disallow /community/attachments/
Disallow /community/goto/
Disallow /community/posts/
Disallow /community/search/
Disallow /community/tags/
Disallow /community/find-new/
Disallow /community/members/
Disallow /community/online/
Disallow /community/recent-activity/
Disallow /community/admin.php
Disallow /community/deferred.php
Allow /

boardreader
boardtracker
gigabot
twiceler
dotbot
baidu
baiduspider
baiduspider-video
baiduspider-image
naverbot
sosospider
yandex
youdaobot
yeti
magpie-crawler
magpie-crawler/1.1
magpie-crawler/1
genieo/x.x
spinn3r
spinn3r 3.1
feedwordpress/2012.1218
feedwordpress
spinn3r (spinn3r 3.1)

Rule Path
Disallow /

Other Records

Field Value
sitemap https://insurance-forums.com/sitemap_index.xml
sitemap https://insurance-forums.com/community/sitemap.php