You can use it as a substitute in basic recipes of that kind. But you may want to select the brand carefully. Well prepared coconut milk will have little coconut flavor, but many brands have a rather distinct flavor of coconut, which is not a bad thing, if you happen to want it. It will not, of course, substitute in anything where you depend on the characteristic reactions of milk that allow you to create cheese, yogurt, etc. You more often see the reverse substitution, milk used in combination with other things as a substitute for coconut milk. So don't commit to a large batch or baked goods without testing. And remember that there's coconut milk and coconut cream that rises when the milk is left to sit. One nice feature, compared to other common milk substitutes, is that coconut cream can be whipped into a credible whipped cream substitute..
(And cheese and butter are usually no problem for the lactose intolerant, as they contain relatively little lactose.)