Although I have had Oat milk and Almond milk, it is not on my grocery lists of things to buy nor make so I can't really help you there.
Have no comparison to purcha$e of coffee filters vs cheese cloth.
When I made jellies I would use cheese cloth. I did reuse them to a certain extent. Maybe twice? Depended on what I used it for. I did a simple up and down in new hot soapy dishwater, after rinsing I'd pour boiling water over them. But after the 2nd time they get messy to use. But they were cheap enough - I just bought more.
My 2nd uses were usually making chicken stock to grab all the little stuff. Then everything got chucked.
Another consideration is quantity at any one time. Coffee filters aren't that big.
Super fine sieves would hold more. Small ones are used for skimming
Something like this?
View attachment 72417 Amazon about 15$ Canadian. Or a set of 3, would you find a use for them in other things?
View attachment 72418