A lot of it also depends on the capacity in which you are using it. When used to flavor and color rice, it lends a subtle earthy and musty flavor but it's not predominant. I've also used annatto to make an oil with and you can taste the flavor but I used a lot of annatto. The color also depends on how much you use. It can color food as dark as reddish-orange, almost like paprika. Annatto's also inexpensive so you shouldn't sweat it if you want to experiment and play around with it.
"Boldness has genius, power, and magic in it."
Johann Wolfgang Von Goethe