I would say it has a gentle sweetness to the fruit. The best way to eat it is to take the fruit from the shell, wrap it in saran wrap and freeze for about an hour. When you take it out, it'll be just a little firm. Bit into it and it's like you're tasting a really creamy custard.
Most people get turned off by the smell and I think that has a lot to do with their reaction to the taste. For me, since I grew up accustomed to the smell, it doesn't have negative connotations (i.e. smells like trash) for me. If you ever get a chance to try it, go for it, just try to keep an open mind.
I liken durian to how most older Asians don't like cheese. They say it smells and tastes funny. When we first came to the U.S. my Mom flat out refused to eat cheese for those very reasons, now she has slowly come around. If it's a light tasting cheese and if it's in moderation, she'll have some.
Ishbel is right though, the durian fragrance penetrates walls!