A lot of the answer depends upon the climate in which you live. As a general rule, only baked goods with fillings and frosting that would spoil or go rancid need to be refrigerated. The fridge does dry out breads, cakes, and pastries.
However, I live in Florida, where mold is a bigger concern than drying. I used to live in Arizona, where nothing ever turned moldy, but everything dried out. To some extent, you have to look at the season and the climate where you live. Unfortunately, there is no one simple answer.