Very important. After all, if we all wanted to go the the fridge and get dinner at different times, we wouldn't go out, would we? That is one reason I often do not send a meal back that by all rights should be; by the time the problem is fixed, I'd be eating my dinner when everyone else is ready to go home (for me, most of the time the problem is a steak I've ordered VERY RARE comes out gray. Unlike when your steak is a little underdone for your taste, you can't uncook a steak that looks like stew meat). If the time difference is much more than 5 minutes, I'd be frustrated. A good measure is that most of us were raised to not start eating until everyone has been served. If the first person served has to eat a cold meal, and really, that doesn't take long, than it has been too long.
You have to understand, I'm talking restaurants here. You know, where you are paying well over what it would cost you to do it at home. If you are at someone's home, then shut up and put up or stay at home. If someone is gracious enough to feed you with friendship or even love, eat it and keep quiet.