Couple of days will be fine. It may get watery. Was it made with eggs or is it an instant or cooked pudding. The latter will last longer--maybe forever.
The problem isn't the bananas, it's the milk and eggs, which are a nearly perfect medium for bacteria to grow in. But I think the sniff test is pretty reliable -- if it looks ok and smells ok and tastes ok, what the heck!
OTOH, no pudding of any kind has ever gone bad around this house! It ain't to look at, girl! EAT IT!