We recently went to a pool party that was simple but very good and very filling. The main feature was a baked potato bar, with sour cream, butter, and all the fixings. There was also a large bowl of salad surrounded by dressings and a variety of salad toppings. The only meat (other than some bacon bits and chopped up deli meats for the salads) was chicken tenders. I was so full I didn't even really look at the desserts, but there were brownies, fresh fruit, and ice cream. There were about 40 of us there, and it went very smoothly. Fifteen to 20 should be a cinch!
Otherwise you could make chili, spaghetti, lasagna, or something else that could be done in slow cookers or casseroles. Or deli sandwiches. You could just put the breads, meats, cheeses, and condiments out and everyone could make their own. Maybe have a pot of soup to go along with them, some fruit, salad, etc.