From what I can tell, sweet potatoes alway have tapered ends while yams can be either rounded and oblong or just globular. Sometimes I see yams that are branched too. If you get a chance to see the plants themselves, yams are climbing vines and sweet potatoes creep along the ground.
Also, sweet potatoes come in a variety of colors from maroon to orange while yams are pretty much always brown skinned. In the Philippines, there's something called ube that's a variety of yam that has deep purple flesh and is used in desserts. I've had ube icecream that was really tastey.