Something like simple syrup can be stored in either glass or plastic, I would store anything containing oil in glass or polycarbonate plastic.
Check out the dollar stores, K-mart, Walmart, Target, etc. - I just recycle bottles from other things, like water bottles that have a little "valve" on the top (pull to open/push to close).