Oil itself is fine at room temperature, but it is the fine particles of food that end up in the oil afterwards that may be worrysome for bacteria growth. Fortunately the oil is reheated to very high temperatures (above 180° F) so any active toxin would be killed. I'm more concerned about the flavor change caused by such contamination. If it sits for any length of time unused, or gets used more than a few times, it gets tossed.
Support bacteria. It's the only culture some people have.