Cutting mold off has always bothered me a little. I think I've only done it once (this was before I starting vacuum sealing cheese). I ended up cutting all six sides...... I figured the mold I could see was one thing, but there might be spores and mold ready to start on other surfaces that I don't see. I probably wasted a lot of good cheddar, but I felt better about using it.
And this from a guy who thaws meat on the counter and cleans a cutting board my splashing dish soap on it and rinsiing with hot water