It depends on the recipe. But - milk contains sugars and fats as well as water. So, just leaving out the milk will alter your cake's texture, flavor and keeping qualities.
If you are trying to make a "non-dairy" cake - look for one that uses apple sauce as a substution. These recipes are adjusted for the lack of dairy.
"It ain't what you don't know that gets you in trouble. It's what you know for sure that just ain't so." - Mark Twain