It depends on the recipe. But - milk contains sugars and fats as well as water. So, just leaving out the milk will alter your cake's texture, flavor and keeping qualities.
If you are trying to make a "non-dairy" cake - look for one that uses apple sauce as a substution. These recipes are adjusted for the lack of dairy.