I like to leave the skin on - you can lightly oil the skin to help with the crisping, it also help keep the flesh from being inadvertently' torn up on the first side down in the pan. probably 'keeps' a bit of flavor as well.
it's easy enough to remove the skin if you prefer to serve it that way - done on a hot pan it generally 'separates' very easily, with potentially just a little adhesion around the thinner edges (depending on how deep the oil is, you can get some 'deep fried' effects around the edges).
exception: if I bread the fish, I remove the skin before breading/pan frying.....