I find the contraction in (2) to be highly awkward, but the other three sentences are fine. My preference, however, would be to speak of his taking or having the day off on Wednesdays rather than of his not working on Wednesdays:
1a) Only on Wednesdays did he have/take the day off.
To me, it unambiguously means: "But he didn't work on Wednesdays." It doesn't say anything about the other days. It doesn't have the same meaning as the others sentences. Your ignoring the inversion rule in (3) causes the meaning of "only" to change to "but" in that sentence.
The first three sentences have that implication, but (4) is ambiguous. The scope of negation with respect to "only" is unclear. The sentence could mean that it is only on Wednesdays that he doesn't work. Or (4) could mean that it is not only on Wednesdays that he works; he works on Wednesdays and on other days too.
4a) He didn't work only on Wednesdays. He also worked on Mondays and Tuesdays, and once a month he came in on Saturday.