Sentence 1 is ambiguous. It could mean that the reason for his not getting the job was that he didn't have enough experience. If this is the case, a comma is desirable:
They didn't give him the job, because his experience was insufficient.
In Sentence 1 boith statements are "new" information.
Sentence 1 could also mean that his not having enough experience wasn't the reason--that there was a different reason.
They didn't give him the job because his experience was insufficient; it was because he was sloppily dressed for the interview
In Sentence 2 we already know that they didn't give him the job. We assume that it was because he didn't have enough experience, and the speaker is correcting our assumption.
Sentence 2 is not ambiguous.
Either "because of his insufficient experience" or "because of his lack of experience" is OK. Also: "because he lacked sufficient /enough experience."