I have always agreed that miles would be the intuitive answer; however, I am a scientist writing scientific professional opinions for clients, and as we both agree, miles is not factually correct when dealing with a fraction of a mile.
I respect your opinion. You don't actually say it, but I assume you mean to imply that you think "0.05 mile" in a sentence like "The sign was 0.05 mile from the house" is grammatically correct. It is obviously correct when functioning attributively, as in "It is a 0.05 mile stretch of sidewalk."
I think the question of the grammatical (as opposed to the mathematical) correctness of "0.05 mile" in "The sign was 0.05 mile from the house" comes down to how "0.05 mile" is pronounced, in speech or just in thought. If pronounced "five hundredths of a mile," it is correct. If pronounced "point zero five mile," it is (to my ears) ungrammatical.
A somewhat similar phenomenon occurs in technical linguistic writing, where "NP" is used to refer to a noun phrase. Interestingly, some writers use the article "a" with it ("a NP"), and others use "an" with it ("an NP"). It's a question of whether one hears or says "NP" as "en pee" or "noun phrase." ♣