You've asked a very interesting question. I agree with DocV's answer and would like to add that everything relates here to subject and object within infinitive clauses. You might not be accustomed to thinking of infinitive constructions as clauses, but they are: they have a subject and a predicate. But the verb in infintiive clauses lacks tense. That's why they are referred to as nonfinite clauses.
(1a) The coffee is too hot for me to drink.
(1b) The coffee is too hot to drink.
(1c) The coffee is too hot to be drunk.
In (1a), the infinitive clause has an overt, expressed subject: "me." That subject is introduced by the complementizer "for," which is not a preposition here. In (1b), the infinitive clause lacks an overt, expressed subject. The implied subject is phonologically null and is semantically equivalent to "anyone": "The coffee is too hot for anyone to drink." In (1c), "the coffee" is itself the implied subject of the infinitive clause, which is passive.
(2) Peter is too young to take care of himself.
(3) Peter is too young to take care of Sally.
(4) Peter is too young for Sally to take care of.
(5) Peter is too young to be taken care of by Sally.
Notice that the grammaticality of the reflexive pronoun ("himself") in (2) demonstrates that "Peter" is the implied, tacit subject of "take care of." If (2) used "him" instead -- "Peter is too young to take care of him" -- "him" would necessarily refer to someone other than Peter. Like DocV, I have done a few other permutations so that you can try to understand the different cases. In (4), Sally is the subject of "to take care of," and Peter is the implied object of "of."