Intermission: What even is an infinitive?
In the grammar taught in schools, english speakers are usually taught that verbs like "to go" are infinitives, without being given a good explanation about what that means or why they're called that.
In the study of latin, verb forms could be categorized based on whether they contained information about subject, and these categories were obvious, because latin verbs have to agree with their subject's gender, just like they can with number/person in english (I go, you go, he goes).
However, some verb forms in latin contain literally no information about the subject. Such verbs are called "non-finite", because they didn't "finish up" (see: FINIte) the information necessary to make a real statement with the verb. Infinitive is the most non-finite a verb can be.
This concept is hard to explain when you move away from "fusional" languages like latin, because you stop containing so much extra information in the verb forms, and you even end up with subtle overlap between infinitives and gerunds.
There's light on the other end of the tunnel. The main use of infinitives in english is to link verbs with words that do have a "finite"ness to them.
"I want to go now."
"It's not that he's going to be late, he just doesn't know what to do".
Here, "to go", "to be late", and "to do" are (respectively) clearly attached to "want", "going", and "what". The rule isn't as simple as "stick two verbs together", and the same is true with japanese's て "form" and the stem the て "form" attaches to.
However, because japanese is a topic-prominent language and has no fusion, the concept of "finite"ness is even harder to understand against it than english. Linguisticians are working hard at problems like this and have a lot of really good ideas, but they're unintelligible to normal language learners.
Because of this, as well as the fact that linking an abstract verb to a less abstract verb is not their only use, the て "form" and its stem won't be called infinitives in this guide. But the comparison to infinitives is glossed over almost everywhere, so I thought I should point it out for the two of you that actually understand infinitives.