Kuen, what you need to understand is what David explained when, in reference to the structure "go (and) infinitive", he said something as simple and clear as:
It refers to one complete action, not to an activity.
I think your sentences 1 to 5 above are not grammatically incorrect but, since some of the infinitives following the verb go indicate activities (a continued action) rather than individual actions that can be viewed as having a beginning and an end, the gerund would be more usual. I find these sentences more likely if you wish to use the infinitive:
1'. I need some exercise. I think I'll walk a mile.
2'. I need some exercise. I think I'll walk (instead of taking a taxi).
3'. Let's shop/ hike/ fish.
4'. Let's shop in the store.
5'. I am going to hike in the woods tomorrow morning.