I'm 6'1 and commute 50 miles along the M6 every day so can sympathise.
Firstly, I can't recommend a good set of earplugs enough, and apparently helmet choice makes a big difference though I have no experience of that, having only ever owned one.
As for the buffeting, I only find it a problem if going over 90 or being too close to the car in front.
The turbulence from a car, van or lorry has a much greater effect on me than higher speeds, does anyone else find this?