Could be wear in the bolt itself.
However, even with all new bearings and spacer collars, it's still common to have around 1mm of vertical movement. One way to reduce it is to slacken all the bolts slightly, let the bike down onto the suspension (ie, on sidestand) and then tighten the bolts again.
Replace on the centrestand and do the final torque-up.