I would've thought the bearings would be a more likely culprit.
Can you not check the axle for straightness by rolling it on something flat?
Agreed. Check the bearings by hand to see if you can feel any notchiness.
Axle is another possibility. A piece of glass is the best surface to test on as they are as flat a surface as you can get.
I was going to suggest a missing spacer, but if you removed the brake caliper and it still does it then unlikely.
If you are replacing sprockets then unless your chain has only done low miles, you should really be replacing the chain also. Otherwise you're likely to wear out the sprockets in double quick time.