suspension set up.

[know your limitations]

should you be able to feel any difference between fully in on the compression damping and fully out?

I have a stock shock.
The difference in feel between softest and hardest damping can go from space hopper on a trampoline to getting booted up the ar5e if you so much as look at a pothole. So yes, I'd say something is amiss with yours.
Look forward to hearing what they have to say! Good luck