The truss rod is not supposed to be used to adjust/lower your action. That is just a by-product of it's primary function - to combat spring tension.
To properly adjust your action without fret buzz etc, you should follow a method like this:
Perhaps the best way to adjust it is like this:
1) Set the bridge as low as you would like it
2) Adjust the trussrod to try and get it to be as buzz-free as possible over the entire neck (I find that it's most crucial around the 12th-15th fret... if you get those right, the rest will generally be fine. I mostly check there, both to make sure it sounds good, and to make sure the action feels nice and light aswell).
3) Adjust the bridge again to remove any remaining buzz (or perhaps set it even lower if it's buzz-free, and insanely low action is your thing).
Repeat as often as necessary.