I may be wrong, but I thought that the torque arm was supposed to pivot, otherwise it will bind and work against the suspension movement.
Based on this theory, you shouldn't have to loosen it to adjust the chain, however, when you find yourself in this scenario, remove, clean and lubricate the torque arm so that it moves properly.