Whilst agreeing that this sounds like a servo fault, you could also check...
The servo arm. Whilst it might look like all the teeth are in good condition, does the horn have a hairline crack in it? If it does, this can cause the horn to jump a tooth.
The turnbuckles. If there is a stripped thread somewhere in one of the ball cups, the turnbuckle can move in the ball cup.
If the amount the car goes off-sentre is a lot, and is usually the same amount and direction when it happens, then suspect one of the above. If it is a little, then swap the servo out for another (if you have one or can borrow one) and see if it cures the problem.
HTH
