I found the front of the side plates to be at a slight angle. Upon releasing the tension I found they were not bent, just under tension. I tweaked some more and it now tracks much better. It wanders back and forth inside the delrin guide.
Thanks a bunch.
EDT:
No. The belt was too lose, and tightening it up until it doesn't slip causes it to go left again.