What you have experienced is typical of not running an Rx pack or booster with a 1S cell through a '2S' speedo. Either a booster or an Rx pack will fix it. It seems the system will work without an Rx pack/booster, but the servo runs very slow.
Having a better servo will make no difference to this situation. You have done exactly the right thing by fitting the LiFe Rx pack. You could also try an RC Dynamic booster instead of the receiver pack, but since you've got that now... HTH