First, regardless of what the speed charts for Forstner bits say, I'd use the slowest speed available. As said, Forstner bits weren't invented for end grain drilling. The problem lies with the cleanout chisels cutting straight across the grain; if you've ever cut end grain with a hand chisel and a hammer, this shouldn't be a surprise. Such cutting is best done with a slicing or paring cut without the hammer. Some day, someone will invent a modified Forstner bit with the cleanout chisels in a spiral; those should work better in end grain.
Drilling on the lathe is substantially self-centering. So, I've gotten best results by starting with smaller bits and working up to the desired diameter. This way, the rim cutters are doing most of the work, and they're better at it than the bottom chisels.
When drilling off the lathe, and without self-centering, it gets a little more complicated, because the center point is need for alignment. The way to go here is to drill a shallow hole with the largest size. Then drill a little deeper with the next smaller size, guided by the previous size's divot; and repeat until the bottom is reached. Having established the diameters for each size, work back up to the final diameter with the rim guided by its previous hole. On the forward pass, the bottom chisels don't have to do much work for shallow cutting; and on the backout pass, the rim cutters do almost all of the work. I said it was complicated, didn't I?
In the past year or so, Irwin has introduced a line of triple-flute auger bits, still using the Speedbor trademark. These are just dandy for end grain drilling. Some folk grind off the threads in the auger to prevent grabbing; for a few species, I've found that the threads fill up with dust and then behave as if the threads were absent anyway. Although they might not have the exact size for the final diameter desired, a smaller size near to the final diameter can be used, with a Forstner bit used for the final diameter. Here again, the rim cutters are doing most of the work.
Joe