I find that dragging the cursor to the right is fast enough. However, removing that initial selection of the entire URL takes a couple of frustrating extra taps.

Removing complete selection in a single-line input is a weak point of editing experience on iOS (it is easy in a textarea since you have space to just tap away).