that bottleneck is a bottleneck specific to the GNFS. There are about a dozen pretty different factoring algorithms known to the public (e.g. trial division, Polard, ECM, quadratic sieve, GNFS). I don't think any mathematicians believe that the GNFS is the optimal factoring algorithm out there. GNFS has runtime L(1/3, cbrt(64/9)) if there is a known algorithm of say L(1/3, 1.5) which would be a pretty minor optimization equivalent to making the GNFS as good as the SNFS, or a bigger breakthrough of say, L(1/4, 2.5), 4096 bit keys could easily be necessary. For reference, this amount of improvement is about the same as the difference between the Quadratic sieve and GNFS.

Why yes, if there was an algorithm known good enough to make 4096 bit keys necessary then 4096 bit keys would be necessary. But there isn't and there has been little progress for a long time now.

> But there isn't and there has been little progress for a long time now.

publicly. The algorithm side and the hardware side are really really different because we can be pretty sure that the NSA doesn't have a giant super computer that is either dramatically more efficient than commercially available computers, or that uses more than say 1% of US electricity consumption. On the algorithm side, we know that the NSA employs a decent number of number theory people, and given the way that math research progresses, it's pretty easy to imagine that one of those people has found a slight algorithmic improvement over public state of the art. CFRAC was invented in the 1970s. QS invented in the 1980s. The NFS was the 1990s with some improvements in the 2000s. If we go another 50 years or so with no progress, it maybe starts to be reasonable to believe that there isn't a secret algorithm that's better, but with 5 improvements in the last 50 years, it seems pretty likely that there could be 1 more that has been found but not published.