Obviously you can do orders of magnitude better than 160 req/s! But if you only have tens of thousands of users or less, like all but the top thousand web sites in the world, there's at best very marginal benefit in doing so. It's spending effort to solve a problem that no longer exists.
I agree that the number I gave is erroneous, but not in the direction you imply. 400ms is extremely conservative, and it's not just starting an executable. I took my 100-millisecond estimate for the time to start up Python and `import cgi`, which loads 126 modules in the version I have here on my cellphone, and multiplied it by a safety factor of 4. Even on my cellphone it doesn't take that long, although `python3 -m cgi` does, because it loads all those modules and then runs a debug dump script. The time I measured for `python3 -m cgi` on my laptop is 90–150ms.
If all you are doing is starting a per-request executable, that will typically take more like 1 millisecond than 100 milliseconds.
Perhaps you meant to suggest that the actual logic triggered by the request would increase this 400ms number significantly.
Consider first the case where it would, for example because you do 200ms of computation in response to each request. In this case the best you could do by switching from CGI to a more efficient mechanism like FastCGI is a 3× increase in req/s. If that allows you to increase from 10 million requests per day on one server to 30 million, that could be worthwhile if you actually have tens of millions of requests per day. But it's kind of a marginal improvement; it saves you from needing two or three servers, probably.
Now consider the case—far more common, I think—where your computation per request is small compared to our hypothetical 400 ms Python CGI overhead. Maybe you do 20ms of computation: 80 million machine instructions or so, enough to tie an IBM PC up for a few minutes. In such a case, the total one-core request time has gone from 400ms to 420ms, so 400ms is a very good estimate, and your criticism is invalid.
> But if you only have tens of thousands of users or less
Even on HN#1, you get like two requests per second for maybe 18 hours. That's three orders of magnitude below 200M/day. Shifting the argument to say that no hobby project needs this is easy. PHP being in common use already proves that starting up and tearing down all context is fine for >99% of websites. But the context we're in is an article saying one can do 200M requests per day with CGI-bin, not whether 99% of sites need that. CGI-bin is simply wasteful if the process takes 400ms before it starts doing useful work, not to mention a noticeable amount of lag for users. (Thankfully TFA specifies they're not doing that but are speaking of compiled languages)
> your computation per request is [commonly] small compared to our hypothetical 400 ms Python CGI overhead. Maybe [that makes it go] from 400ms to 420ms
At "only" 420ms total request time, I'd still want to take that prototype code out of this situation. Might be as easy as wrapping the code so that the former process entrypoint (__main__ in python) gets, instead, called by some webserver within the same process (Flask as the first python example that comes to mind). Going from 420ms request handling time to slightly over 20ms seems rather worth an hour of development time to me if you have this sort of traffic volume
It's simply wasteful, just like spawning off a new process for every shell command. But we can afford to waste the thing it's wasting—unless it produces a noticeable amount of lag for users, as you're saying it would
I agree that the number I gave is erroneous, but not in the direction you imply. 400ms is extremely conservative, and it's not just starting an executable. I took my 100-millisecond estimate for the time to start up Python and `import cgi`, which loads 126 modules in the version I have here on my cellphone, and multiplied it by a safety factor of 4. Even on my cellphone it doesn't take that long, although `python3 -m cgi` does, because it loads all those modules and then runs a debug dump script. The time I measured for `python3 -m cgi` on my laptop is 90–150ms.
If all you are doing is starting a per-request executable, that will typically take more like 1 millisecond than 100 milliseconds.
Perhaps you meant to suggest that the actual logic triggered by the request would increase this 400ms number significantly.
Consider first the case where it would, for example because you do 200ms of computation in response to each request. In this case the best you could do by switching from CGI to a more efficient mechanism like FastCGI is a 3× increase in req/s. If that allows you to increase from 10 million requests per day on one server to 30 million, that could be worthwhile if you actually have tens of millions of requests per day. But it's kind of a marginal improvement; it saves you from needing two or three servers, probably.
Now consider the case—far more common, I think—where your computation per request is small compared to our hypothetical 400 ms Python CGI overhead. Maybe you do 20ms of computation: 80 million machine instructions or so, enough to tie an IBM PC up for a few minutes. In such a case, the total one-core request time has gone from 400ms to 420ms, so 400ms is a very good estimate, and your criticism is invalid.