Pretty good run down. Actually, a very good run down.
Sphinx is cool, but there's no reason to be afraid of solr/lucene. It takes _very_ little java knowledge to get it up and running and it's very very, really, crazy fast. Like, hundreds of thousands of searches a day on millions of documents and it's totally stable. knocks on wood
Also, Passenger is indeed better than all this mongrel + god + tweaks they're talking about it. AboutUs.org (my employeer) is the largest site on Passenger (that we can find) and we've had 2 actual crashes in the last 6 months, and they were fixed with rebooting Apache.
From my experience (and I have used Sphinx, Lucene, Solr and Xapian in production) is that Lucene/Solr have a pretty bad perfomance compared to Sphinx or Xapian.
My Lucene setup began to throw deadlocks and memory exceptions pretty early on. Searching on "deadlock lucene" on Google yields 25000 results. I have later rewritten the system to Xapian where it has run without any problems.
For live updates I would recommend using Xapian. For fairly static indexes I would recommend Sphinx (as it's _really_ fast for both indexing and searching, but it does not support live index updates yet).
Thanks for the suggestions! Will probably explore Solr down the road. I've written search on top of Lucene before, just didn't have time to work too hard on getting search going in between all the other features we're trying to push out.
Passenger I have heard great things -- been meaning to try since it does fix the mongrel queue problem.
Sphinx is cool, but there's no reason to be afraid of solr/lucene. It takes _very_ little java knowledge to get it up and running and it's very very, really, crazy fast. Like, hundreds of thousands of searches a day on millions of documents and it's totally stable. knocks on wood
Also, Passenger is indeed better than all this mongrel + god + tweaks they're talking about it. AboutUs.org (my employeer) is the largest site on Passenger (that we can find) and we've had 2 actual crashes in the last 6 months, and they were fixed with rebooting Apache.