There has been a lot of progress in this field in the research community. Two good papers:

* https://ojs.aaai.org/index.php/AAAI/article/view/11027 * https://arxiv.org/abs/2212.03978

Errrrr... how is the first paper related to A* and path finding?

It's about "planning" (ie. searching an action space) which does not generally have a heuristic, so it needs to do "exploration" (which is exactly the opposite of what you want for optimal path finding) to counteract getting stuck in local optima due to greedy search...

EBMs show up all over the place, apparently even your classifier is an EBM :) (https://arxiv.org/abs/1912.03263).

You can take many equivalent perspectives on learning systems, but mostly it reduces to "messing with denominators in Bayes' rule". This is no different.

EBMs today aren't used because first you have to fit the joint model, then you have to fix some inputs, then fit the other inputs in a second optimization step. That's just too much compute for today's workloads compared to feedforward NNs.

GPT-based search engines usually use some sort of a database to retrieve context for the LLM to summarize first. This is what people refer to as RAG these days: https://blogs.nvidia.com/blog/what-is-retrieval-augmented-ge....

Some of these GPT engines maintain their own vector DB to do semantic search, others are directly hooked into Bing / Google. So pubmedisearch.com would be one component of a GPT-based engine. We actually have a GPT-based engine here: https://medisearch.io/.

Lots of annoying edge cases as you can imagine, nothing particularly glamorous.

Done! Let me know if you have other feedback.

Thanks! Looks quite relevant

Training for multiple epochs is a bit like that :)

We use pinecone and it is not ideal, looking at https://turbopuffer.com/ now. They look quite promising :)

Did you compare pinecone against pgvector with Postgres? Self hosted of course

Isn't it funny how the best Choice somehow always comes back to Postgres in the end XD (for most)

Yes, that’s where I’m these days. I don’t even think of venturing outside of Postgres these days, except for say things like Redis etc. where there are mature and established options for specific use cases.

Will definitely check pgvector, thanks for the pointer.

What kinda dimensions did you keep it relatively low to keep costs down?

1. We cover all the articles on PMC. The exact cost is hard to estimate because we did a lot of iterations.

2. We do weight those ... it is a lot of trial and error and you have to have good & exhaustive benchmarks.

Yes