Hacker Newsnew | past | comments | ask | show | jobs | submit | nighthawk454's commentslogin

The 'Elbow method' [1], yeah. Sweep a bunch of different k values and pick the one where your target metric starts having diminishing returns. There are a lot of heuristics you could use.

There's still not one particularly great way to pick k for this type of clustering. There kind of can't be as clustering is sort of inherently subjective.

Other methods like HDBSCAN can find the number of clusters themselves. There's still a 'one key parameter' you have to pick yourself, but it can be more intuitive and less sensitive for some datasets, when you don't know how many clusters there are / how many you want.

Sometimes, picking k exactly is useful for practical reasons too. Like fixing the size of your codebook. Or partitioning a dataset into k equal-ish pieces, say to spread across a pool of servers of a known/fixed size.

[1] https://en.wikipedia.org/wiki/Elbow_method_(clustering)


Yeah, I guess you can't really avoid picking k without hierarchical clustering.

Yeah, it's not a well-defined problem, so there's no one true solution. Then only question left is "well what do you want?". Always have to pick something.

https://en.wikipedia.org/wiki/Cluster_analysis#Definition


This guy did it to Van Halen, and it renders it utterly lifeless.

https://youtu.be/p9rVKnEvAQ8?t=240


At the risk of oversimplifying, it seems the field looked at neuron activations and couldn't find the pattern, so a series of convoluted hypothesis were proposed to explain it away. When the actual answer was the pattern was very simple they just missed it. Little bit embarrassing really.

The good news I guess is the actual patterns make a lot of sense with what you'd expect the neurons to do, and don't require nearly as much 'black magic who knows what wonders scale can create'.


which amounts to nearest-neighbor


Anyone remember Universal Transformers paper (Dehghani et al) from back in 2018? Recurrent transformers have a history as long as transformers themselves.

Somewhat unclear how particularly novel this is vs a way to save compute.


> it still needs to bootstrap the connection via central server. I think that always might be a limitation of P2P

I think Iroh has a way of doing it using either DNS or BitTorrent DHT. Not perfect but fairly decentralized


Iroh's discovery is genuinely neat, and node-IDs-as-public-keys is similar to bitbang.

A browser can't join a DHT though because of no UDP sockets, so the connecting side would need a native client, which is the thing bitbang avoids. And DHT lets you locate the peer, but you still need somewhere to trade SDP and ICE candidates, so the rendezvous stays either way I think.

What I can do is keep the broker small enough that self-hosting is realistic, which is where it is now -- no accounts, no registry, nothing that grants access, only a simple broker.


Ah good point, no udp. I was looking at webtorrent but it does share the same limitation, unless you’re on the nodejs (non-browser) version.

Looking forward to trying bitbang! Seems super neat


You can add a library view! Checkout this Bookshelf plugin:

https://github.com/AndyHazz/bookshelf.koplugin


This tracks with what I’ve heard around as well. What changed in the financial instruments?

My understanding is a lot of the loans have gone PIK or otherwise essentially aren’t serviceable at current prices. Do you think that’s resolvable somehow or just lagging implosion?


Nothing changed in financial instruments - we just don't usually have a lot of office tenants decide to walk away like we have with the advent of remote work. The loans will work themselves out. We've seen a couple of buildings trade for way less than pre-Covid numbers, and we'll see more.

In another 10 years downtown Seattle will be aligned with the rest of the market again.


Some potentially related stuff on the topic:

Anisotropy in word embeddings dates back to at least 2017 with word2vec - where there were zero layers.

The cone-shaped anisotropy in transformers is known since at least Gao et al. 2019. That lineage explained it fairly intuitively as an artifact of word frequency and softmax geometry (so a training dynamic).

A variety of papers followed up by adding post-hoc ‘whitening’ steps (from classical statistics/NLP), then adding regularizers to the loss to penalize the anisotropy, eventually penalizing the covariance matrix (a la VICReg), and then the SIGReg method as a computationally much cheaper way to approximate the full covariance.

As another commenter pointed out it’s also similar to the InfoNCE/contrastive learning objectives. Where terms to increase uniformity (spread out evenly) on the hyper sphere were added. Like the SimCSE (Gao 2021) paper or the excellent alignment/uniformity breakdown from Wang & Isola 2020.

This proposed dispersion loss seems to be similar in that it pushes things apart by penalizing cosine similarity. Although this one works on the tokens within one sequence. Usually contrastive methods mean pool the sequences and then contrast against the other pooled sequences in the batch.


> Greenbacks … a form of fiat money

https://en.wikipedia.org/wiki/Greenback_(1860s_money)


This was a wartime measure. The US went back to a hard money system soon after the civil war ended


Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: