CNNs excel in vision tasks where you have limited compute, limited memory, limited data, and want something that works super well and quick. People usually don't hook CNNs up to a transformer to get language understanding either, you have to train bespoke CNNs for specific tasks
ViTs excel where you're unbounded in compute + data and also want text understanding or have a conversation about an image
These are vibes. ViT has been shown to work fine on small data with proper hyperparam and most of what you mention is actually doable just fine with the other architecture as well.
I spent part of my life growing up in a major Asian city where communal ties are very strong, and spent a very significant fraction of my childhood outside of my house, just because there was so much to do - whether it was playing soccer, playing hide and seek, or taking the local bus/train with friends to shows, coffee shops or restaurants. All of this was before cell phones. If I got lost or needed help I'd ask a stranger for directions or navigation.
Nothing bad ever happened to me, and most people were actually quite kind and helpful to a 14 year old kid asking for help and directions. That sense of confidence in myself - that I could figure out how to pretty much do anything - has stuck with me in a way that a lot of my friends who grew up in sanitized suburban neighborhoods just don't have.
My PhD advisor found out that my English writing skills are quite good and the rest of their lab were Chinese internationals, so they started making me write all of their research grants. 30 pages every 2 months, pre-ChatGPT days.
Unfortunately, consciousness is deceptively hard to define, and so any benchmark to measure or quantify it can be endlessly debated.
You can argue that it's a property that all living beings have in common - and even among *unconscious* beings there's a form of consciousness and self-awareness that's ever present, but definitions are elusive and vague and tough to pin down.
The mechanistic argument against LLMs - that they're just matrix multiplications - breaks down because they can clearly pass the Turing Test which was the gold standard for what intelligent behavior really meant, thus breaking the old notion that intelligence has to have some form of biological basis. Yet its clear that there are forms of intelligence that rats have which the frontier LLMs don't possess (is that consciousness? or a different kind of intelligence), and its hard to pinpoint what exactly that is, so we probably need the philosophy departments of major universities to come up with newer definitions of intelligence and consciousness.
I personally believe that intelligence and consciousness are 2 separate forms of emergence from simple automata that may occur together (such as in humans) or not (such as consciousness in plants and intelligence in LLMs)
Wait, why does the Turing test invalidate the mechanistic argument against LLMs? Isn’t it possible the Turing test isn’t sufficient to measure intelligence?
Depends on who you ask. From my grandma (97), it apparently stops being as fun when you start losing autonomy and all your relatives your age are dead.
Nobody wants to hear this because it departs from the 'billionaire bad' trope. But Thiel has been remarkably consistent in his criticism of housing being the center of all of the Millenial economic woes.
Sure, Thiel is identifying some of the same problems, but the solutions he's proposing are basically the opposite of the ones Mamdani is. Unsurprisingly, he's proposing solutions that benefit billionaires rather than everyone else (e.g. just les us build whatever wherever we want, of course we'll build cheap housing that brings property houses down! Who could ever imagine we'll build luxury mansions that keep property prices high?).
How many times must it be explained that building luxury mansions still brings property prices down. Nobody ever voluntarily builds crappy low income housing. That’s never how development works. You let people build the new fancy buildings they want to build with all the margins and high prices. Then, when a bunch of rich people move in, that’s people that are no longer chasing all the other apartments. Eventually, way down the road, these swanky apartments will be tomorrow’s old and crappy ones in the neighborhood that’s not hip anymore, and low income people can rent them. This is how things actually work, and it’s fine.
What is NOT fine is when you have banks and private equity bullshit chasing homes purely as an asset to flip. That’s the thing we need to curtail, because it’s just money laundering at the expense of the American homeowner.
> Then, when a bunch of rich people move in, that’s people that are no longer chasing all the other apartments.
Maybe? Seems to me that there's a certain level of wealth where this no longer is true. Housing has (unfortunately in my eyes) become one of those black boxes that you put money in and money comes out; it's an investment. But what you're telling me goes contrary to what I know about the housing market: no, actually, houses depreciate in value because they'll have to ask poor people to buy / rent the place at some point. Can I go buy a mansion built in 1930 for a bargain price?
(I do agree about the private equity part, just the first bit doesn't pass a sniff test from me)
Oh, understood, the housing will "trickle down", right?
How many times do we need to learn that trickle down economics doesn't work? Making the rich richer and happier will never "trickle down" to the poor, it will stop at the rich.
With land, this is particularly obvious. There is a finite amount of land. The more of it is occupied with luxury mansions, the less land will be available for high-density housing. Building 1 new luxury mansion removes land from the pool that could house dozens if not hundreds of people. And rich people don't move in from cheaper housing to more expensive, they just keep both, or they buy the new mansion as a vacation home.
Property is an investment, and there are huge vested interests in keeping property values going up - and not just from rich people, but virtually everyone who owns their own home. You have to fight a lot of these interests to force prices to go down. "Just build more" doesn't work, the space in and around a city is limited.
Building a luxury mansion on top of unused and uninhabited marginal land - sure, that can bring prices down. But building a luxury mansion for one family by replacing dilapidated, and therefore cheap, high-density housing which used to house a dozen families brings prices up. And in practice, that is what is happening in cities. Old low-rise multi-family buildings in nice neighborhoods are being modernized and then converted into single-family mansions.
More housing, regardless of type, is directly correlated to cheaper average rents [0]. So Thiel is right in that regard, the red tape for construction and opposition from NIMBYs must be curtailed.
NIMBYs typically oppose especially lower cost housing, so it's not surprising that cutting red tape does help reduce housing prices, or at least slows down the rise.
The question is if it is enough - and I would be quite certain it's just a band aid. Unless you impose the construction of cheap housing, cheap housing will tend to not get built - almost every interest is opposed to it.
Good thing Mamdani is doing both, cutting red tape and constructing new housing, at least that's his plan, we shall see what others in his governance think about it.
Yeah, that's correct. Democrats aren't made the same. Austin has built the most housing. YIMBY-ism is just free market conservatism economic policy dressed up in liberal language. Most people vote based off of vibes, not actual policy. If you restate the case for YIMBY-ism in conservative terms, you get conservatives to vote for you and liberals shy away. If you restate it in social justice terms, then you get liberals, and conservatives shy away. Policy is the same. The people are stupid.
Palantir isn't collecting it, they are storing it and processing it after it has been collected by ICE.
Their defacto scan everyone isn't to determine citizenship, it is to collect the data in the first place. They want to get as much data into their system as possible.
The whole point of Apple intelligence is that it shouldn’t feel like you’re talking with an LLM, it’s that it fades into the OS but delivers new features that make the OS more intelligent. I’m actually very bullish on it
