LLMs can execute searches? You can absolutely send ChatGPT to look for a cheap flight and it will do pretty well. And because I am paying for ChatGPT rather than the advertiser's, I am the customer and not the product.
You may pay to ChatGPT, but sooner or later you will become their product too. All the conversations you had or will have will be turned into signals to match you with products from advertisers, maybe not directly in the conversation with them, but anywhere else. It's not a mater of if, but looking at the pace things are going, and how financially pressured openai is, it's only a matter of time that their conversations with them will be turned into profit in some way or another, they basically have no choice financially.
I disagree. Julia has correctnes issues because it chose maximum composability over specifying interfaces explicitly. And those are not just in immature packages but also in complex packages. Compared to other languages, Julia has no facilities to help structure large complex code bases. And this also leads to bad error messages and bad documentation.
Recently we got the public keyword, but even the PR there says:
"NOTE: This PR is not a complete solution to the "public interfaces are typically not well specified in Julia" problem. We would need to implement much than this to get to that point. Work on that problem is ongoing in Base and packages and contributions are welcome."
What if I want the nth element up to the math element? arr[n:m]. And if I want to split the array into two parts, one until the nth element and the other from the m+1st element arr[1:m] and arr[(m+1):end]. Julia matches how people speak about arrays, including C programmers in their comments. Arrays are (conceptually) not pointer arithmetic. Also for your usecase typically you would just use a 2d array and write a[n,:].
so you just need to overload the syntax of intervals even more to make it work
> arr[0..m], arr[m..]
now `m` refers to different things depending on which side of the interval it's on. less characters doesn't mean nicer
I get it though, I was skeptical about 1-based indexing when I started Julia. By the nature of indices vs length there will always be an off-by-one problem: either you have elements [n, m - 1] with length (m - n) or [n, m] with length (m - n + 1). Unless you're doing a bunch of pointer arithmetic type stuff, I find the symmetry of a inclusive-inclusive interval to be a better default.
Your second point is the main argument for me personally. Numbers in brackets always mean the same thing: the ordinal number of the references object in an ordered collection. In 0 based indexing you can think of the number as refering to the space between the referenced objects. But that is simply an additional mental image on top of the original one.
As a neat bonus, in Julia 1:5 is just the iterator for the numbers 1 to 5. So slicing is typically not some special syntax either. It all works rather nicely.
So if I have a row of 5 apples, I can say "take the second and third apple" or I can say "take the apples between one apple length and three apple lengths from the start".
Which is more natural? The ruler is exactly the right mental image if an array to you is a partitioned region of memory starting at a specific pointer location. If an array to you is an ordered collection of objects, you would never invent 0-based indexing or inclusive-exclusive slicing.
Either way, it's not a big deal. I have lived in both worlds, I have come to think Julia is a bit more natural and easier to teach. But it ls really the silliest bike shedding complaint, given that the language has considerable real trade offs.
This is such a classic example of online discourse in general. There are two options, and folks tribally cling to one or the other without realizing that both are legitimate and well-suited for different situations.
Yes, of course distances are measured starting from 0. But we count discrete things starting at 1. You can do mental gymnastics to enumerate from zero and many programmers are (unfortunately IMO) taught to do so. It's a hard thing to learn that way, so for the folks that have done so, it often becomes a point of pride and a shibboleth.
As a classic example, a four story building has four floors. But you only need to go up three flights to get to the top. You can legitimately call the top floor either 3 or 4, and folks are similarly tribal about their own cultural norms around this one, too.
Fully agreed. I first struggled when switching from python to Julia, then ended up finding it slightly better for my use cases (which includes teaching scientists who are not programmers). But it's simply not a big deal either way. I am also reminded of the significant whitespace objections to python in the old days, before python took over everything...
> There are two options, and folks tribally cling to one or the other without realizing that both are legitimate and well-suited for different situations.
No I disagree entirely. One is simply better.
> It's a hard thing to learn that way, so for the folks that have done so, it often becomes a point of pride and a shibboleth.
It is not hard. It's not better because it's hard-won knowledge. It's better because it leads to simpler, more elegant code. Simple as.
Thanks for correcting; I know that "loss function" is not a good term when it comes to transformer models.
Since I've forgotten every sliver I ever knew about artificial neural networks and related basics, gradient descent, even linear algebra... what's a thorough definition of "next token prediction" though?
The definition of the token space and the probabilities that determine the next token, layers, weights, feedback (or -forward?), I didn't mention any of these terms because I'm unable to define them properly.
I was using the term "loss function" specifically because I was thinking about post-training and reinforcement learning. But to be honest, a less technical term would have been better.
I just meant the general idea of reward or "punishment" considering the idea of an AI black box.
The parent comment probably forgot about the RLHF (reinforcement learning) where predicting the next token from reference text is no longer the goal.
But even regular next token prediction doesn't necessarily preclude it from also learning to give correct and satisfying answers, if that helps it better predict its training data.
I didn't, hence the "first". It's clear that being good at next token prediction forces the models to learn a lot, including giving such answers. But it's not their loss function. Presumably they would be capable of lying and insulting you with the right system prompt just as well. And I doubt RLHF gets rid of this ability.
If you didn't forget about the RLHF, your comment is oddly pedantic, confusing and misleading. "Correct and satisfying answers" is roughly the loss function for RLHF, assuming the humans favor satisfying answers, and using "loss function" loosely, as you yourself do, by gesturing at what the loss function is meant to do rather than formally describing an actual function. The comment you responded to didn't say this was the only loss function during all stages of training. Just that "When your loss function is X", then Y happens.
You could have just acknowledged they are roughly correct about RLHF, but brought up issues caused by pretraining.
> And I doubt RLHF gets rid of this ability.
The commenter you were replying to is worried the RLHF causes lying.
The EU doesn't really fund many things directly. It's total annual budget is just 170 billion euros. It can fund research and coordination projects but at the end of the day the EU is mostly a coordination mechanism for sovereign states. Looking purely at EU projects is not really a useful lense to get an idea of what is happening...
> Or you put the data centers at different points on earth?
> Or you float them on the ocean circumnavigating the earth?
What that does have to do with anything? If you want to solar-power them, you still are subject to terrestrial effects. You can't just shut off a data center at night.
> Or we put the datacenters on giant Zeppelins orbiting above the clouds?
They'd have to fly at 50,000+ ft to be clear of clouds, I doubt you can lift heavy payloads this high using bouyancy given the low air density. High risk to people on the ground in case of failure because no re-entry.
> If we are doing fantasy tech solutions to space problems, why not for a million other more sensible options?
How is this a fantasy? With Starlink operational, this hardly seems a mere 'fantasy'.
A capacity problem can be solved by having another data center the other side of the earth.
If it's that the power cycling causes equipment to fail earlier, then that can be addressed far more easily than radiation hardening all equipment so that it can function in space.
Because GPUs are expensive, much more expensive than launch costs if they get starship to the low end of the range they’re aiming for, and you want your expensive equipment running as much as possible to amortize the cost down?
But the GPUs on the ground will be a lot cheaper to manufacture as they don't have to deal with space conditions.
It seems a real stretch to me to assume that costs for putting GPUs into space can ever come within a factor of 2-3 of putting them on the ground, even neglecting launch costs.
The political issues in space are mostly launch related, right? Once you have the birds up nobody cares about anything except space junk and bandwidth. They're getting experience of solving those with Starlink already. And if you can find a way to put the satellites really far out there's plenty of space - inferencing satellites don't need to be close to Earth, low latency chat stuff can stay on the ground and the flying servers can just do batch.
The politics on the ground is much harder. Countries own the land, you need lots of permits, electricity generation is in contest with other uses.
This is science for science sake. To advance knowledge. You argue for direct investment for building space faring capabilities, rather than for advancing knowledge, but then you should state why you want these space faring capabilities.
Because they are cool? I think that's essentially the reasoning behind putting people on the Moon. If you believe there are valifld economic reasons for space, why do we need tax dollars? And also: I don't see it. Space is so hostile an environment to humans, that it's hard to understand why we should invest in capability to be there personally. People aren't even investing to harvest Greenland's resources, and they are infinitely more accessible.
For starts for science itself. If transporting things to space wasn't so costly, so much more experimentation would better be done in space.
Yes it's cool, but that isn't why. It's because of resources and human progress.
I want mirrors in space that direct/magnify sun light to destinations on earth for example.
So many of human problems today have to do with resource scarcity. The "old" world had this, and the discovery of the "new" world in the west solved lots of problems.
Every tech we have to day, every advancement in medicine, industry,etc.. from steam engines to the internet and AI is a result of the discovery of the americas. the center of trade and commerce shifted to western europe, and western europeans used lots of means including conquest of the americas by force , enslaving africans,etc.. to get gold and riches from the americas back to europe. The improvement in the quality of life for western europeans meant they could focus less on subsistence and survival, and focus on science, industry and reformations.
Space is extremely hostile to humans. You're not wrong about that, but it isn't beyond humans' capability to conquer it. Solving the obstacles in the way of space expansion requires solving things that have the ability to improve humans' lives greatly. It means we could also conquer the polar zones and tundras much more easily, and be more resilient to climate change. Like how western europe benefited from the Americas, and how they pillaged gold from the americans, so is there gold and riches in space to be pillaged and improve the lives of the world on earth. Back in the 1800s, the west was a hostile (no comparison to space of course) place people with no options went to, space could be that for a while.
Humanity can't survive in a stagnant way. We will always need more space and more energy.
Quantum Mechanics, protons, electrons... That's the theory of everyday matter. You don't need very special situations to see their effects. Understanding the underlying equations enabled us to do more with what we already have.
High energy stuff only exists unstably for fractions of seconds. I find the idea that any of Standard Model physics, nevermind beyond standard model physics, could lead to a technological advance like the transistor extremely unconvincing.
Technological advance and scientific advance sometimes align. But there is no law that the former by necessity follows from the former. The expectation that they do is an extrapolation from a very brief period of human history.
reply