for
> Calculate the number of bits to encode that value using log2(938933556), which is ~29.8
This is roughly same as saying: "If you rewrite 938933556 as a binary number / usize, it will need 30 bits".
Sanity check: 1101111111|0110111111|0100110100 (| delimits every 10 bigits).
> Since the file is 128 bits long, one would expect this place to be around the 2*128th bit.
This statement is a bit more subtle. As a first ord approximation, we can see pi sort of as a RNG.
If we write pi (ignore the decimal point), as a binary number, we get:
11011001111111011110010101011110001010101111101101110001001100001...
You can... kind of squint and pretend this is a random sequence of 1s and 0s.
Now, if you had a file that is 128 bits (so lots of intermingling 0s and 1s), and each next digit of pi is effectively a coin flip. Pretend 1s are heads, and 0s are tails. You basically have to get the exact 128 consecutive coin flips of the same result as your file to get your file back.
Imagine now, PI not as a number, but a sequence of experiments of flipping the coin 128 times.
You have to try, on expectation, quite a few times to win this game! Now, you could easily get lucky for sure. But on average, your chance of winning per attempt is roughly 0.5^128! So, how many times do you have to try to win this game? Something like 2^128 times - and you have to consider that each attempt uses 128 bits as well. So more like 2^135. But you don't have to start fresh in each attempt, you can see it as like this:
- 11011................00100...
- ( 128 flips )
- ( another 128 )
- ( )
- ... so on and so on
And indeed, I think we're the only agentic coding environment with jj support.
The most difficult code in the 1.0 release is some gymnastics to avoid the appearance of a concurrency conflict with a user running their own jj commands, made at the request of the person who introduced me to jj.
Yes*. At least from my limited usage of deepseek-flash for a few billion tokens on openrouter, the cache-hit rate is >95%. And I simply used the claude code harness pointed at the openrouter anthropic compatible endpoint with no fluff.
Did you get proper tool use? Some CC-driven models seem to get a bit off when it comes to MCP usage. For example: I really struggled to get Kimi to use Serena, which I think ended up costing too many tokens.
I've been using the em-dash for years, having started doing so well before the dawn of LLMs -- needless to say the fact it is used as a telltale sign of LLM writing, doesn't gladden me one bit. Also because I value concise writing through picking correct grammatical elements -- like the em-dash.
I am not sure what context Jensen said that. But midjourney uses tpu. Apple uses tpu. They are no other frontier labs that use it, but Google + Anthropic is 2 out of 3 frontier lab so.....
You could reasonably say that "A majority of frontier labs uses TPU to train and serve their model."
Mayhaps! But I think as far as google, anthropic[1] and apple[2] goes, they do use the tpus for training. Ofc v4 and v5 (older generations of tpus) were more specialized for search related embedding workloads and i could see people not using them for training.
[1]: We train and run Claude on a range of AI hardware—AWS Trainium, Google TPUs - April 6th, Anthropic on Google and Broadcom partnership
[2]: "[Apple foundation model]... builds on top of JAX and XLA, and allows us to train the models with high efficiency and scalability on various training hardware and cloud platforms, including TPUs and both cloud and on-premise GPUs" - Apple in 2024
There are many good criticisms against data center. And yet, the water issue always comes up first. Must we spew false/untruthhood just so our political message is catchy? I suppose yes - in times of war/politics, the laws/truths are silent. But it doesn't have to be so here.
I've never had it come up first. Neat how 2 people can have 2 opposite experiences based on their different life paths.
Anyways: Between our 2 opposite experiences, it might as well be totally random, so I don't think the ordering of concerns is that important. Better to focus on substance, like the concerns themselves.
Huh. I initially thought this is just another finetuning end point. But apparently they are partnering up with customers on the pretraining side as well. But RL as well? Jeez RL env are really hard to get right. Best wishes I guess.
