Hilarious name for an open source project released by a government lab.

Reminds me of Moloch, the packet capture and indexing software[0]. I wonder why it was renamed and moved out of the AOL github org...

[0]: https://news.ycombinator.com/item?id=22949604

Based on the name initially I thought Wiretap was a MITM proxy for Wireguard.

> For me, it was simply a gut feeling. I’ve been talking to founders and doing deep dives into technology companies for decades. It’s been my entire professional life as a writer. And because of that experience, there must be a pattern-matching algorithm churning away somewhere in my subconscious. I don’t know how I know, I just do. SBF is a winner.

This is a fantastic insight into journalists. They need to fit their story into a narrative. If you match their stereotype, then they'll spin whatever tale they can imagine to convince themselves that it's really true. The level of self-deception that's needed is difficult to comprehend for me.

Is it common to have the same number of employees as there are companies funded at a VC? Is each one assigned a company?

Brilliant way to get thousands of tech people to look at your business card.

Edit: This was updated, it used to point to his real Github username. Not that there's anything wrong with that! It's clever marketing.

I didn't look what does it say?

It says he works at Dunder Mifflin, it doesn't look like its about self promotion

Aside from the project itself, the production value of this video is crazy good.

JAX is a DSL on top of XLA, instead of writing Python. Example: a JAX for loop looks like this:

   def summ(i, v): return i + v
   x = jax.lax.fori_loop(0, 100, summ, 5)

A for loop in TinyGrad or PyTorch looks like regular Python:

   x = 5
   for i in range(0, 100):
      x += 1

By the way, PyTorch also has JIT.

I've just tried making a loop in a jit-compiled function and it just worked:

    >>> import jax
    >>> def a(y):
    ...   x = 0
    ...   for i in range(5):
    ...     x += y
    ...   return x
    ...
    >>> a(5)
    25
    >>> a_jit = jax.jit(a)
    >>> a_jit(5)
    DeviceArray(25, dtype=int32, weak_type=True)

It definitely works, JAX only sees the unrolled loop:

  x = 0
  x += y
  x += y
  x += y
  x += y
  x += y
  return x

The reason you might need `jax.lax.fori_loop` or some such is if you have a long loop with a complex body. Replicating a complex body many times means you end up with a huge computation graph and slow compilation.

And how does TinyGrad solve this?

Fused into one operation since the Tensor isn't resolved until I call .numpy()

  kafka@tubby:/tmp$ cat fuse.py 
  from tinygrad.tensor import Tensor
  x = Tensor.zeros(1)
  for i in range(5):
    x += i
  print(x.numpy())

  kafka@tubby:/tmp$ OPT=2 GPU=1 DEBUG=2 python3 fuse.py 
  using [<pyopencl.Device 'Apple M1 Max' on 'Apple' at 0x1027f00>]
  **CL**      0 elementwise_0        args     1  kernels [1, 1, 1]          None         OPs     0.0M/   0.00G  mem  0.00 GB tm      0.15us/     0.00ms (    0.03 GFLOPS)
  **CL**        copy OUT (1,)
  [10.]

How does this differ from XLA? Would tinygrad's lazy approach also just see the same unrolled loop right before compilation?

Previous discussion: https://news.ycombinator.com/item?id=22374825

Wonder if anyone's turned Instagram's version of Blurhash into a library?

I can't think of anything that neural nets can't beat, except small tabular data with boosted decision trees. Can you give some examples?

Explicability is a big part of it It is often worth being a percent less accurat but having an explainable result.

I've been on a lot of ML teams and outside of Finance and a few other sensitive topics explainability has always been irrelevant.

What happens, when your model exhibits a discriminating bias? How do you find out, what is going wrong? Knowing, what the model pays attention to can be pretty helpful.

Not aware of any court cases where someone successfully sued because they were shown one product recommendation or Ad on a webpage instead of another.

(I don't really agree with GP's point but for the sake of answering your question)

1. Collaborative filtering based on a sparse dataset of implicit interactions.

2. Many time series applications.

Didn't all recommendations engines move to two-towers like models? I remember that it "solved" the freshness problem (ie when adding a new item to your catalog how do you recommend it to users if there are no ratings/interactions). Of course as long as you have a good model that creates items embeddings.

Regarding time series, don't everyone moved to attention based models?

Not challenging your answer, just curious. I work mostly with Graph NNs and quite a bit out of touch with the rest of the field.

Small data problems, where’re never the less have a really good idea of how things are causally related.

> we often use ML over DL in scientific analysis because we need models that can be inspected/explained not just results

> also, DL generally requires more data whereas you can get by with ML on less data if you have domain knowledge

The black box nature of a neural net is a problem. For model based design, a bit more accuracy out of a black box doesn't really help when you need, for example, state space matrices in a control design.

The code is very easy to read. Doesn't seem like there's data/model parallelism support for training, which will be important for real-world use.

https://github.com/intel/intel-extension-for-pytorch PyTorch version. Looks like it's been supported a lot longer than TensorFlow, the repo dates back over a year.