Very cool. I do something like this but with Playwright. It used to be a real token hog though, and got expensive fast. So much so that I built a wrapper to dump results to disk first then let the agent query instead. https://uisnap.dev/

Will check this out to see if they’ve solved the token burn problem.

I use playwright CLI. Wrote a skill for it, and after a bit of tuning it's about 1-2k context per interaction which is fine. The key was that Claude only needs screenshots initially and then can query the dev tools for logs as needed.

Mostly, yes: https://github.com/microsoft/playwright-cli

my workaround for this was to make a wrapper mcp server which uses claude haiku to summarize the page snapshot returned in the response of each playwright mcp call, and that has worked pretty well for me: https://github.com/jsdf/playwright-slim-mcp

One of the creators of R2D3 here. Funny to wake up to this today! Happy to answer questions here or on bsky

Any plans for more articles, 10 years later?

If I would like to build a visualization like this, but for a data ingestion pipeline, any tips on where to start?

I have it visually in my head, but it feels overwhelming getting it into a website.

fwiw I work on data ingestion pipelines and I've found that starting with just boxes-and-arrows in something like Excalidraw gets you 80% of the way to knowing what you actually want. The gap between "I can picture it" and "I can build it on a webpage" is mostly a d3 learning curve problem, not a design problem.

xyflow that the creator mentioned is probably the right call for pipeline DAGs though -- we use it internally for visualizing our scraping workflows and it was surprisingly painless to get running

Sort of like this? https://docs.tecton.ai/docs/introduction/interactive-tour I used https://github.com/xyflow/xyflow for this, with css animations for the edges. It’s probably easier now with coding agents and what not

That workflow sounds amazing. How do you set that up? Got any code for it that we can look at?

Yeah! This was done before CSS scroll-snap is a thing. I'd probably use scroll snap now. See: https://developer.mozilla.org/en-US/docs/Web/CSS/CSS_Scroll_...

Second this. Idyll is the closest thing I know that makes this sort of thing approachable. Those dot animations are unfortunately very custom.

Funny to see this on the homepage today. Part 1 was done so long ago now that looking back on that code is equal parts intimidating and embarrassing.

Maybe this is the motivation I need to actually fix that stupid feet/meter thing.

Haha. I remember this. Thanks for creating this back in the day. It was good fun and a good intro.

Author here – Part 2 uses:

- React for DOM rendering

- D3 for visualization geometry

- RxJS for handling state (i.e. turning scroll position into animated properties)

- PIXI for WebGL bits (i.e. all the little circles moving around)

Ah shoulda mentioned I looked at part 1's minified code and assumed the same for part 2. Thanks for clarifying :)

Thanks Brandon! We're certainly thinking about it. We'll work out way up there.

OP here. What kinds of sources are you looking for? We built this based on some of the lectures in Stanford's statistical learning class (http://online.stanford.edu/course/statistical-learning-winte...) as well as just hanging out with the ML team at http://siftscience.com

Do you have a gh repo? I really am interested in the story telling and ML aspects of the post.

The decision tree model was generated from http://scikit-learn.org in Python. The JS is a complete mess, but I'll try to write up how it works in the next couple days.

I'd love to see the JS too. Great viz!

Thanks! Yeah absolutely. We are working our way up to those. Our tentative plan is to tackle bias-variance trade-off next, then random forests, then neural networks.

oh for sure, just jumping in to these topics is difficult enough. i'm looking forward to what you guys put together next