This is awesome! Worked on something similar albeit a different industry.
For the more challenging scrapes, would highly recommend using the Chrome Devtools MCP to be able to attach the network requests, being made by the browser to the site, as context for your agent/LLM chat - this approach really helped me to write a solid API-based scraper (also using curl_cffi) and bypassed the old tedious playwright-based approach I used to rely on.
Nice thinking. Hadn't thought of DevTools MCP that way. Curl_cffi I've used for TLS fingerprinting (Edinburgh was the first one) but the discovery side I've been doing manually... open DevTools, look at the request, copy as cURL, work out which params can be pruned. Automating that loop with an LLM in the middle would speed things up a lot, especially for the bespoke long tail. Will look into that this week. Thanks!
Anything that makes email development easier is great I guess, but have personally found MJML great for solving the issues you'd run into, and not sure I want yet another abstraction layer on top of that which makes it more limited...
If the goal is to write emails purely using AI, then it is trivial to attach the MJML documentation as context to your LLM using context7 MCP or something of the sort. It's not a very complex language and its documentation is not large at all.
That's assuming the crawlers haven't ingested it all already...
For the more challenging scrapes, would highly recommend using the Chrome Devtools MCP to be able to attach the network requests, being made by the browser to the site, as context for your agent/LLM chat - this approach really helped me to write a solid API-based scraper (also using curl_cffi) and bypassed the old tedious playwright-based approach I used to rely on.