Hacker Newsnew | past | comments | ask | show | jobs | submit | larsf's commentslogin

A Composable Just-In-Time Programming Framework with LLMs and FBP


Composable is amazing at ETL - https://composableanalytics.com

It blows things like Alteryx, NiFi, Airflow out of the water.


Please add a disclaimer if you work for these guys... I personally generally dislike enterprise solutions because I hate talking to sales people - but that's just me.


Yes - I'm the founder. And you don't have to talk to me if you don't want to :)


You can use Composable's Fluent API to author and run dataflows in code. No GUI required. Composable's platform is open and completely extensible in sense that anyone can add applications or first class modules to the system, including swapping out some of the more internal components.


Pipes are probably the original instantiation of dataflow processing (dating back to the 1960s). I gave a tech talk on some of the frameworks: https://www.youtube.com/watch?v=3oaelUXh7sE

And my company creates a cool dataflow platform - https://composableanalytics.com


http://doc.cat-v.org/unix/pipes/ . And there's a bit more about how pipes came to be in unix here: http://cm.bell-labs.com/who/dmr/hist.html


Composable Analytics is a similar solution, but users create the apps using a dataflow methodology (sexy apps). Allows for much more flexibility because you can change the guts of the data processing. It's also very performant because parts of the app can be parallelized.

http://composableanalytics.com


I think businesses still need flexible / easy to use systems that allow end-users to create solutions quickly. This may involve analysts, IT professionals, and devs working together on the same platform. For examaple, an IT pro writes the sql queries, an analyst writes the regression algorithms, and the devs writes the output adapters.

Typically, by the time you get the dev team to fully implement the solution, it has missed its mark and the analysts have moved on.

Players in the mashup landscape are "trying" to provide scalable and robust, yet flexible and easy-to-use systems.

plug - flowreports.co is one of these ... and it can be self-hosted.


Businesses have hundreds of flexible easy systems to let end-users create quick business rule based solutions (particularly for reporting purposes) and have had them since the late 80s maybe earlier, the corporate landscape is littered with them. I wish you the best of luck, that's a tough market to get into.


Thank you for the feedback. Perhaps if we add things like support for version control systems, play well with releasing code to multiple environments, and add UI elements that enforce cloning chunks of code so that changes are isolated and the system can maintain it's coherence even among disparate users we can avoid becoming some of those issues. Something to think on...


That's definitely a step in the right direction. The biggest hurdle you'll have to overcome is getting your application to enforce a process, that's a lot harder than you think it is because people tend to take the path of least resistance and process is rarely that path. You'll have to get buy-in from very high levels of any organization you work with, otherwise things will devolve quickly. Either way, good luck, I'll check out your product demo when you've completed it.


Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: