How would you get ordering for millions of users? How many Kafka topics would you create? How would you managed failed events, would you reorder whole queue?
I don't think it is inefficient. Segment blog linked below talks about specifics of the problem.
One issue I had with Segment, I couldn't run real-time transformation of the event to join data from our data tables. We eventually got over it with AWS Lambda and sending it back to Segment. Segment recently announced functions to help on this, still could not get my hands on it.
Yes, that is exactly the use case of our "user defined" transformation functions. You can define any javascript function (right now by modifying the code but will be available from the UI in the release coming next week). Inside that function you can filter/transform/enhance the event in any way you like. You can lookup your DB, call external APIs etc. You can also combine multiple events into one.
Since this whole thing runs inside your VPC, you don't have to open up your production database to 3rd party as you have to do with segment.
Happy to work with you on your use case. Please email [email protected]
Hey there! I work at Segment, and I'm one of the engineers working on Segment Functions. If you let me know your Segment workspace's name (get in touch through [email protected]), I can grant you beta access.
Code Astra | Hyderabad, India | JS Developer | Full Time
CodeAstra is an IIT Alumni founded tech startup located in Hyderabad.
CodeAstra strives to provide the best solutions to its client by working on cutting-edge and apt technologies. The team consists of ninja developers, all from IIT Madras. Now the team is looking for rockstar developers to increase the team size and reach for the clients.
Benefits:
* Flat hierarchy, report directly to CEO/CTO
* Weekly tech talks, OSS development
* Cutting-Edge technologies, Fast Paced environment
https://github.com/rudderlabs/rudderstack-helm