> No built-in cross-database querying, making complex analytics difficult without a dedicated data lake
I've looked at Turso before and this is exactly what came to mind. I do see some use cases where it could work, but for most real-world applications, this is an absolute red flag that shouldn't be overlooked.
This seems to be the biggest hesitation I've heard over and over by far. There absolutely needs to be a good story here for both (a) ad-hoc cross-partition queries and (b) automatically building a datalake without having to know what ETL stands for.
However, this isn't so much different from Cassandra/DynamoDB which have a similar problem. You _can_ query cross-partition, but it's strongly discouraged and will strain any reasonably sized cluster.
Hustle culture IMO is when you impose long working hours to your employees, when you judge your coworkers by their working time or do not give a promotion to someone because he has a healthy work-time balance.
It's not about how much you work.
I've been working hard at times, I still do from time to time but I never impose it to anyone. And when I'm in one of these periods when I push myself I do extra care of not showing it; not pushing code out of normal hours, not sending slack or emails,...
Because even showing you're doing it is already a step in the bad direction, it normalizes it, and ultimately create this hustle culture.
Getaround EU | https://uk.getaround.com | Paris, anywhere in France or Belgium ONSITE or REMOTE | Backend, Full-stack, iOS, Analytics Engineer, Product Designer, Data Analyst
About Getaround Europe:
We are on mission to remove car ownership in European dense cities. We believe cars should be shared.
* We're a ~50 people P&E Team
* We have tons of challenges; pricing recommendations, ranking, connected cars that needs to be ultra reliable (it needs to replace your owned car), we provide APIs to different typologies of partners,...
We use Google for oauth, map and geolocation APIs for which we pay thousands of dollars every month and all my interactions with them have been very painful.
I remember when we asked for a meeting to discuss price raise on Google Maps APIs; The raise was ~100x for us.
They came with the head of the local market and 4 people. Basically told us to fuck ourselves and that from now on we'd have to be billed by a reseller instead of being billed by Google.
And then they spent an hour trying to convince us to move from AWS to GCP. I had the impression I was at a car dealership.
This is just an example out of many so even if I don't have direct experience with GCP I'm not keen to try.
I've been managing the AWS company account and the collaboration with AWS for the past 7 years.
And while not everything is perfect, my experience was great overall and I would do it again.
We're currently using InfluxDB but maintenance is something we'd like to stop doing on our monitoring stack.
Datadog is too expensive because of the number of hosts we have. So we're thinking of eventually going to a hosted InfluxDB setup.
But we also want to revisit other hosted solutions. Does someone have some experience with using Cloudwatch + Grafana? I've used Cloudwatch many years ago and it was clearly subpar to something like Influx. Is is better nowadays?
Thanks for the article.
I've been in tech for the past 10 years, working in or around devops teams for the most part but I don't get all the fuss about k8; yes it's an amazing tool doing a lot more than any other.
But it has a big learning curve and setup & maintenance are very costly. I don't understand why most orgs are moving to k8 considering this. When talking to my peers I often see numbers like 6 months to 2 years full migration with a very small added value - at least for 90%+ of the companies using it.
It usually boils down to attracting talents and keeping them excited trying the new shit.
There is a learning curve either way. It’s either an in house container orchestration platform or one of the open source ones. Coinbase seems to have chosen the former.
K8s is simply a good default environment which provides rock solid stability for your applications by outsourcing the distributed systems complexity to your infrastructure team (whether its internal to your company or to a managed one like GKE). Teams are not using it just because it’s “cool” (maybe some are), there is no need to develop in house strategies to deploy and keep an app running and scale it (among other things; this is the lowest common denominator).
It’s the same reason why big data tech has somewhat standardized on a set of tech (spark, airflow etc): once people learn the system, they can focus on building products that provide value rather than building the products and the relevant infrastructure.
Everyhting goes through S3 because Snowflake storage is on it.
dbt is amazing, we began using it a month ago and it already transformed the way our data team work. It really is a value multiplier for everyone. Data engineers are happier because they don't need to write and maintain data transformations, analysts are happier because they can maintain their own SQL pipelines & the whole company is happier because we now have a great documentation tool to explore our data.
We also are big fans of Snowflake, make operating a data warehouse a breeze.
Then, we use a mix of Redash & Tableau for reporting.
Redash for static reporting (open to the whole company) & Tableau to create more complex data tools we expose to some internal teams; Marketing, Risk, Finance ...
https://michaelbensoussan.com - personal website
https://touslesmemes.fr - FR politician quizz game (who said what?)
reply