In my experience, it's not that directories are inherently important, it's that an organization mechanism is, in the service of a few key problems:

1. Privacy and data handling requirements

2. Versioning

3. Partitioning

4. Probably some others I'm forgetting

Hierarchical storage is a useful all-purpose tool for these things.

I'm not so sure they actually believe the results are authoritative, I think they're being lazy and hoping you will believe it.

There's a small but substantial number of engineers out there who haven't operated at the kinds of scales where hyperscalers' limits become normal architectural problems and don't have the humility to imagine that it could be the case. (e.g. blob stores do in fact have limits you can hit, and when you operate at petabyte scales you have to anticipate in the architecture that you can hit them for even trivial operations.) I also work on petabyte datastores and have encountered a bunch of those engineers over time.

To be fair though, that's the small minority of engineers I've encountered, and if it wasn't arguing about the types of scale problems unique to petabyte scales, it'd be about some other nuanced subject matter. It's a humility problem.

* Conway's law causing multiple different data science toolchains, different philosophies on model training, data handling, schema and protocol, data retention policies, etc.

* Coming up with tech solutions to try to mitigate the impact of multiple silos insisting on doing things their own way while also insisting that other silos do it their way because they need to access other silos' data.

And the reason standardization won't happen: the feudal lords of each of those branches of the hierarchy strongly believe their way is the only way that can meet their business/tech needs. As someone who gets to see all of those approaches - most of their approaches are both valid and flawed and often not in the way their leaders think. A few are "it's not going to work" levels of flawed as a result of an architect or leadership lacking operating experience.

So yeah, it might look like technical problems on the surface, but it's really people problems.

- Requirements are rarely clear from the beginning;

- We (DE) are not enabling self-service and automation so we are drowned in small requests (add this column for example;

- Upstream rarely notify us about the changes so we only know when downstream alerts us. We end up building expensive pipelines to scan and send alerts. Sometimes the cost of alerts > cost of pipeline itself;

- We have so many ad-hoc requests that sprint is meaningless. If I were the manager I'd abolish sprint completely;

- Shadow knowledge that no one bothered to write down. I tried to write down as much as possible, but there are always more unknowns than knowns;

Working in DE definitely gives me enough motivation to teach myself about lower level CS.

Wrong answers only please.

To solve this, one can be an instrument for change. Network, band people together, evangelize better ways forward, all while not angering management by operating transparently.

Sometimes, that can work... up to a point. To broadcast real change, quickly, you really need anyone managing all the stakeholders to lead the charge and/or delegate a person or people to get it done. So the behavior of directors and VPs counts a lot for both the problem and the solution. It's not impossible to manage up into that state with a lot of talking and lobbying, but it's also not easy.

I'll add that technological transformation of the workplace is so hard to do, Amazon published a guide on how to do this for AWS. As a blueprint for doing this insanely hard task, I think it holds up as a way to implement just about any level of tech change. It also hammers home the idea that you need backing and buy-in from key players in the workforce before everyone else will follow. https://docs.aws.amazon.com/prescriptive-guidance/latest/clo...

Yup, this is the key issue and what makes it primarily a people problem. Technical solutions don't work if the main problem is getting buy-in to spec/build/adopt one, unless you're willing to build a lot of things you end up throwing out. So instead the bulk of the high risk work is actually negotiation between people.

I work in implementation of large enterprise wide systems. When I do projects that span departments/divisions/agencies what you’re describing is the biggest hurdle. The project always starts with “we’re bringing everyone together into one solution” but as time goes on it starts to diverge. It’s so easy to end up with a project per department vs one project for all. You have to have someone with the authority to force/threaten/manipulate all the players onto the same page. It’s so easy to give in to one groups specific requirements and then you’ve opened Pandora’s box as word spreads. It’s very hard to pull off.

I think public sector (governments) is the hardest because the agencies seem to sincerely hate each other. I’ve been in requirements gathering meetings where people refused to join because someone they didn’t like was on the invite. At least in a for profit company the common denominator for everyone is keeping their job.

* You have a disability that hinders your ability to type on a keyboard, so you need extra time to type the essay based exam through vocal transcription.

* You broke your dominant hand (accidents happen) so even though you know all of the material, you just can't write fast enough within normal "reasonable" time limits.

* You are blind, you need some way to be able to read the questions in the test. People who can see normally shouldn't need those accommodations.

I don't think those are cases where you are lowering the bar. Not by more than you are allowing the test taker a fairer chance, anyway.

The problem is when you get into the gray area where it's not clear than an accommodation should be given.

But to quote the article linked in the parent comment:

> The increase is driven by more young people getting diagnosed with conditions such as ADHD, anxiety, and depression, and by universities making the process of getting accommodations easier.

These disabilities are more complex for multiple reasons.

One is the classification criteria. A broken hand or blindness is fairly discrete, anxiety is not. All people experience some anxiety; some experience very little, some people a great deal, and everything in between. The line between regular anxiety and clinical anxiety is inherently fuzzy. Further, a clinical anxiety diagnosis is usually made on the basis of patient questionnaires and interviews where a patient self-reports their symptoms. This is fine in the context of medicine, but if patients have an incentive to game these interviews (like more test time), it is pretty trvial to game a GAD-7 questionnaire for the desired outcome. There are no objective biomarkers we can use to make a clinical anxiety diagnosis.

Another is the scope of accommodation. The above examples have an accommodation narrowly tailored to the disability in a way that maintains fairness. Blind users get a braille test that is of no use to other students anyway. A student with a broken hand might get more time on an eassy test, but presumably would receive no extra time on a multiple choice test and their accommodation is for a period of months, not indefinite.

For better or for worse, politics and randomization is just a thing in our jobs, and for at least some people in your team that means part of career growth is learning to handle it. If you, the manager, are the sole person capable of being the "shit umbrella" for the team, that's another way that your team gets a bus number of 1. (I learned this the hard way.)

In an ideal world you have some senior engineers who are more of the "don't bother me and let me cook" persuasion, and then you have at least one who is probably on track to become a manager, and they are your backup when you can't be in two places at once.

You could also argue that 100 out of 200 on a fair coin is more likely than any other specific outcome, such as 93/200, so if the argument is that the coin is "too perfect", you then also have to consider the possibility that the coin is somehow biased to produce 93/200 much more often than anything else, vs. 100/200.

I see why you'd think this, but I disagree. In my opinion it's two sides of the same coin, and the key moral question is whether you use those skills in a moral way. I've seen both well-meaning and charismatic people and not so well-meaning charismatic people, and at the end of the day I believe that charisma is a powerful tool, but it's not fundamentally good or bad.

Social interactions have always felt like a game whose rules I don't intuitively understand, and I've always envied people like my wife who handle it much more naturally and fluidly. The same way that I'm comfortable and capable in analytical settings, they navigate social settings with just as much finesse. I've personally spent a big chunk of my adult life trying to learn to navigate social interactions more comfortably and more intuitively, so I can see some parallels with what the author writes about. (For the record, I'm neurotypical, just awkward.)

For most people I don't think it's about charming, manipulating, or gaming social interactions, I think it's about wanting to make connections and friends because that leads to being happier.