It works great when everyone is delivering day-long or week-long incremental features that lend themselves to nice screenshots.

But then you slowly start accumulating a backlog of difficult tasks that everyone avoids because they won’t sound good for the week. Technical debt accumulates as people cut corners in the interest of getting their weekly presentations out.

You can theoretically avoid it with enough trust, but the trigger for our collapse was a round of layoffs where the people with the most visible features were spared while people working on important backend work and bug fixes got cut. After that, it was game over.

While working at a big corporation we had a velocity initiative supposedly aimed to lead the company toward continuous integration.

"How long a PR stays open" was one of the KPIs in a dashboard.

I said: "Be careful with that!"

People started to close PRs and reopen new PRs with the same code.

Middle managers and sometimes the person in the division that was the point of contact for the velocity initiative were asking to do that.

The script measuring this KPI was improved to look at the branch name and the code diff. Result? People changing branch name and a change of EOL encoding in the new PR.

Learnings? B and C players with questionable ethics screw companies quite rapidly.

In this climate KPIs and aligning them with company values is futile.

Within a couple of weeks, scripts were circulating to auto-generate and auto-commit tests. If the JVM we were using had a bug in adding random numbers together, we'd have known about it very quickly.

That's about the time I decided I should move on. I'm glad I did. The company I joined treated developers like adults and developers acted like adults. And we had great automated test coverage.

Obligatory mention: https://en.wikipedia.org/wiki/Goodhart%27s_law

However, I don’t think mindlessly creating tests is acting in good faith. It’s bordering on malicious compliance. I doubt they were thinking they can just knock out that metric so they can otherwise create better test coverage. (The OP conceded their coverage wasn’t good). Better employees would work to create a better understanding/goal. All of that points to some cultural problems.

If I was a developer there, I'd totally started adding those generated tests. I have a job to do (ship products) and you put in some stupid requirements which actually interfere with it (since my time is limited I can either work on tests or my daily tasks like developing new features or fixing bugs), but we both know that if my primary job suffers, I'll pay for it. So the best solution, from my point of view, is the one that takes that requirement away and lets me go on actually working.

When we had a similar problem in a previous company, we just created an epic, assigned a couple of people to it and have them churning out tickets for specific improvements, like "Class X has a coverage of Y on this method, add tests for the missing execution branches", which were clear, non-generic and fully integrated in our flow. If anybody complained about our velocity or whatever, we could show them the full trail which lead us to choose to work on that ticket and how much we spent on it. The coverage issue got solved in less than a month.

If I were a manager there, I probably wouldn’t hire you. You want people who are actively solving problems that matter, not just automatons beating their chest about how stupid everyone else is while they add to existing problems.

Any smart manager, having to deal with this kind of policy, would either push back or approve any way to game it and then get the work done. But I doubt that a company which allows this kind of BS can retain smart managers for more than a couple of months.

I just hope to never work with you and, especially, never use anything you helped building.

I made the point that the metric used (raw number of tests) is a bad way to enforce a good goal (higher code quality). I also made the point that metrics aren’t inherently bad, but they have to be chosen judiciously.

You seemed to take those two points and extrapolate an entirely different story as a personal affront and apply motives that, frankly, reads as a bit unhinged. So i also hope we don’t cross paths, because in safety critical code development where I come from where bad practices and toxic teammates can kill people.

But if the devs at the org all (or even substantially) tend towards malicious compliance, that's a sign that something has gone wrong, relations-wise

I scrap all incentivized metrics when working on something urgent (important and soon), which is often the case. If the metrics somehow incentivized, we'll start gaming it.

Now if a dev has part on production support and automated tests are purposed to reduce the workload for support then it has time slots for that, I bet everyone will start doing it.

The company tanked six months after that, now it doesn't exist anymore.

There's only so much you can do when the management is hellbent on doing stupid things.

In that context, I’m not really sure what point you’re making, unless it’s just to share a personal anecdote. Are you implying that management shouldn’t have any quantitative measures and should only be qualitative?

If you sell, say, water bottles, you probably want to know how many of them you can sell at any given moment, in order to not overbook and have to reimburse people. In this case, keeping track of how many water bottles you do have in stock probably helps, keeping track of how many labels with funny jokes you can stick on a shipping box in an hour doesn't. But if you start tracking the latter and handing down bonuses and layoffs based on it, people will max that metrics out - at the expense of your actual stock capacity.

Quantitative measures are dangerous, especially in the hands of people who believe they are better than qualitative ones because they're "objective" or whatever. Because not only they aren't, but they are also better than qualitative ones at hiding their biases and soothing your own.

> Are you implying that management shouldn’t have any quantitative measures and should only be qualitative?

Many managers would do a lot better this way. They'd still make stuff up, but would at least be forced to admit it.

You can make an engine as effective as the laws of physics let you, but you can't solve the limits of thermodynamics. You can only do your best within them. Same with this, for maybe the same reason.

It helps to have someone with a hacker mindset think about the metric being designed so the obvious ways in which it could be games are taken care of and their own metrics/incentives are aligned with the company goal.

How do you know what was really going on?

"He created a requirement that every developer commit two new tests to the code base every workday" seem a stupid requirement if you do not control the quality of the tests.

The same big corporation I wrote above had a goal of 80% code coverage reported on dashboards.

I saw people writing tests just to run lines of code, without effectively testing anything.

Others people were "smarter" and completely excluded folders and modules in the codebase with low coverage from coverage testing.

Code coverage percentage numbers on a dashboard are a risky business. They can give you a false sense of confidence. Because you can have 100% code coverage and be plagued by multitude of bugs if you do not test what the code is supposed to do.

Code coverage helps to see where you have untested code and if it is very low (ex: less 50%) tells you that you need more tests. An high code coverage percentage is desirable but should not be a target.

The real problem is again the culture.

A culture where it is ok to have critical parts of the code not being tested. A large part of the solution here is helping people to understand the consequences of low code coverage. For example collecting experiences and during retrospectives point out where tests saved the day or how a test may have saved the day so people can see how test may save them a lot of frustration.

But again, when you give people a target and it is the only thing they care about, people find a way to hit it.

> When I worked at a FAANG ~15 years ago, a new VP came in and heard, correctly, that our group didn't have enough automated tests.

If a team has identified a weakness in testing and transparently reported it, presumably with the intention of making it better, then why would we assume that setting arbitrary targets based on some metric with no direct connection to the real problem would help them do that?

if team does not have automated test, but still manages to deliver working software - maybe tests are not adding as much value as VP thinks?

the most important is feature delivery, and integration test, not automated unit test where you test getters and setters with mock dependencies - absolutely useless busywork

Schedule isn't always the most important thing either. It's possible delivery the software may just mean you've been rolling the dice and getting lucky. The Boeing 737MAX scenario gives a concrete example of where delivery was paramount. It's a cognitive bias to assume that "since nothing bad has happened yet, it must mean it's good practice"

Also "not testing a lot" is not a chesterton's fence. "not testing a lot" can't be load-bearing.

The culture lead to fake tests instead of adding tests that were legitimately lacking.

Is that so different from saying they weren't acting like adults?

The phrasing could be called dismissive but I give that a pass because it was mimicking the phrasing from the post it replied to. The underlying sentiment doesn't seem wrong to me.

Or, conversly, I've been in charge by really awkwardly testable code.. which ends up being really reliable. Plugin loading, config loading (this was before you spring boot'ed everything in java). We had almost no tests in that context, because testing the edge cases there would've been a real mess.

But at the same time, if we messed up, no dev environment and no test environment would work anymore at all, and we would know. Very quickly. From a lot of sides, with a lot of anger in there. So we were fine.

If end of sprint came and you weren't done, the manager would close out the ticket, then reopen another similar one named "Module phase 2" or something similar for next sprint. One guy was an expert at gaming the system, and his ticket got closed and opened anew for about 3 or 4 sprints.

No one should be surprised when employees respond to incentives, and blaming them seems a clear indicator of managerial failure: failure to tend to morale, failure to reward actually useful behavior, failure to articulate a vision.

Also, with an outstanding company culture, KPIs aren't really necessary.

So, when would they be useful?

I'm not as negative on KPIs as the previous line suggests though. They can be useful to shape direction when used carefully. But don't make them too long-lived, discard and create new ones as soon as they become gameable.

Gameable KPIs offer windows into the souls of your colleagues.

"This week we found the root cause of why operation X sometimes fails - it's a slow database query which we plan to fix next week. For those interested, here's a command-line demo of the issue."

"With 60 pull requests being submitted per week, and each one triggering a 12 minute automated test run, we were wasting a quite a bit of developer resources. This week we brought that time down to 4 minutes. For those interested, here's how we parallelised the tests."

"Remember how the last accounting feature broke a lot of different parts of the system and it took 4 extra weeks to fix? This week we migrated 4 out of 18 core accounting functions to a service completely separate from our main code base. Once the remaining 14 are moved over during the next three weeks, accounting features can be built and tested independently without affecting the main system."

I almost always tell management that by default I'm spending, say, 30% of my time on technical quality and ask if they'd like to dial it up or down temporarily because of a holiday or a deadline they can. I will track this and show it to whomever asks (e.g. their boss and boss's boss might be interested if they've explicitly asked for 8 straight weeks of 0% work on quality).

The economics and tactics around technology has been revolutionized a dozen times over in the last 4 decades. Now, maybe a few rare individuals have kept up, but most likely, they all rely on outdated strategies from out of touch MBA programs & buzz words.

Its the same reason the market kills public technology companies' innovation and they rely on acquisition.

Its like gunpowder has just been invented and leadership still wants large formations marching against each other. "what if we invested in medical training and add washing our hands so we can keep more people alive?" "nah, more guns and marching"

I think its work experience that matters most with management. You need folks who are people oriented but understand the job that the folks they manage are doing, and the challenges that come with it, both obvious and non obvious, and can communicate the importance of such work to the broader organization

Ah, the cult of youth. It is cute, when it isn't killing startups with rookie moves.

It almost sounds like you think management is something like practicing law - like there's a set of rules that are revised on a schedule and they need CLE credits to keep them current.

Incidentally, marching large formations against each other has been the dominant strategy for most of Gunpowder's history. Only in the last 150 years did guns become lethal enough to make this a bad idea.

The counter-theory is that as companies mature, they have to devote a disproportionate amount of time to maintenance. From that perspective, it's a risk-based approach to maintain the products and services that already have a market and instead effectively out-source the high risk operations of innovation. New companies don't have comparatively much to maintain, so they can focus on innovation. It's not quite such an either-or, but a balancing act. It's just easier to balance when you can outsource risk.

>Average CEO is 58 years old. Their first jobs were in the late 1980s.

I believe there's some neuroscience theory that may illuminate this. When we're young, our intelligence is more plastic and can more readily innovate. But as we get older it becomes more crystalline. What we may lose some of that novelty generation, we can gain in understanding the greater levels of experience. Experience is necessary for putting ideas in their appropriate context. I would argue that understanding context is more important at high, strategic-level positions.

> the Russian nuclear sub B-59, which had been running submerged for days, was cornered by 11 US destroyers and the aircraft carrier USS Randolph. The US ships began dropping depth charges around the sub.

Officers aboard had every reason to believe that their American counterparts were trying to sink them.

Cut off from outside contact, buffeted by depth charges, the most obvious conclusion for the officers of B-59 was that global war had already begun.

The submarine was armed with 10-kiloton nuclear torpedos and its officers had permission from their superiors to launch it without confirmation from Moscow.

Two out of three serior officers onboard agreed to launch, but unanimous vote was required.

I was specifically thinking of the idea that large formations being idiocy after the invention of gunpowder. Brett Devereaux's 4-part series "The Universal Warrior" had an excellent discussion of why a unit of soldiers might fight in large formation. https://acoup.blog/2021/02/12/collections-the-universal-warr... He's verbose, but thoroughly enjoyable.

I can imagine a modern tech leader living in those times and giving the orders to charge the German panzer formation, sabers overhead.

It's like imaginary tech debt.

But modern organizations are quick to destroy trust on any whim.

It is up to the team to decide on the best way to approach this, and what works best for that team, and the team is free to do the work as they see it, with the only requirement that if something does negatively affect the Milestone(s), it gets raised quickly and early, in case of re-adjustment.

This however, means:

- Product management has to be competent enough to present a relatively elastic vision that is not so concrete its essentially a waterfall, but not so vague that its unclear whats being built. Wireframes are usually a good signal here.

- Engineering Management has to be competent enough to communicate (or allow others to communicate) technical challenges that may be involved, and more importantly, what may be unknown, to Product management

- Everyone has to agree that demoing work is more important than talking about work, whenever possible

- Trust in everyone doing the right thing needs to be high, constant interference and meetings will kill this from working right.

I think I'm luckily grandfathered in due to working fully remotely for years before the pandemic, but if that happened to me I would absolutely start looking for a new job.

I've asked before if I can work remotely for a few weeks at a time and have been told yes, as long as I don't leave the state. In some places, just moving to a different county or city can trigger different tax requirements.

> State corporate or other business activity taxes can apply, if even a single employee is working in a state. In effect, if an employer did not previously have a recognized office in a state, but one employee starts working from there, this can trigger entirely new registration requirements and tax liabilities. It may be necessary to register with the secretary of state and relevant tax authorities, provide a registered agent address, and pay corporate and business activity taxes, sales taxes and employment taxes, including employee withholding. There are often state and local licenses and business permits as well.

https://www.adp.com/spark/articles/2022/06/implications-of-w...

So sure, if you’re a company doing only local sales. Or a company doing less than 3-400k of revenue a year, this is probably complicated. Once you hit ~1m in revenue, there is probably AT LEAST one other state you are paying taxes to. Adding states should already be a defined process (or being defined) and an employee moving there is only a one or two lines different.

1: https://salestax247.com/sales-tax-thresholds-by-state

And all states have long required employers to register with the state government for tax purposes if they employ someone working in the state, as well as comply with the state’s labor laws.

They back peddled pretty quickly and switched it to anyone within 10 miles of the office has to come back. There is only 1 person that close. He's the "office mom" and he's been in basically every day since the pandemic started.

Even if half the teams hadn't moved further away, there is no way most of us would have gone back to commuting 2-4 hours per day.

We used goals and velocity metrics on the highest performing team I worked on. This was also a high trust team that happily raised concerns and adjusted priorities/velocity/etc.

The goals and velocity were still extremely useful for getting everybody on the same page for what we were looking to accomplish and how long it would take. We needed to land in the general area, but never got caught up in meaningless drivel over metrics.

The problem is management wants consistency and expects an explanation when things change. I've found it a team is perceived as failing if they're actually realistic with goals and capabilities.

So be careful what you measure and you reward.

For every company cash flow and profits are undoubtedly the most important metric. It is almost impossible to argue that maximize those numbers should NOT be a goal.

At the same time when that becomes the ONLY target that matters, the consequences are dreadful.

At least in US, that is how we ended up with appliances that only last a small fraction of time that used to last 40 years ago.

And even worse it is how we ended up with the food industry creating more and more addicting food resulting in 70% of the population be obese. And it is how we ended up with a heath system that costs multiples of what costs in any other country in the world, that, instead of healing people for good, make them "less sick" addicting them to a few pills for the rest of their life. Because there is no money to be made with a healthy person.

When you build KPIs, make sure you "think a few moves ahead" and you put other correcting metrics and checks in place. At least make sure who establishes the metrics has a way to become aware of the possible shortcomings and plan corrections in a timely manner.

Except for VC backed startups. Actually, there are a lot of exceptions. But those mostly revolve around caveats surrounding riskiness and timelines.

It seems like such a natural division of labor that it appears everywhere. But I also feel like I've never seen a company explicitly optimize processes around it.

So I'm curious to hear any battle stories.

In a codified "trailblazers and firefighters" model, the trailblazers are much more obviously tied to the company's bottom line. Feature X is required to close deal with company Y, this software team built this feature. Meanwhile, the people frantically cleaning up the mess the first team left is only nebulously connected to the company raking in more cash, relegating them to a second-tier position despite having harder work.

I think it ultimately comes down to human values. What's important to the founders and the team? Do they have a clear articulation of those values? KPIs are useful if they're grounded in values that the organization absolutely won't compromise on; KPIs should be a means to an end, not the end themselves.

Whether those values are "we want to make the most money" or "we want to make customers happy" or "we want a sustainable lifestyle for our team," KPIs will only help you pursue them, not define or prioritize them.

Also, if your team is optimising for the status report, your manager has already failed you.

With time and experience, you'll learn that those who can quickly churn out features and bug fixes are deemed extremely valuable to the company.

Let's say we have a wicked bug that has very little chance of happening. But if it does, it'll bankrupt the company, no questions asked. Spending 3 weeks fixing it is nowhere as impressive to the company as Joe who's churned out 10 features per week while your status updates are "Hunting for the wicked bug", even if you describe it it more details.

Yes, because this one task appears as a single line item for the manager's manager. They don't care for the complexity or consequences until the fire actually happens. And if the fire happens, they just blame engineers.

If the industry incentives were changed such that manager heads would roll for mass outages, managers would start appreciating big fixes.

If you're providing a status report, you're optimizing for the status report. Period.

Engineers are not providing status reports. The manager is collating them. How is it any different to a sprint summary?

That's why these kind of incentive structures are dangerous and why things like OKRs put such heavy emphasis on regular, company wide, failure. (I.e. they punish 100success rates)

Team A: Added a wizzbazz button that says "Wizz!" when you push it with a cool noise! Look at this cool screenshot!

Team B: Worked on fixing an elusive bug causing rare data corruption, but couldn't figure out what was causing it.

Team A: Re-colored the header to the CEO's favorite color! Look at this cool screenshot!

Team B: Fixed bug causing rare data corruption. Started looking into strange performance bottlenecks in UI.

Team A: Added spiffy animations that play every time the user goes to the next page. Watch this cool video!

Team B: Improved page responsiveness issue introduced by the wizzbazz button. Look at this graph; pay attention to the ninetieth-percentile time-to-first-render (dotted blue line) for dashboard users. It does actually go down quite a bit if you look.

Team A: Added a thousand lines of code to the fluxborg module! Look at this graph! It went from FIFTY lines of code to a THOUSAND lines of code! Look at how much that is on this graph!

Team B: Removed two thousand unnecessary lines of code from the wizzbazz module. We decided not to show a graph because the line goes down and we know you'll all think that's a bad thing.

Which team is getting the axe when it comes time to lay off employees? You know. Come on. You know it's Team B. You know it's true.

And unfortunately for engineers, they have to dance to this director's negligence even if it comes at the cost of their own sanity.

The real gold is in improving what in bad need of improvement.

The companies that stay afloat are often those that are abundant in people that know what really matter and find a way to do it regardless what the middle management thinks.

Which is hard to come by, because those people aren't rewarded.

For each delivered feature, there could be an overview, e.g.

"Users can now print reports easier. On our user request tracking platform, this issue had 321 upvotes, and 5 Premium clients requested this as well. We placed the print button THERE because amongst the three designs, this performed 23% better according to METRIC. For more info on the experiments, see link or ask on #mobile-team. The feature was released for two weeks behind a feature flag, it performed great, and we made it available for all since last Monday. 5%, of users who visited the report page used this feature and we generated already 5000 PDF reports. Big client A already complimented the feature, and it unblocked the sales process with Client B."

At the end of each email, there could be an analytics overview of the product with overall useful metrics, significant changes etc.

This format also helps with people being on holidays, sick, etc.

For instance, when executives contested the efficacy of our registration form for X reasons, it became incumbent upon us to ascertain its current standing. This involved computing the ratio of registrations to site visits and conducting an in-depth analysis of the errors that users encountered while interacting with the form. We achieved this by integrating Google Analytics events with error messages, among other techniques. We provided first a picture, then we proceeded to make the changes.

This systematic approach enabled us to gain a comprehensive understanding of the situation, facilitating the implementation of purposeful changes. Subsequently, we gauged the impact of these alterations using key performance indicators (KPIs) such as the registration-to-visits ratio, and the previous events tracking changes too to understand if we really made a difference.

I'm confused by your comment. How did you decide this was something worthy of fixing if it "doesn't show any good metrics"? If you can't quantify the issue in any manner, how do you determine it's worth doing?

This is especially important when metrics would not be expected to be available -- for example, if you're designing a nuclear reactor, you need to think hard about ways to prevent a meltdown in advance, rather than collecting meltdown statistics and then fixing the piping problems that correlated with the most nuclear meltdowns.

This is also necessary when the true metric that matters is very hard to evaluate counterfactually. For example, perhaps your real task is "maximize profit for the company", but you can't actually evaluate how your actions have influenced that metric, even though you can see the number going up and down.

And necessary as well when a goal is too abstract to directly capture by metrics, resulting in bad surrogate metrics: for example, "improve user experience" is hard to measure directly, so "increase time spent interacting with website" might be measured as a substitute, with predictable outcomes that bad UI design can force users to waste more time on a page trying to find what they came for.

All of these problems are faced by metric designers, who need to pick directly-measurable metric B (UX design metric) in order to maximize metric A (long-term profits) that the shareholders actually care about, but they cannot evaluate the quality of their own metrics by a metric, for the same reason that they were not using metric A directly to begin with.

(See also the McNamara fallacy, which parent comment is a splendid example of: https://en.m.wikipedia.org/wiki/McNamara_fallacy )

Impact on end users? Nothing measurable. Impact on developers? Frustrating and slows them down, but by how much? It's impossible to say. How often does it happen? Well, difficult to count what isn't there. Would fixing the issue lead to a measurable increase in stories completed per week, or lines of code written, or employee retention? Probably not, as those are very noisy measures.

Nonetheless, that is not a fault I would tolerate or ignore.

Because sometimes some things without metrics are incidental to the actual thing you set out to do.

E.g. a large refactor that switches libraries which is necessary for your new service that give 10% lower latency. But that library refactor will need to be done and it will take 2 months.

1. Determine whether there is supporting data for the proposed change. 2. If no data is available, implement tracking to establish a reference point. 3. Measure the reference point and ascertain what could lead to success. 4. Implement the change. 5. Review the new data and compare it against past values.

"What does a user see? Nothing, but the old one had bugs, including at least one CVE. Now we don't."

That seems like a reasonable content for demo and with just the right amount of self-deprecating humor it can be a slightly entertaining 2 minute presentation.

Unfortunately, too few people think that way.

It reminds me of deviantArt tech dev meetings when I worked there (~10 years ago)

Every Monday, each tech/product team did a small demo for the whole tech org (maybe 50 people, growing to nearly 100 by the time I left).

Teams without an interactive demo would just put together a web page with a few pictures and text describing what they accomplished and team lead would present it to the whole org.

Teams competed for dubious accolades like most lines of code deleted, most embarrassing feature, best meme. Prizes were arbitrarily awarded by the VP of technology in the form of credits to buy art from their prints shop.

Similarly to what you describe, this practice relied heavily on feature flags enabling us to release features for testing while they were still in early stages of development. This worked out really well for getting feedback and QA testing early on, while also keeping everyone up to date about everything that was happening outside of our immediate area of focus. It was fun and motivating as well. deviantArt did a lot of things really right though back in the early 2010s. IMO it was a really incredible engineering org and probably the best job I ever had.

Generally that was good enough.

Because naive senior vice president MBAs believe there never should have been bugs in the first place, and every fix is an admission that the bug existed at all.

Because every email is now a weekly contest between teams, and when hard times come the ones who reported on performance improvements and bug fixes will lose their jobs long before the ones who made the header the president's favorite color.

Because every email is your team trying to convince the company of your own worth and it's much harder to show a pretty screenshot of a performance improvement than a new feature (Graphs of before/after performance every week? Every week?)

Because bugs don't always follow your bullshit schedule made by a bullshit manager who is looking for a pat on the back by replacing bullshit A with bullshit B, instead of anyone anywhere in the company ever just trusting anyone.

Exactly. What does "performance improvement" even mean if you aren't measuring performance before and after?

- Improved p50 latency of endpoint /very/important/endpoint by 22% [attached mini-graph showing the change]

- Fixed this bug that had been reported by 37 clients in the last 30 days as seldomly affecting them

etc

https://www.youtube.com/watch?v=a-BOSpxYJ9M

Comms. It always boils down to comms. Specifically, comms that drive understanding.

KPIs = knowledge.

But what you did, increased understanding.

And what I like to say is: "(Knowledge isn't power.) Understanding Is Power."

Knowing the dots doesn't mean you can connect them. And thus, "Understanding Is Power."

Anyway I think we are probably saying the same thing and discussing semantics.(and to be fair it is probably just me:) )

I am not sure this scales very well with company size. ;)

When it comes time for me to move on from the company, one of my antics is going to be to reply-all to each and every one of them with the word "unsubscribe."

[...preamble] it was not even forgery. It was merely the substitution of one piece of nonsense for another. Most of the material that you were dealing with had no connexion with anything in the real world, not even the kind of connexion that is contained in a direct lie. Statistics were just as much a fantasy in their original version as in their rectified version. A great deal of the time you were expected to make them up out of your head."

- George Orwell, 1984.

All one knew was that every quarter astronomical numbers of features and improvements were produced on paper, while perhaps half the users of SaaS products complained or left.

Edit: I see the team is made up of 50 people across multiple teams. That makes a bit more sense.

Question: what the format you have used (number of words)? Kind of a release notes? Was anyone actually reading those?

"Agents can now see their up-to-date balance on the app (previously they had to wait for the weekly email). Here's a screenshot. Here's a direct link to the QA environment so you can try it out yourself."

That's usually it for a given feature.

These types of updates only work in an environment where people actively read them.

Our CEO had the philosophy that if you achieved 100% of your goals every quarter, you weren't being ambitious enough. As a result, we would stop logging progress at ~80% of our goals, which was the threshold to qualify for your bonus

However, scope on projects would change during a quarter, and sometimes the VP would become distracted by another shiny new thing and make us start on his new pet project. As a result, some OKRs were abandoned and would remain at 0% by the end of the quarter.

Now, no manager (especially an engineering manager) wants to be a bad guy with his employees, so my boss often just changed our OKRs (or, if we only achieved 50% of our KPI, would lower the KPI's target) on the last day of the quarter so that we would all qualify for a bonus.

The company was losing 18mil a year and was sold to our competitor at a 100mil loss for the parent company

This is the problem: tying OKRs to bonuses. It sounds so logical from a naive standpoint and yet it has so many detrimental side-effects.

Where I'm at, we have a hybrid system that works quite well. OKRs are divorced from bonuses; and they are always "Stretch OKRs", ie. ambitious ones. Essentially they are the compass where team is headed. Then there are MBOs which are tied to bonuses. These are very conservative, so that the bonuses are attainable.

With this system, in practice, the MBOs are the subset of the Key Results that actually seem achievable within the quarter.

Of course there's a functional way to play this game and a dysfunctional one.

Working, functional version: spend time defining actual Objectives, regardless of whether we know how to measure them. That last point is very important. Only when the Objectives are defined (and they are almost always qualitative in nature), THEN come up with Key Results, to try to quantify the objective (and be open to adusting that). Finally, pick out the KRs that can be realistic, and make them the MBOs.

Dysfuntional version: start at the opposite side. Define MBOs. Then Key Results. And finally, try to BS some form of Objectives to make it all sound coherent. This biases towards both what is easy to measure and easy to quantify, to the detriment of business needs.

Goodhart's Law: "Any observed statistical regularity will tend to collapse once pressure is placed upon it for control purposes."

[0] https://en.wikipedia.org/wiki/Goodhart%27s_law

As an oversight measure, they fail for the same reasons as every other oversight measure.

Hire people you can trust, give them skin in the game, and then trust them. If you exclusively interact with me via the carrot and stick, I will recognize the lack of trust and respond with an equal lack of trust.

There is too much of this on our industry, and inevitably the churn in our industry matches the service industry.

And then management is surprised when engineers burn out and stop caring. I mean, do they not understand reciprocation? Did they really think treating people like horses will keep people honest and invested in the success of the company and the team?

Now of course, whether OKR works depends on culture as well. Sometimes, asking people to stretch amounts to asking them to work harder, so naturally the system will be gamed in that context.

You can set up OKRs like improving customer experience and, at the end of the day, someone will declare that they have been accomplished or not based on their preference of employees or mood. That is functionally the same as saying this bonus is optional and we might give it to you if we feel like it.

It is a pity because well chosen KPIs that employees try to optimize for can make a company work great. But they need to be _very_ well chosen. This, like economics indicators, often requires creativity and technical knowledge.

I have seen this over and over again - a company made up of good engineers without focus doesn't do very well, unless its has accidentally fallen in a market where they have few to no competitors and this circumstance encourages even more lack of focus, because who are you even competing against?.

Even companies that have bad practices and processes, their eng isn't that great, and often putrid management behavior, the focus on a specific goal that customers want made MONEY and sold companies.

In my experience almost all of that came down to the CEO wanting to hold the line and do the thing that we needed.

The CEO directly created the perverse incentive: actually hitting your goals only meant more work.

> How you apply it matters.

The CEO applied KPIs in a way that disincentived hitting your targets, and the team responded accordingly.

You say this as if employees are absolved of any responsibility to the performance of the company. You're not clever for saying, "Oh, 100% means we weren't ambitious? Then we'll work 80% as hard." In fact, I think you'd have to be pretty dense to not understand the intention of the KPI: set lofty goals and try to get there.

If you don't think it's working, why not go back with ideas to improve it, rather than doing less work? I mean, this company literally failed (from the sounds of it) and people here are blaming the CEOs KPIs versus the employees who intentionally work less to game the system. Strange.

If you create a system which is abused by 100% of employees, that's on you. Everyone who works at a company does so for their own personal benefit. You can moralize about it if you want, but I doubt you would work for a company if it didn't provide an incentive structure of some kind.

> why not go back with ideas to improve it

I was a junior developer working for a 1500 person company. What should I do, demand an audience with the CEO, who lived in a different country?

Most people willingly turn their brain off when it comes to evaluating their daily/quarterly performance against that simple truth.

I do care about my professional reputation. I'm highly skilled and understand how to improve social dynamics on a team, making me very valuable to have around. Clever leadership will figure out a way to align my skills with their own goals. But, if they don't, that's their problem.

That's the bottom line, isn't it? If the CEO wants to align employees' behavior and motivation with his own, he needs to structure the employees' compensation to be like his own compensation. If my compensation is just a "competitive" salary and a few token stonks, then I'm going to do a 9-5 job and hit the required 80% of my KPIs before I go home for the day. If my compensation consists of life-changing equity? I'll work days, nights, weekends, and holidays.

As I said, I care about my professional reputation. When/if the company dies I want every single one of my former coworkers fighting to get me hired at wherever they land. You don't get former coworkers clamoring to get you in at their new shop by being a bad worker.

Companies want me to work for them, and former coworkers want me to work with them. I am just not emotionally invested in the companies who pay me a wage, I'm going to set reasonable boundaries on my time, and I'm generally going to respond rationally to incentives. Responding rationally to incentives includes considering if the extra work required to achieve a particular bonus is worth the effort, often it is not.

You think most 200+ Employee companies are unicorns?

So, hypothetically if I busted my ass for three months straight to hit 100%, just to be told "Oh, your goals were set too low, so here's more work", what incentive at that point do I have to work even harder? The new target's 80% was my last target's 100%, which I struggled to hit (because as you said, these goals are lofty). So if I bust my ass even more, do even more overtime, what do I get then? "Your goals were set too low, so here's more work." Do I just continue this until I burn out? Or do I hit 80 of my target so my workload doesn't spiral out of control, and I get exactly the same bonus?

> I mean, this company literally failed (from the sounds of it) and people here are blaming the CEOs KPIs versus the employees who intentionally work less to game the system. Strange.

Employees gaming their KPIs is a reaction to and a direct result of the CEO's "if you hit 100%, you didn't work hard enough" policy. CEOs using constantly moving targets and increasingly unrealistic expectations to extract every last drop of productivity out of their employees is met with an equally maximizing reaction: employees find a maximum amount of time and energy they can spend such that the status quo is preserved and the bar doesn't move as high next time.

I would bet KPIs were not the main reason that company went under. Obviously we don't have specifics, and companies rarely fail for one reason. I didn't blame the CEO for the company going under either - I simply pointed out the perverse incentive.

Doesn't sound like "less work", it sounds like they did the amount of work management wanted them to do. Just like working 40 hours a week is "less work" than 60 hours a week, but I'm not going to work 60 hour weeks just because of some sentimentality around the company being successful.

It boils down to that ridiculous "stretch goals shouldn't be achievable because you should be shooting for the unattainable". If you want to have that mindset, don't beat the exployees for achieving everything. Beat yourself for not curating their goals.

Tracking and measuring KPI is good, without it the company won't know it's current performance and how to fix / improve it (though beware on burnout). However when it's become target with financial incentive, it ceases to be good measure, because everyone will try to optimize things to reach the target.

Velocity (work complete / planned * 100%) should always be a measurement, because it can help to identify errors in workflow / team member, and prevent demotivation when they need to perform task outside their goal.

the description give didn't make me think it's the kind of workplace where the CEO is going around asking people, "what can I fundamentally change about our goal setting process?"

in any case, goals should be achievable, so if you punish someone for achieving them, you messed up

If you are hitting your goals every quarter, that is a great thing - but I tend to agree, at the other end of the spectrum you have people setting easily achieved goals to check a box.

The real issue is weeding out people just there to check a box when you're small, and when you're large, it's designing a system that internalizes a lot of your employees are there to check a box.

Well the paycheck ends when the company folds. "Doing good work" is in the best interest of all parties.

You know what happened?

1. I got even more work. So much more that I no longer had the freedom to explore and think.

2. The carrot on the stick would be re-attached to a longer stick. It was never enough. I was performing "exceptionally" but my yearly review would never move me closer to my goal.

3. I would end up getting tied down in a bunch of stuff from (1) that ended up leading to burnout because I was no longer working in a place where I felt useful and needed.

4. I still got laid off.

Now, I just go to work and come home. I am pissed I even have to do pagerduty because 9/10 times the problem is someone wasn't given enough time, creating a bug, which was then subsequently never addressed because aGiLe methodology says you only move forward.

"High power" CEOs and PMP-certified project morons are the reason why people's care for your product ends with their paycheck. No amount of "demo days", "email updates", or metrics will fix it. I, and everyone who is like me, will game your metrics until they stop being useful. It's not malicious. It's an optimization. If you want me to do exactly what your metrics ask I will. Nothing more, nothing less. You pay me for 80 hours a check, you get 80 hours. Nothing more, nothing less. If you don't want me to think I won't. Afterall, I'd rather save my brain cycles for things I enjoy. You're paying me to squander my talent. That's YOUR problem. Not mine.

So, you get another job. With great odds of increasing your salary more than you could at your current position. There's really few downsides for workers in having this attitude.

Option A is being an average worker, not trying especially hard, not thinking about work outside of work hours, and having absolutely no emotional investment whatsoever in the wellbeing of the company - and potentially having to get a new job in a few years time, often with a pay increase

Option B is "doing good work", "going the extra mile", "being a rock star", putting in lots extra time and effort and emotional energy (for years!) - just on the tiny chance that your particular efforts will be the difference between the company folding vs. being successful

Option A sounds a hell of a lot better to me than option B.

Additionally, at such a large company (1500 people), my individual efforts have little to no effect on whether or not a multinational corporation goes under.

More that you have to have realistic expectations when you structure an org, that people doing the work are going to pad and sandbag in successive tiers up, and people in charge of outcomes are going to push for bigger and more down.

Of course, we first would need to agree that this is desirable and that's probably the point where it breaks down.

And it's not just individual people modifying their behavior to game things, but more of an evolutionary process that the people who tend towards gaming things will get better bonuses, better reviews, better CVs, hence better next jobs, in perpetuity.

"Don't game it" can't be a simple solution. If there are people who game things, and they accumulate rewards for it, you will just see them out compete the nongamers. You are basically telling people to sacrifice their and their family's financial position for the sake of the corporation.

Everything can and will be gamed. But at the same time if many game it, managers will really want to see evidence of nongaming, so employees are also incentivized to signal their honesty, but this must be given some kind of channel too. If managers myopically focus on gameable metrics, then their loss. You must alwas leave a door open for receiving honesty signals through some kinds of side channels. These must always be somewhat amorphous because the moment you define them, they become gameable.

In other words, no recipe, stay alert, try to cooperate but don't be a pushover, etc.

What a cop-out. Everyone should want the company to succeed. So why not say "Hey, boss, this KPI won't work, here's what might happen..." instead of literally working less than needed to prove some point? It's dishonest.

The company has no interest in my success, why should I care if the company succeeds? It has no effect on me if it does.

"That's my only real motivation is not to be hassled, that and the fear of losing my job. But you know, Bob, that will only make someone work just hard enough not to get fired"

- Office Space

Put yourself in the same boat: if you had a team to manage, with the freedom to give them incentives, and they didn't have the effect you expected, you should be able to notice it and try something else.

That's great if the team is helpful and tells you exactly what's wrong, but you should also be able to evaluate how it's going and react accordingly, otherwise you're just asleep at the wheel.

The real issue is - KPIs are used to incentivize or PIP employees. Why are you setting the employees in a game that is in conflict with your company's overarching goals?

The converse is that there are also always novel ways to differentiate yourself bottom-up from the mere shortcutters. This is basically how fads form. Something novel appears that is first a great signal for honest quality work, then it gets formalized and imitated and it becomes less predictive, so those who are in the business of creating true value must come up with new ways to demonstrate this.

But still sabotage seems to be frequent - as part of "empire building", as part of gaming the compensation metrics. And "boys will be boys" is not the a response that steers toward the desired behavior.

One also has to consider whether the people who define the KPIs are themselves properly incentivized. How are they evaluated? Could there be some perverse incentive to set gameable KPIs that get hit easier? Maybe the KPI setter is in some sense in on the game and complicit? Very complicated stuff. What if the business gets destroyed in the process? Who does that hurt in reality and how much?

I'm not sure where you see the correct part.

Upton Sinclair — 'It is difficult to get a man to understand something, when his salary depends on his not understanding it.'

Management is a service task, it does not produce value itself. Good management is important to prioritize, allocate resources and resolve conflicts. But that is not how MBAs seem themselves, they see themselves as the most important thing around, they're the ones in charge after all. And damn physics and reality, the spreadsheets say you can get get a baby out in a month if you use 9 women to do it. Soon management captures all the attention and sucks away all the money, and shortly thereafter end up with a moribund shell of the former company. You can tell how close to moribund a company is by looking at the manager to worker ratio. Get out of there once it climbs much above 1 to 10.

There are good managers around, but they're almost as rare as unicorns.

And re ex-engineer H1Bs: just because you used to be an individual contributor before becoming management does not imply you retain your individual contributor skills nor that you'll be a good manager.

Most people across my career that became managers did so because the pay was better, workload was lower and promotion ladder more straightforward. Worst managers were usually the type that expected and demanded to be management after X years as an individual contributor; this is particularly true of people from some countries, where it is culturally expected that you'll be a manager within at most 10 years of starting.

Most companies do treat moving to the management ladder as a promotion, which create all sorts of perverse incentives for many to become managers. Reminds me of a blog from a few years back:

https://charity.wtf/2020/09/06/if-management-isnt-a-promotio...

So instead, bring back the secretaries. A separate org vertical (i.e., they don't have hiring/firing power over ICs), which handles the secretarial half of your current managers' jobs, and isn't paid more than your engineers. The engineering decision-making half of the job (along with hire/fire power) should be handed over to one or two senior engineers, who are now the actually powerful ones in the engineering branch of your org chart. This keeps your technically talented people working in technical roles. Basically, instead of

       C-suite             C-Suite 
       /      \              /  \
      M        M            EM   SM
     / \      / \          / \  / \
    M   M    M   M        E   ES   S
   / \ / \  / \ / \
  E  EE   EE   EE  E

Further, I also think that EMs need to then become more technical aka a real "leader" of the services they are responsible for. So hiring, firing, perf, BS meetings becomes only 5% of their job. 95% of their job is to ensure that services keep running and projects get delivered - like a true leader. ES can help them and ICs with administrative tasks like HR follow ups, OKR tracking, meeting scribing etc.

I can't believe how much work is being done by low level EMs and ICs in tech companies. It is almost exploitative in one way.

It’s pretty common ATM for product managers to not have direct programming or software backgrounds and instead have a career like “MBB consultant -> MBA -> product manager”. Big tech companies also insource a lot of “strategy consulting” which are MBA types. And marketing/biz dev have plenty of MBAs too.

While someone like a “director of engineering” probably doesn’t have an MBA, there’s a good chance their boss does (especially since these days VPs seem to mostly be former product managers rather than former software engineers).

By the way, Tim Cook, Satya Nadella, Sundar Pichai, Andrew Jassy, and Cheryl Sandberg all have American MBAs.

Scoping was whatever manager decided the right scope and timeline was.

It's almost embarrassing to see it in action. I am embarrassed that I am being asked to toil and grind for such stupidity.