Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

We really need a strategic transformer reserve. Maybe this could be done as a global international effort since so much commerce is on the line.


Or use a cheap fuse to protect the expensive transformers: https://en.wikipedia.org/wiki/Fuse_cutout


The transformer lead time issue is to some extent FUD. The lead times are large because there is enough financial incentive in having transformers last for 40 years that they on the extreme end of the spectrum of possible engineering and manufacturing (of transformers).

If there were a national/international massacre of transformers (also, mostly FUD*) there would be an almost overnight response of making dead-simple, dead-easily made transformers and grids would begin being resupplied within weeks. Remember, we could mass manufacture transformers over a 100 years ago.

Of course such a long outage may well be a civilizational disaster, but it wouldn't be years of lead time for replacement.

* Transmission providers have begun monitoring transformers for DC flows on the neutral conductor, which is what damages them in a GMD condition. Once detected, they can drop affected transformers (causing an outage), and even dispatch line workers to physically disconnect if need be.


This 'simply use century-old manufacturing tech and make millions of units fast' skips multiple problems. Any large manufacturing effort will require moving the inluts to the mfg locations - difficult with extremely limited ability to move fuel. Also needs power transported, oh right, the power transport system is down which is why we need all the transformers yesterday). And, what assurance is there that these century-old tech transformers will even fit & work in the modern system?

I'm all for MacGyver-ing solutions quickly, but thus seems like an incredibly bad plan


Saying that we could make transformers 100 years ago is not the same as saying we would use 100 year old transformer tech. It simply points out that there is a scale of technology, and we can use whatever point is appropriate. Manufacturers know quite well which requirements are longevity- and cost- related, and which are must-have functionality.

As to the other objections, do we really think that manufacturing and transport would not be given top priority? The US military is the largest, most capable logistics operation on the planet. They specialize in black-start operations. They can make sure manufacturers get materials, and products get delivered.

In any case the idea of a transformer massacre by GMD is a fading one because the cost-effective way to address the issue, by monitoring and mitigating the effects, are under way and improving over time.


Yes, the US Military is likely the greatest logistics organization in the history of the planet, and manufacturing would be prioritized.

And sure, THEORETICALLY, we could install mitigating tech to instantly disconnect each transformer the millisecond induced over-currents are detected. But in REALITY power companies are insanely tech-backwards. Even today, in a major developed area like around Boston, the technology for DETECTING outages is [wait for customers to call in] and [drive a crew out to look for a break]. Last month, there was a storm that caused widespread outages in much of Maine and NH. Crews were in for over a week, brought in from over a 1000-mile radius; I personally was up there and saw convoys of crews from Tennessee, Pennsylvania, Canada, etc.. It took over a week to restore power in just that region, with zero shortage of supplies. They rely on mutual-aid cross-covering regions to get things working again. In a global-scale event, EVERYWHERE is down at once, and the crews are each on their own.

I had to call in repeatedly about a serious undervolt situation (99V-103V) that persisted for hours and was damaging equipment, and they had no clue about it. They are not even installing technology to detect outages; they sure as hell aren't spending money to mitigate them.

Now, seriously consider a Carrington+ event or EMP attack. The reasonable assumption, even with the mitigation tech you promote, is that almost nothing works. Perhaps a few zones got disconnected in time, but no grid.

This means the entire population will have to survive on the available stock of just-in-time fuel, food, and other supplies.

Most vehicles will not work. Only pre-1980s vehicles and vehicles stored well underground or in basically Faraday cages will work. Charging systems won't work. Fuel pumping stations won't work. So, we've got manual pumping for single-digit percentages of working vehicles, which need to deliver remaining food stocks, and all the equipment for your manufacturing effort.

Most telecomms do not work.

What's in most people's refrigerators will last a few days. Skip a few meals and civil disorder starts. There are at most a few weeks before serious starvation and civil unrest starts.

Now, under these conditions, you propose to gather enough people to:

1) design these new models of transformers that are easy-to-construct-but-still-work-in-modern-systems,

2) determine the logistics of finding factories that can work, getting those factories working, and sourcing material,

3) moving that material to those factories

4) obtaining, moving, watering, and feeding the people to man the factories,

5) actually building the transformers,

6) distributing the transformers to where they are needed.

7) FINALLY, we can get to work rebuilding and cold-starting the grid

There is no way even in a Polyanna++ optimistic scenario that this will work within three-six weeks.

By the time even some of the first 3-5 steps are completed, much of the population will have passed the starvation window of a few weeks, and that is assuming no ad-hoc local wars, and that it wasn't an EMP attack as the start of a war.

In contrast, spending only about $500 million on a reserve inventory of transformers, pre-cashed in appropriate locations, we can jump directly to step 6 or seven on day-2.

Even under the pre-stocked transformers scenario, it's unlikely that everything could be restarted in time to prevent serious follow-on societal-scale damage.

Plus, since each transformer would need to be disconnected (it's the length of attached lines that generate the damaging current), will the cost of building and installing mitigation devices on every transformer, and a system to operate them, actually be cheaper than building spares? It actually seems like and warehousing building spares is likely a cheaper option.


Now you are conflating a GMD with an EMP, which isn't anything like accurate.

A GMD is a long-duration event that creates movement in the Earth's magnetic field, which induces low-frequency ground currents (literally: current flows in the earth) in the surface of Earth.

An EMP is a very short duration, high-frequency pulse of energy in a local or regional electric field. The effects are completely different, the objects that are affected are completely different, they are completely different cases to consider.

Your anecdote also conflates wind storm damage, which affects local distribution systems, with (hypothetical) GMD damage, which affects long-distance transmission systems. With wind lines, poles, and the occasional transformer, is damaged and there is a lot of labor-intensive repair. With GMD, a very small number of high-capacity transformers are potentially affected. Different deal completely.

So talking about cars that don't start, etc is just nonsense when talking about GMDs.

I don't see any way to continue this dialogue given that we aren't even talking about the same thing.


I wasn't conflating the two, I understand that the effects are different, and m recognizing that both are a serious present threats to the same system, but if you don't want to talk about EMP, ok.

Even if we have a mild-Carrington event that directly takes out "only" the tens of thousands of transmission-level transformers, there are still orders of magnitude more distribution-level transformers, and even though many of those are likely to be damaged by the disregulated power system as it goes down, let's assume those are OK.

So, we've "only" got to replace hundreds of high-voltage transformers in every metro area, and then reboot the grid. And there are no spares.

But, although cars and trucks are mostly undamaged, there is no power to pump fuel, and there is no power for refrigeration.

So, we still have to perform all of the 7 steps above (plus a BIG one I forgot 5A: Test all the new transformers).

And we still have to do it under conditions where the entire economy is based on just-in-time delivery and stocks of food are rapidly being eaten down, and rotting in unpowered refrigerators, and the entire fuel distribution network is unpowered.

What is your actual plan to design, build, test, and install these thousands of high-voltage transmission-level transformers, and do it before the society rips itself apart from lack of basic supplies?

Saying "oh we could just use old lower-tech solutions adapted for a high-tech grid. is extremely hand-wavy.

And, how is it actually even more cost-effective to design, build, install, and maintain equipment to automatically protect the transformers, when the power grid operators are so backwards half a century after automated monitoring was available but they still refuse to install it?

I'd seriously like to see a good answer, because nothing I've seen from any official or your proposals even remotely seems to solve the problem. What real technology would actually apply? What military op would scale to the entire nation's needs, that wouldn't already be allocated elsewhere?


What does this have to do with failing gps devices


A sufficiently powerful solar storm could fry most or all of the energized transformers in its path through induction causing overheating.

Without transformers you have no electrical distribution, which would lead to societal collapse very quickly, as there are not enough spare transformers to replace all of the existing ones. We are heavily reliant on refrigerated supply chains for food distribution.

Without electricity, all of the other utilities you rely on would cease. Water, natural gas, internet, sewer, etc all require electricity.


wouldn't disconnecting them during the event prevent that?


To an extent, sure. But then part of the problem is that these are planet-scale events: you’d need to disconnect them all at once (coordinating that between independent operators, and as I understand it isolating these giant grid-scale transformers isn’t just a question of flipping a switch). Then you need to ramp down power production in a balanced way as you ramp down places for it to go, and if you succeed at the whole endeavor then you don’t have a grid anymore. Even if it’s possible to do that before the storm, it’s hard to know whether it’s possible to “cold start” the grid at all, much less timely.


> it’s hard to know whether it’s possible to “cold start” the grid at all, much less timely.

This is called "black start", and it's something all grid operators plan for and regularly test.

How it works is very simple: some power plants are able to start up without the grid (they have their own emergency diesel generators, which in turn have batteries for their own start up), and these power plants can run for a while without any external load. Then it's a careful dance of powering up some high voltage circuits and substations, putting some load on them, powering up more power plants and circuits and substations, putting more load on them, and so on, until all the loads and most of the power plants are connected.

It's not a fast process, but it's also not that slow; from those I've read about or experienced, it tends to take less than a day for most of the grid to be back online (and the places which don't come back online are usually due to some local defect).

> [...] you’d need to disconnect them all at once (coordinating that between independent operators, and as I understand it isolating these giant grid-scale transformers isn’t just a question of flipping a switch). Then you need to ramp down power production in a balanced way as you ramp down places for it to go [...]

It's the opposite. Powering down can be very fast, by design: whenever there's a fault, or parameters like voltage or frequency or power go too far out of nominal, protective devices react by disconnecting whatever they're protecting, so everything on the power grid has to be designed to tolerate being instantly cut off. The worst that happens is that, without anywhere for the power to go, the protective devices on the generators quickly power them down too, and some kinds of generators are very slow to power back up (it can take a whole day for some thermal power plants).


> designed to tolerate being instantly cut off

Tolerate is often key. It can still be better to do gradual shutdown even if you can do the instant one.


They don't know if they could restart the grid? Do they think the turbines will not start, or what?

Why isn't there an emergency plan and scheduled failure testing in place? Netflix randomly power cycles entire datacenters to verify their coordination and fail over plans work. Why does my Parks&Rec stream have better disaster planning than critical infrastructure?


Are forecasting models accurate enough to know when disconnection is necessary? We know how strong the CMEs are but not how they'll recombine on their way to earth.


That would be possible, but then you would need to blackstart the entire grid which would be a long and painful process.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: