BES Operation Horror Story 2 – International Embarrassment!

An executive visit to a South-American country is in schedule, one of the three executives backs off so I was volunteered to go. Our mission is to reach an understanding, to start studying the economics of laying an underwater transmission cable interconnecting both systems.
Everything is going well, presentation after presentation of markets, prices, fuels, etc. On the second day, what I considered the highlight of the trip (for me anyway), a visit to their energy control center, up in the image above.
A mezzanine over the control center gives you a vantage point, an eagle-eye view of everything going on. After their Operations chief recites through what seems the standard spiel, he’s about to end the presentation then I start with specific system operation questions; generation, loads, EMS, frequency bias, ACE, contingencies, SPS’s, UFLS, etc. All my questions are answered very professionally and then, with an very seriously concerned face, “our system is very stable akin to an infinite bus, unlike your island in the Caribbean”.
Before I can recover from what he just said and try to politely answer or not his statement, behind him I happen to notice the ACE going to around +100 MW and the frequency up to about 60.18 Hz. Aha! How often the ACE goes +/- 100? I casually ask and he answers “once a month, maybe”. “Oh, just like right now” I say. That was a too-easy cheap backhand shot I had to take.
As he turns toward the glass pane, the mapboard now has 2 breakers flashing green at one substation and down at the bottom another 2 green flashers…then they all turn solid green so I ask about what just happened.
Takes a look and tells me that two transmission ties with a neighboring country to the south tripped and I know he keeps talking but my attention is already locked in the mapboard and the situation below.
One of the bottom breakers flashes red then goes dark, the same with the other sometime later. (Aha! Lines’ energized)
The big screen to the right of the mapboard has the transmission alarms summary page up, one of the open breakers turns red then green and I see the fail to operate alarm popping up in the display, this happens twice. Then the exact thing happens with the other breaker…My questions keep coming as if nothing happened:
So, are there any other ties to this country? No
Not even at any other voltage? No
“Well the breakers in your country are not closing because both areas are out of sync” I, matter-of-fact say.
The operations chief’s face turns cherry red, grabs a flip-phone out of his pocket and angrily barks a couple of instructions to the operators below…while trying not to raise his voice.
The submarine cable project never happened, the economics did not support the investment.

BES Operation Horror Story – From 60 to 0 Hz in 30 minutes

From 60 to 0 Hz in 30 minutes

I was in my office in the operations department training three new recruits to become transmission system operators, one of their tasks involved data input in another boring MS Access form. It’s about 8:15 in the morning and the lights blink when instinctively my heart skips a beat but the lights remained on so I continued about the training. Another coworker walks in and says they had a 230 KV line trip and reclose; a once every other month event – nothing to worry about too much.
Work was getting a little bit routine in the transmission clearance office but to keep things interesting I had an analog frequency meter over my monitor just in case and earlier that morning I was showing the trainees how to change the alarm settings on the meter by adjusting two movable indicators on the dial, crude but did the job. I remember those alarms got to be distracting at first so I set them up at 57.5 Hz(the lowest step of UFLS where it would never go!) and 60.2 Hz. About an hour later I jump from my chair as the alarm goes off and a quick gander to the now left leaning needle 57 Hz!
Nah, probably a malfunction but I walk as fast as I could to the Energy Control Center and the card reader doesn’t work, through the glass wall and doors the operators have a deer-in-the-headlights look on their faces and after what seemed an eternity the door finally opens

54.2 Hz

the frequency meter read. The operations division head tells me in a broken voice to open the substation 1346 breaker (the local 115/13 KV substation that also serves the ECC, about 30 MW’s of load- we still had a wired control of it from the mapboard). As I walk towards the mapboard and right before I select the breaker to open I look back again at the frequency meter:

40.2 Hz

The breaker opens and the faint drone of the emergency generator running is the only thing heard amidst an eerie silence as I glance back

0.0 Hz

Someone received a call from the alternate control center in the south of the island, they were still on! “go to the helipad they’re going to pick up and fly you to Ponce” with a broken-trembling voice the division head said, it was the first time I had to fly in a helicopter with no doors. In no time I was operating a system controlling the frequency by instructing hydro generators on one phone while giving instructions to larger thermal units on the other.

From 3,300 to 800 MW. What happened?

When the 230 KV line was first tripped by a tree in the right of way, the automatic reclosing of it was blocked by the operator under instructions to avoid “the switching disturbance” in an election year. The line tripped 20 minutes later and it remained out. Meanwhile a high percentage of the system generation remained tied to the system through a single 230 KV line, this line went immediately to 98% of its rated capacity. The operator redispatched the generation to minimize the possible effect of the contingency but this line tripped about 25 minutes later (it was later determined the line sagged to the point of breaking clearance against a distribution line).
When the second contingency happened, the system split into two separate systems, the generation-poor side to the northeast blacked out but the generation-rich one to the southwest remained online and stable.
One of the operators in training never came back.