From 60 to 0 Hz in 30 minutes
I was in my office in the operations department training three new recruits to become transmission system operators, one of their tasks involved data input in another boring MS Access form. It’s about 8:15 in the morning and the lights blink when instinctively my heart skips a beat but the lights remained on so I continued about the training. Another coworker walks in and says they had a 230 KV line trip and reclose; a once every other month event – nothing to worry about too much.
Work was getting a little bit routine in the transmission clearance office but to keep things interesting I had an analog frequency meter over my monitor just in case and earlier that morning I was showing the trainees how to change the alarm settings on the meter by adjusting two movable indicators on the dial, crude but did the job. I remember those alarms got to be distracting at first so I set them up at 57.5 Hz(the lowest step of UFLS where it would never go!) and 60.2 Hz. About an hour later I jump from my chair as the alarm goes off and a quick gander to the now left leaning needle 57 Hz!
Nah, probably a malfunction but I walk as fast as I could to the Energy Control Center and the card reader doesn’t work, through the glass wall and doors the operators have a deer-in-the-headlights look on their faces and after what seemed an eternity the door finally opens
54.2 Hz
the frequency meter read. The operations division head tells me in a broken voice to open the substation 1346 breaker (the local 115/13 KV substation that also serves the ECC, about 30 MW’s of load- we still had a wired control of it from the mapboard). As I walk towards the mapboard and right before I select the breaker to open I look back again at the frequency meter:
40.2 Hz
The breaker opens and the faint drone of the emergency generator running is the only thing heard amidst an eerie silence as I glance back
0.0 Hz
Someone received a call from the alternate control center in the south of the island, they were still on! “go to the helipad they’re going to pick up and fly you to Ponce” with a broken-trembling voice the division head said, it was the first time I had to fly in a helicopter with no doors. In no time I was operating a system controlling the frequency by instructing hydro generators on one phone while giving instructions to larger thermal units on the other.
From 3,300 to 800 MW. What happened?
When the 230 KV line was first tripped by a tree in the right of way, the automatic reclosing of it was blocked by the operator under instructions to avoid “the switching disturbance” in an election year. The line tripped 20 minutes later and it remained out. Meanwhile a high percentage of the system generation remained tied to the system through a single 230 KV line, this line went immediately to 98% of its rated capacity. The operator redispatched the generation to minimize the possible effect of the contingency but this line tripped about 25 minutes later (it was later determined the line sagged to the point of breaking clearance against a distribution line).
When the second contingency happened, the system split into two separate systems, the generation-poor side to the northeast blacked out but the generation-rich one to the southwest remained online and stable.
One of the operators in training never came back.