A GE F-Class unit using Mark V controls suffered a major failure. The root cause was not identified, however, it was related to an apparent overloading of instructions to the Mark V system. The system behaved abnormally, leading to the failure.
A recent online article suggested possible actions to protect against the failure, but did not suggest steps to prevent the failure from occurring in the first place.
Subsequently, GE Energy published a technical bulletin that referred to the article. The bulletin described the events surrounding the failure in detail and provided specific communication management criteria, which, if adhered to, would likely preclude a failure of this type. It also recommended against implementing the protective measures suggested in the original article.
For many users, modifying their existing <HMI>, DCS and other interconnected systems to comply with these communication limitation requirements may represent a significant engineering exercise and may alter or limit the functionality of their overall DCS, SCADA and PLC systems. In addition, it may still be difficult to guarantee that a communication overload is not possible under every set of circumstances for each possible system configuration.
One solution to this problem is the Turbine Monitoring System (TMOS), available through Turbine Technology Services (TTS). The TMOS is a direct replacement for existing Mark V <I> and <HMI> systems and one of its main features is that it actively manages and regulates the transfer of instructions from all site devices (BOP, DCS, SCADA, Operator Stations) to the Mark V ensuring that the type of communication overload associated with the recent failure is not possible.
TMOS has been in the field since 2001 and has maintained an outstanding record for safety and reliability for more than 60 million operating hours. TMOS is typically a simple and quick “plug and play” replacement for single Mark V systems and can be used to dramatically simplify multi-unit systems. TMOS is an established product and there are already hundreds of machines around the globe that have been upgraded to this system.
We asked the TMOS’ developer Thomas Finstermann, CEO, of Industrial Turbine Services to comment on how TMOS addresses the recommendations in GE’s Bulletin Advisory. He highlighted the following points:
- TMOS has a built-in traffic control algorithm that buffers the command traffic from all connected clients (operator stations, DCS systems, SCADA systems, etc.) and sends them to each MKV with a delay which ensures compliance with GE’s Advisory requirement. Operator Display command push buttons are given priority over Modbus, OPC, or other third party clients.
- The traffic control algorithm consolidates identical commands arriving simultaneously at TMOS from multiple devices into a single command which is then forwarded to the Mark V.
- The TMOS communication architecture ensures that the number of TMOS (HMI) Servers on any Mark V ARCNET system does not exceed two (2) while providing full server and HMI system redundancy in a “hot backup” configuration.
- TMOS Servers act as a historian and online site monitoring system. There are no additional read requests issued to the Mark V panel.
- The TMOS traffic control will only allow a predefined number of sessions and it is safe to use them during unit operation.
- The traffic control algorithm will not allow the same signal to be requested from the Mark V twice. If a signal is required for a fast trending operation, but the same signal is already used in a slow updating display, the signal will be moved from the slower to the faster task.
- There is no screen cache limitation in TMOS. If two or more operator displays or other devices are requesting the same signal at the same time, the Mark V will receive only one request.
Neither the original article nor the GE bulletin discusses or identifies a root cause mechanism for the failure and while it is important to ensure that each Mark V system complies with the GE advisory recommendations, it remains possible that other factors could have contributed to this specific failure.
TMOS provides the most established, rigorous and predictable solution to the issue of managing traffic to a Mark V system and ensuring system compliance with the fundamental GE recommendations in the bulletin. TMOS will also simplify the overall Mark V system communication architecture while allowing users to maintain all their existing SCADA, DCS, PLC and other communication interfaces intact and without modification.