From high resolution graphics to more interconnectivity and mobility, the HMI has become instrumental to plant productivity.
Around 40 years ago, operator interfaces on the factory floor consisted of manual push buttons, switches, and dials. More sophisticated versions might incorporate a few indicator lights that could flash or change color to communicate alerts sent by the programmable logic controller (PLC). Today, most of those buttons and switches are gone, replaced by touchscreen human machine interfaces (HMIs).
Modern wide-screen HMIs use high-resolution, 24-bit color displays to deliver detailed information on machine performance, operation, and condition in a highly consumable format. Packed with processing power, memory, and high-speed networking capabilities, a single HMI can now access dozens of PLCs throughout a facility – and make them accessible to a variety of locations both on site and off.
Here, we chronicle the evolution of today’s high-functionality HMIs, reviewing cutting-edge technologies and providing a snapshot of how the industrial internet of things (IIoT), cloud connectivity, and even artificial intelligence (AI) and machine learning will augment HMI usability in the future.
A Brief History of HMIs
Early HMIs consisted primarily of monochrome text displays, sometimes used only to label the manual push buttons that surrounded the screen (see table). At best, they functioned as message centers capable of displaying pre-programmed text. Adding the ability to support color graphics offered real potential, but most implementations simply replaced the minimal lines of text with icons. The screens still primarily provided information about the function of a button or switch at any given time.
When touchscreens emerged in the 90s, technology was just put to use creating a display of virtual push buttons. Then, in 2007, the iPhone changed the world – and the world of HMIs changed with it.
“HMI users had a phone in their hand every day and witnessed its capabilities,” said Greg Philbrook, Automation Direct’s HMI and CLICK PLC product manager. “That began to change expectations for the industrial market. They now wanted their HMIs to have really crisp graphics and increased functionality with easy access to many layers of information.”
The pace of HMI development has accelerated rapidly to meet those expectations, particularly over the last several years. Now, a number of capabilities that are standard in top-of-the-line HMIs have changed the face of industrial operations.
How Core HMI Capabilities Upgrade Industrial Operations
Data Visualization
The most fundamental need of anybody in industrial operations is finding a way to keep the process running. Mark Gentry, founder and CEO of Gentry Controls, polled his customer base to find out what they wanted from an HMI. “The number one answer from 100% of my customers was, ‘When something stops working, tell me what it is and why.’” The PLCs and various sensors may have the data, but the HMI makes it consumable. “If the PLC is not doing something, then you can look at the HMI and find out why.”
Nick Bell, controls engineer at InterConnecting Automation, saw the impact of accessible data in a poultry processing plant. The end users were highly skeptical of automation, having been burned in the past by expensive technology that went down far too frequently with no easy way to troubleshoot. Then Bell’s team fired up the new system.
“They were blown away just by the pure data,” he said. The screens his team had configured made it easy to monitor key performance indicators such as room pressures and temperature flows, with trending and averaging over time.
The implementation included over a thousand distinct alarms, each of which provided detailed information. “Previously, the system would stop working, and they wouldn’t know why,” Bell said. “Now, they can read that this motor on that conveyor belt has tripped with this fault, and then they can go fix it within a couple of minutes.” He credits newer HMIs with more functionality and greater memory being able to both track and serve up the data. This particular system has over 5,000 different memory registers. Another project he’s working on has nearly three times that many.
Other simple but valuable data consists of basic statistics like uptime, downtime, parts per hour, and more, allowing managers to closely monitor overall equipment effectiveness (OEE). Bell points to a recent project at a facility that had never used touchscreen HMIs. The operations manager made daily estimates tracked on paper. “We finally upgraded them to a little display. The manager wasn’t sure if he wanted it on his machine.” And then Bell’s team demonstrated the system. “Now he wants one on every machine just because he can go around, plug in, see what the run time is, what’s the hour meter. Whereas before, he didn’t know.”
More Detailed Data Display
The size and resolution of HMI screens have been rising steadily. The latest generations of HMIs provide plenty of room for high-resolution graphics (see Figure 1). Instead of a tank fill level appearing as a percentage, for example, it can be displayed graphically as part of a larger facility layout. HMI screens can be configured to show the rotation direction of motors, fault status, and more, and the additional real estate and crisp graphics make it possible to present that information without risk of confusion. Buttons or active areas can be spaced far enough apart that even operators wearing safety gloves don’t run the risk of accidentally executing the wrong function.
In addition, larger HMIs equipped with Ethernet ports now make it possible to tie together the entire plant. “We’ve been putting them in as plant-level touchscreens, almost like SCADA systems,” Bell said. “Before, every machine was its own independent little cell. Now it’s not.”
Given the amount of data involved, sometimes even the 22-inch screens aren’t big enough. In these cases, a “headless” HMI can be used to transfer the HMI data to a larger screen (see Figure 2).
Depending on the location, that might involve a 32-inch touchscreen (see Figure 3). For a manager who simply wants to view data at a distance – or from the office – a 75-inch monitor might be an even better choice.
“When we go into an older plant, one of the most common questions is, ‘How can I sit in my office and see everything in one place?’” said Bell. “With the headless unit, you can just run a couple of Ethernet cables and there’s all the data in one spot.” In another useful implementation, HMI screens can be sent to a display in the maintenance shop. When an alarm shows on the HMI on the floor, the same alert appears on the screen in the maintenance shop. The technician has all the information necessary to get started on a fix immediately, rather than waiting for a call on the radio.
Remote Access
Almost every integrator and OEM has a horror story about hours spent traveling to a customer location to perform a fix that took perhaps 15 minutes. “More than 90% of the time, there’s nothing wrong with the control system. It’s the operator not understanding what is being presented,” said Gentry. “If I can just see the screen that they are seeing, then it’s a 30-second conversation to get them going again. So being able to remotely log in and look at the screen has just been a game changer.”
He estimates that 90% of his customers have given him remote access to their equipment for purposes of troubleshooting and repair. This approach can dramatically reduce the duration and costs of unscheduled downtime, not to mention travel time and expenses. OEMs and integrators also have the option of offering troubleshooting and monitoring to end users as a subscription service.
Allowing integrator and OEM connectivity for troubleshooting is one thing, but what about remote access for the broader organization? Industrial users have traditionally been skittish about connecting equipment to the Internet. Indeed, there are still plenty of organizations that have significant concerns about the technology or simply don’t see any value in it.
“Due to limited in-house IT capabilities and cyber security issues, only 10 to 20% of my clients seek cloud capabilities such as the IIoT, cloud-based data storage, and remote viewing,” said Scott Martin of KCC software, whose customer base spans a range of industries. Of the organizations that do take advantage, he notes, the most common requests are for data storage and remote viewing, typically on spreadsheets and PCs. “I do have some clients who prefer to see what is happening when they are distant from a system, but that is not a frequent concern.”
As time has passed and the technology has matured, however, other industrial organizations have become more open to the idea. The pervasive use of connectivity for consumer products has also helped change the dialogue from, “No way am I going to connect anything from my manufacturing floor to the Internet” to “I can change my home thermostat setting from my smartphone app. Why can’t I oversee my production line from my office – or my house?”
Bell also sees a distinct split between those who do not want remote access and those who are highly committed to the idea, but the needle is moving in the direction of acceptance. “You have the few who don’t want it or consider it cost prohibitive. We see the trend going towards more companies wanting remote access, though,” he said. “Of the systems we’re putting in as of this year, probably 85% of them have some sort of remote access capability.”
Don’t make the mistake of assuming that remote viewing uptake is related to market sector. Martin has a number of very high-tech clients, while Gentry, who exclusively serves the cotton gin industry, estimates that as many as 75% of his clients are using remote-access solutions for staff.
“It’s been driven by smartphones and COVID,” said Daniel Caldwell, president of Caldwell Electrical Contractors. “Our trade also has a labor shortage. People are retiring, and companies are feeling that. With remote access, they are able to use one technician to troubleshoot problems or have them assist an inexperienced onsite technician remotely, so I think that’s another motivating factor.”
In the case of very large facilities, industrial end users are increasingly experimenting with mobile HMIs (see Figure 4). Rather than walking a quarter mile to the controls panel, they can glance down at the tablet in their hand. “We’ve seen a lot of requests over the past year for mobile HMIs,” said Bell. “We [recently] installed it for a customer, and he was sitting on his porch with his iPad, watching his system run.”
[Put figure 4 here]
The Cutting-Edge Technologies
The HMI as a Gateway Device
To support the convergence of operational technology (OT) and IIoT, the latest generation of HMIs has begun to transform into edge devices. “The path right now is the HMI is becoming the IIoT connection to everything,” said Philbrook. “There are some intelligent PLCs that can collect data from IIoT devices, but most PLCs are just trying to keep production running. But now, many HMIs have greater levels of functionality, with more sophisticated operating systems that support a variety of protocols.”
The CM5, the C-more brand’s latest generation of HMI, supports MQTT, a lightweight publish/subscribe protocol designed for massively parallel machine-to-machine communications. It’s ideal for exchanging data with the rapidly proliferating sensors and smart devices in the modern plant. “The HMI can gather data from the PLC, then distribute it to different systems, whether they be business systems in the local plant or cloud systems and remote locations,” says Philbrook.
The Shift Toward OPC UA
According to Hiroyuki Kuramoto, HMI and PLC project planner at JTEKT Electronics, the next step in the OT to IIoT convergence is the shift toward OPC UA. OPC UA is based on a client-server model. The server performs data sharing, while the client side connects to PLCs. This could be particularly useful for HMIs.
As HMIs connect to more and more PLCs, the challenge for HMI vendors, said Kuramoto, is implementing, testing, and maintaining the PLC protocols for different PLCs vendors. “By implementing the OPC UA client, it will be easier to connect to PLCs with OPC UA server built in. We believe that as an HMI vendor, we will be free from this difficulty in the future,” he said. “My opinion is that 10 years from now, most HMI and PLC connections will be OPC UA.
Future Technologies
Asked about the future, Kuramoto predicts two major changes in HMI functionality:
- HMI functionality will be built into the PLC
- HMI functions will run in the cloud.
HMI Functionality Built into the PLC
Developing controllers capable of controlling both the PLC and HMI in a single unit will reduce footprint and potentially streamline integration. “We believe these will be employed in relatively small facilities, such as food machinery,” Kuramoto adds.
Cloud-Based HMIs
The vision of running HMI functions in the cloud satisfies the desire of end users to access their equipment from anywhere. In a sample implementation, the data would remain on premises with the data source either a PLC or a headless HMI capable of aggregating data from multiple PLCs. The physical HMI on the machine would be fairly simple. The cloud-based HMI would exist in the form of a webserver that users would access through tablets or smartphones. This sort of solution would be a good fit for a large facility that needed to consolidate data.
There are two key technologies that will be key to supporting cloud-based HMIs – time-sensitive networking (TSN), implemented as OPC UA over TSN, and 5G/6G wireless communications. “We believe that we will enter the era of cloud HMI in conjunction with 6G penetration around 2035 to 2040,” Kuramoto said.
Machine Learning – Smart HMIs Get Smarter
As a final glimpse of a possible HMI future, we consider machine learning. “I think eventually it’s going to play a big role,” said Jordan Hulsey, director of service and automation at Caldwell Electric. “Maybe in the future, something as simple as facial recognition cameras setups on HMIs will be possible. This would allow touchless operator interaction for permission levels and vision tracking for HMI manipulation. You might have [two] operators, and facial recognition could log them into different permissions on the HMI.”
Conclusion
HMIs have come a long way since the days of manual pushbuttons. By putting actionable intelligence at the fingertips of operators, managers, and maintenance, modern HMIs maximize throughput and minimize downtime. Emerging capabilities promise to make these workhorse devices more powerful still. Given this, it’s easy to overlook perhaps the most important advance – ease of use. From programming to data visualization to integration, the latest HMIs are designed to be as easy to operate as they are to install. The guiding design principle makes every person around these components in the plant more efficient at the core mandate of maximizing throughput and profitability.