Inference by recipe

OPC UA introduces AI-based image processing to the field level

Important components of industrial applications are machine vision systems, which are becoming more and more precise and work intelligently. Ethernet-based industrial protocols such as OPC UA also enable their direct integration into factory automation without any effort or additional hardware. In conjunction with the turnkey solution IDS NXT Experience Kit, IDS NXT industrial cameras with their Vision App-based operating system meet the exact requirements of automation engineers. Equipped with OPC UA through a software update, they make their AI capabilities available via a "recipe collection" in the network and communicate results directly from the field level to the machine and control level without a detour. It's so easy. This means that IDS NXT cameras are not just a building block, but a milestone for industry 4.0 systems.

Although there is no cookbook with recipes for inference applications available in stores yet, with the all-in-one inference camera solution IDS NXT Experience Kit, you can train an AI classifier with your own image data without any prior knowledge of artificial intelligence, which is immediately executable on IDS NXT cameras. Thanks to the integrated OPC UA server, the inference tasks and their results can be made available to every client in the industrial network via recipe and result management.

"The goal must be to consistently further develop industrial cameras and machine vision products for industry and make them easy to control. OPC UA paves the way for easy and fast integration of state-of-the-art technology into industrial applications."

— Maximillian Keller, OPC UA expert at IDS —


OPC UA is a key technology for exchanging data and information in industry between devices, machines and services, even from different sectors. The industrial protocol is establishing itself more and more as an open standard in Industry 4.0 through features such as vendor independence, scalability and communication security. This is also due to the fact that OPC UA works via web protocols in any TCP/IP network, making communication completely platform-independent. All this has led to a rapid spread of the IT standard. OPC UA has thus managed to reach through and connect the entire factory. The uniform exchange across all physical layers and ethernet-based fieldbuses up to the cloud brings an enormous simplification of the engineering effort if data does not have to be read, converted and synchronized via different protocols.

OPC UA unifies the language of the devices among themselves and defines how devices describe themselves to clients, or which information and which services they offer. The advantage is that machine data can not only be transported, but can also be described semantically in a machine-readable form. OPC UA uses so-called companion specifications, which comprehensively "describe" devices of a certain industry and thus provide a generalized and simplified view of proprietary device information, while the content remains vendor-specific and is treated as a black box.

The available device services - complex image processing tasks in case of an IDS NXT inference camera - are encapsulated by OPC UA in function blocks, so-called "recipes". Their exact mode of operation does not have to be known by the automation engineer or the user for pure use. Likewise, necessary and sometimes in-depth settings for special applications are summarized and stored in configuration data records. Without having to know the device-specific setting options in detail, these "presets" help the user to optimally adjust the devices for the execution of special recipes. This simplifies integration processes and commissioning of new devices in the application workflow.

Self-describing inference camera - "Hello Industry 4.0"

In the world of automation and industrial IoT there are many devices and sensors that provide services and information. For this purpose they must act as servers in the OPC UA network. Depending on what information a client needs for further processing, it may need to connect to several of these servers. Image suppliers such as industrial cameras are essentially not typical OPC UA devices, as they do not provide directly usable information or results. The situation is different with intelligent cameras that have independent evaluation functionality. Especially for embedded vision devices like IDS NXT inference cameras, OPC UA is the perfect language for self-description in Industry 4.0. As the first inference camera with its own OPC UA server, they can be used directly by control manufacturers. They work like a vision sensor providing results, variable information and services in the OPC UA environment due to their flexible mode of operation.

Direct integration of IDS NXT cameras via OPC UA
Direct integration of IDS NXT cameras via OPC UA

Product change at the push of a button

The cameras provide image processing tasks and their results via vision apps, which are as easy to use and change as apps on a smartphone. On the hardware side, the camera platform is supported by a parallel processing FPGA that can be programmed at run-time and which, as an AI accelerator, can accelerate many already known architectures of neural networks. The fast reconfiguration of the dedicated processor offers additional advantages in terms of future-proofing, low recurring costs and time-to-market. In addition, switching between several loaded CNNs can be performed at runtime in a few milliseconds. The use of the vision apps is completely mapped in the industrial network by the OPC UA recipe and configuration management and thus enables, for example, a product change at the push of a button.

Plug and produce

Device information and result data of the different vision apps of an IDS NXT camera are disclosed by the OPC UA information models and the companion specification for machine vision systems as an object-oriented data structure. Every client can use it to obtain the information he needs or even be informed about changes. Uniform communication via OPC UA thus creates the basis for "plug and produce" with new devices. IDS NXT inference cameras can thus be quickly and easily used in an OPC UA network as a machine vision system. The commissioning time is thus reduced and simplified to a minimum.

Interface combination

The REST (Representational State Transfer) web service and the OPC UA industrial protocol offer standardized, platform-independent interfaces that allow easy networking with other web-enabled device classes without additional software or gateways. In addition, it is the decisive interface when it comes to setting up the inference tasks with the associated settings and making them available in the OPC world via recipes and configurations. This is because the OPC UA specification itself does not provide function calls for the device- and manufacturer-specific processes. Since image processing systems are hardly comparable in terms of their control, mode of operation and the settings and data used. This makes it difficult to create a standard without limiting the functionality of the very individual systems or inflating a generic interface too much. This means that unique selling points for device manufacturers remain possible and OPC UA communication remains lean and scalable. OPC UA is therefore the ideal supplement for IDS NXT cameras to make camera configurations and Vision App-based inference tasks in industrial environments accessible in a format that is compatible and readable for OPC UA clients.

OPC UA and device-specific REST communication
OPC UA and device-specific REST communication

"You have to improve the usability of the systems to make complex topics easy to master. Then the automation specialists can make the best possible use of their expertise."

— Patrick Schick, Product Marketing Manager at IDS —

Usability taken a step further

It is not enough to simply make the capabilities of a machine vision system, for which you sometimes need specialist knowledge, known on the network. For automation engineers, machine vision solutions should be easy to use. Then you yourself are able to describe how an image must be processed without having to call in specialists. The all-in-one inference camera solution IDS NXT Experience Kit has been developed for exactly this application. Their usability is simple enough to intuitively realize even complex analyses without image processing and AI knowledge. With the cloud-based software solution IDS lighthouse, application images for AI-based image processing tasks can be managed and individual neural networks can be trained within minutes. Only three steps are necessary: Upload training images, evaluate and label training images according to the application and finally train the desired network fully automatically, which is then immediately ready to run on IDS NXT cameras. Users only need the knowledge of their images and their application in order to reliably identify even highly varying objects - such as the classification of different types of fruit or the detection of damaged areas on fruit. With OPC UA, the required encapsulation of the function blocks by means of standardized communication is now also possible. This finally brings industrial image processing and PLC closer together.

Future-proof through software update

Thanks to the app-based concept, the IDS NXT camera platform is as versatile as smartphones. Recurring tasks, such as reading codes, characters or license plates, as well as finding, measuring, counting or identifying objects, can be set up and changed quickly. With vision apps such as "Classifier" and "Object Detector", turnkey solutions are available to classify image components or to identify specific objects using artificial intelligence. The IDS NXT Vision App Creator SDK, on the other hand, allows developers and AI specialists to delve even deeper into the system by creating individual vision apps, allowing them to customize IDS NXT cameras even better to their own needs. The design possibilities and application scenarios are therefore virtually unlimited.

But also the AI technology is advancing so rapidly that new frameworks and architectures are added every month. IDS is therefore constantly developing the camera platform and providing new functions and tools with software updates. This allows users to expand the capabilities and possibilities of their IDS cameras without having to buy new hardware.


With simple usability and highly flexible and expandable deployment options, IDS NXT inference cameras deliver directly actionable process decisions to the OPC UA network. This saves time and costs in automation and ensures rapid integration into industrial plants. With the OPC Machine Vision Companion specification, the cross-manufacturer and cross-industry communication technology provides the ideal interface extension to easily "refine" factory automation with the AI-based "recipes" of IDS NXT embedded systems. Since neither previous knowledge of image processing nor machine learning is required, IDS NXT inference cameras are a milestone for Industry 4.0.