Interfacing IDS NXT
How to integrate IDS NXT cameras in your factory automation
With IDS NXT we have created a new generation of vision systems for industrial applications which can be used as powerful vision sensors in factory automation. Also, running IDS NXT Experience Kit in conjunction with Deep Learning is particularly easy to use. The all-in-one solution provides everything needed to create powerful inference cameras by running individual neural networks directly on-the-edge. From acquiring and managing images, to training the AI, to the final working IDS NXT inference camera, it's done in only a few steps. But how are the inference cameras integrated into the own environment?
In this Tech Tip we particularly address the interfaces for integrating IDS NXT cameras. And we show comprehensibly how you can use inference results in your own environment.
IDS NXT Interfaces
As compact embedded vision systems, IDS NXT cameras can handle image processing tasks directly on-the-edge and provide results without additional PCs. As a result, they can be used as autonomous vision sensors. In terms of access and data exchange requirements for external systems, they differ fundamentally from standard industrial cameras, which mainly serve as image providers for PC-based image processing. Therefore, we have equipped our IDS NXT cameras with interfaces that are easier and more flexible to use in IoT-type application scenarios.
IDS NXT cockpit
The IDS NXT cockpit provides access to all important settings and functions of IDS NXT industrial cameras and is completely based on their REST interface.
Additionally, you can install or configure Vision Apps. No matter which interface you use to work with the camera later on - the IDS NXT cockpit features an easy-to-use GUI to set up camera applications comfortably and configure them for runtime operation.
IO and RS-232
Even in the age of Industry 4.0, digital inputs/outputs and UART-based serial interfaces and the fieldbuses based on them offer numerous advantages over their state-of-the-art successors. Even for complex part inspections and image-based verifications, the supposedly simple results IO or NIO are usually sufficient to quickly and easily control products in large and modern industrial lines. Hardly any other interface can be implemented as easily and inexpensively as digital inputs or outputs. Similarly, serial device communication with RS-232 is supported by so many manufacturers and users. For no other is there a more comprehensive range of proven accessories. This is why RS-232 is still one of the standards in industrial communication today.
With IO and RS-232 interfaces, IDS NXT cameras complement many existing systems with compact and powerful embedded vision solutions without having to change the existing infrastructure.
IDS NXT REST Interface
With a REST (Representational State Transfer) web service, IDS NXT cameras offer a standardized open interface for machine-to-machine communication that allows easy networking. This enables parameters and results of the cameras to be changed or queried. The REST web service is based on the HTTP protocol and is therefore available in most networks.
The standard HTTP methods such as GET, POST, PUT, PATCH, etc. are used as a uniform interface. This means that the IDS NXT REST interface is not only platform-independent, but also, due to the widespread use of HTTP methods, can be used in a wide variety of clients (PCs, smartphones, tablets). While C++ applications require a completely set up development environment, REST(ful) APIs can be served by web-enabled device classes without additional software or gateways. This not only makes it easier to get started, but also makes it possible to package extensive setup actions into simple scripts.
For generic communication with autonomously operating vision sensors, REST can therefore be used in the world of IoT (Internet of Things) as well as IIoT (Industrial Internet of Things) more easily and with less overhead than a PC-based interface, such as GenICam. For IDS NXT cameras, REST is not only the "runtime" interface for data and result transmission, but is also used for the configuration (e.g. IDS NXT cockpit) of the camera, its apps and interfaces (IO, RS232).
Why REST?
- Stateless
Stateless communication ensures that each client-server message is self-contained and includes all necessary information about the current application status. This means that the REST service does not need to store anything between executions. Session management in the server is therefore not necessary, allowing the Web service to be scaled as required in the network. This makes REST ideal for use in IoT. - Unified and platform-independent
With REST, systems offer a standardized open interface based on web technology that allows for easy networking. Data and results can be exchanged between all (I)IoT devices without protocol conversion or gateways. - Easy and resource-saving
Building, passing, and possibly processing the results of simple REST API calls, in most cases, requires little processing power. Moreover, due to the widespread use of standard HTTP methods, the necessary technical requirements for REST communication can already be used on most web-enabled devices without additional software. - Location-independent
Clients and servers can be operated separately ("distributed system"). Even if they are hosted by different servers, communication via REST is still possible. - Industry and web-compatible
Easy access and transfer of data is an essential part of IIoT. With its flexibility and ability to use widely available resources, REST can also simplify industrial automation. Many systems and devices can be integrated quickly and easily by using cross-application and cross-device communication. Web-compatibility is a door-opener for a more comprehensive integration strategy. The evolution and development of the industry also benefits from the openness to new systems and technologies.
The IDS NXT REST web service is implemented with the following capabilities:
- The REST web service is always active during regular operation.
- Access protection is implemented via HTTP Basic Authentication. This means each request requires the user data of an authorized user for authentication, which is then transmitted Base64-encoded.
- The responses are transmitted in JSON or XML format.
Documentation
The documentation of the IDS NXT REST interface and Login information for IDS NXT cameras (IDS NXT cockpit manual, chapter "Logging on to the device") can be found in the download area of the IDS website for your IDS NXT camera model.
IDS NXT communication via IOs and RS-232
In order to communicate camera image processing results via digital inputs/outputs or the serial interface, you simply need to start the two IDS NXT Vision Apps "GPIO Gateway" and "RS-232 Gateway" via the IDS NXT cockpit and set from which vision app you want to output specific results. No programming is required. All settings can be done via the GUI.
As an example, you can configure the digital outputs to be active for a certain time each time the camera AI (CNN manager with "ImageNet1000") has classified specific objects (e.g. "Good" or "Bad"). Or you can communicate the detected object class and the probability of inference as result values over the serial connection.
Documentation
A detailed description of the two vision apps and their configuration can be found in the download area for your IDS NXT camera on the IDS website.
Download "GPIO Gateway" documentation (EN) Download "RS-232 Gateway" documentation (EN)
IDS NXT REST Communication
The big advantage of REST for developers is its web-compatibility and thus the simplicity of the required tools. Therefore, we show examples how you can easily use the HTTP methods GET, POST, PUT and DELETE to retrieve data from an IDS NXT camera.
Please note that this tech tip is not intended to be a comprehensive documentation for using the REST API. We would rather like to give you basic hints to make your start easier. The "rest" is up to you!
To be able to reproduce the following examples, you need an IDS NXT camera with access via a Windows PC in the same network. To avoid stupid queries of parameters, we will let the camera do something useful. We use the camera AI and the pre-installed CNN "KritzelNN". This enables the camera AI to recognize drawings of people, dogs, cats and cameras. Via the REST-API we request what it recognizes on a self-painted drawings.
Test scenario "I see something you don't."
- Network with IDS NXT rio or rome camera and a Windows PC
- Running vision app "CNN Manager" with the pre-installed neural network "KritzelNN".
- Various drawings of a person, a camera, a cat or a dog.
Documentation
Documentation on how to use the CNN Manager including the required REST-URLs and how to start the "KritzelNet" can be found in the download area for your IDS NXT camera on the IDS website. Download "CNN manager" documentation (EN)
REST call with "cURL" (command line)
cURL is a command line tool for transferring data via URLs, which you can simply run at the command prompt (also cmd.exe). To query the inference result (KritzelNN) of the last camera image, we send a 'GET' request with the desired resource '/vapps/cnnmanager/resultsources/last'.
Camera image:
C:\Users\ids>curl -sX GET http://192.168.2.102/vapps/cnnmanager/resultsources/last --user admin:ids
{
"inference": {
"Top1": "person",
"Top2": "camera",
"Top3": "cat",
"Top4": "dog"
},
"inference_propability": {
"Top1": "0.99",
"Top2": "0.01",
"Top3": "0.00",
"Top4": "0.00"
},
"inferencetime": {
"Content": "43"
}
}If you want to filter the JSON response data, you can use an additional tool like 'jq'. The lightweight and flexible command line JSON processor does not require any other dependencies. To output only the top1 inference result and its probability, extend the request:
| jq ".inference.Top1, .inference_propability.Top1, .inferencetime.Content"
C:\Users\ids>curl -sX GET http://192.168.2.102/vapps/cnnmanager/resultsources/last --user admin:ids | jq ".inference.Top1, .inference_propability.Top1"
"person"
"0.95"Now place various drawings in front of the camera lens for testing and call up the respective inference results.
REST call in the browser
Directly via the address line
The inference query can also be very easily done via web browser (e.g. Firefox or Google Chrome). To do this, enter the following URL in the address line:http://admin:ids@192.168.188.21/vapps/cnnmanager/resultsources/lastThe user credentials are transferred in the standard HTTP authorization header.
Unfortunately, for security reasons not all browsers (e.g. IE) support embedded credentials in the URL anymore!
With Browser Plugins
Rest API Browser Plugins
There are also a number of useful browser plugins with very clear GUIs for testing REST calls.
We test our query of the inference result with the Google Chrome Browser and the extension "YuiAPI".
REST call in special client
Probably the most comfortable way to communicate with a REST web service are clients such as "Postman". With the clearly arranged GUI and a lot of support by status codes, response times, an automatic speech recognition as well as link and syntax highlighting when creating REST calls, even non-experts will quickly gain access to REST. The analysis and filtering of response data is also simplified. Another advantage is the possibility to import complete API descriptions or to create them yourself by using one of the known API specifications such as RAML, WADL or OpenAPI.
Postman is a visual development environment for communicating with the REST web service of IDS NXT cameras. REST calls can not only be created, but can also be stored and organized in collections.
Data that you repeatedly need can thus be conveniently provided in variables. But probably the most important function of Postman is its code generation. At the push of a button, the client creates fully usable code snippets from your tested and saved REST calls for many common programming languages and other clients, such as cURL, C, C#, Python, Java or Swift.
The Postman code snippets provide you with a good base for integrating IDS NXT cameras into self-programmed applications.
REST API integration into programmed applications
Although no development SDK is provided for IDS NXT cameras, for e.g. C++ applications, this does not mean that a programmatic integration of the camera is not possible. Using REST or HTTP connections is possible in most programming languages without additional software from the manufacturer. That is why a REST client like Postman is able to provide you with fully functional code snippets for direct use.
Using our AI camera result as an example, we show you how to program the REST call in Python.
REST call with Python
import http.client
import mimetypes
conn = http.client.HTTPSConnection("192.168.188.21")
payload = ''
headers = {
'Authorization': 'Basic YWRtaW46aWRz'
}
conn.request("GET", "/vapps/cnnmanager/resultsources/last", payload, headers)
res = conn.getresponse()
data = res.read()
# output camera response
print(data.decode("utf-8"))Python is a very common programming language in IoT. We would therefore like to explain the integration of our IDS NXT cameras into a production automation system in more detail using Python.
Interactive development with Python
With the tech tip, we provide you with additional Python code snippets in a software package, which show the basic handling of IDS NXT cameras and their interfaces.
With the included Jupyter notebooks you will be able to test and understand the Python examples and the data exchange with your IDS NXT camera in a live open source web application.
1. Log-in and image transfer (connect_and_get_image.ipynb)
The first example shows how to use the GET command to authorize and retrieve an image from your IDS NXT camera.
2. Change exposure time (change_exposure.ipynb)
The second example shows how to request and modify camera parameters. To do this, we change the exposure value using the PATCH command.
3. Trigger image acquisition (trigger_image_acquisition.ipynb)
Using the POST command, we will show in the third example how to trigger an image capture to retrieve the image content of a modified scene.
4. Configure the IO Gateway (set_gpio_gateway.ipynb)
In the fourth example we will show you how to toggle the cameras digital outputs depending on the results of the CNN manager vision app.
Python code examples Download the Python code examples as Jupyter notebooks
How to use the Jupyter notebooks and test the examples is explained in the appendix "Talking to the IDS NXT with Python".
Summary
With digital inputs and outputs, RS-232 and a REST web service, IDS NXT cameras provide standard interfaces that allow them to be used very flexibly. Embedded vision devices that perform image processing tasks independently without a host PC therefore do not require PC support for communication with remote stations. The stateless REST interface enables connection to all web-compatible devices, which means the cameras are optimally positioned for an IoT network structure. With suitable REST clients (such as Postman), software wrappers for many common programming languages can be created at the push of a button.
With the existing tools and interfaces of the IDS NXT inference camera platform, integration into existing installations with proven technology is just as easy as into future installations with web-compatible IoT structures. For more information about IDS NXT or its integration, please visit our knowledge base or contact our system consultants.
Tip: IDS NXT Webinar Video
Our webinar video "Integration of IDS NXT in Factory Automation" demonstrates the communication via the IDS NXT camera interfaces using Python sample codes and Jupiter notebooks.
Read more Tip: Integrate IDS NXT cameras into factory automation using OPC UA
In the sequel "Interfacing IDS NXT using OPC UA", we focus on the new communication possibilities of IDS NXT cameras through their integrated OPC UA Server. Using an AI sample workflow, we show how you can activate saved camera settings, start an inference task and query results via an OPC UA client. This makes it even easier to integrate IDS NXT cameras into OPC UA-based industrial applications.