Stereo 3D camera helps robots get to grips with bin picking!

3D machine vision is the future for robot vision. Many applications in robotics and automated serial production can only be satisfactorily implemented using three-dimensional data. This applies equally to challenging assembly processes such as bonding and welding, and to the notoriously tricky process of bin picking.

The ability to accurately detect the precise position of the unsorted parts is essential for reliable bin picking by  robots. The Ensenso stereo 3D camera from IDS represents a solution for 3D image capture that is impressively precise, cost-efficient and fast. The German company bsAutomatisierung GmbH is using the new options available with this camera to achieve some seriously speedy bin-picking solutions.

Bin-picking robot cell from bsAutomatisierung
Bin-picking robot cell from bsAutomatisierung

Bin picking remains one of the most difficult tasks to execute in robot-supported production. In order to be able to pick unsorted parts out of a wire-mesh pallet or stacking container, it is necessary for the robot to be able to accurately detect the shape, size, position and alignment of the objects. All of this information is essential to the ad-hoc generation of collision-free robot paths.

However, challenges arise as soon as you begin to capture 3D data because short cycle times and process stability are required in addition to accuracy and completeness. The small number of solutions that have been available up to now are either too slow or fail to achieve the required level of process stability to be suitable for use in serial production. Alternatively, they are simply too expensive.

The Ensenso stereo 3D camera from IDS successfully covers all bases in terms of the requirements for cycle times, availability and cost efficiency that apply to robot vision applications. The industrial camera with a USB connection is the first of its kind on the market, integrating, as it does, not only two global shutter CMOS sensors with WVGA resolution and powerful software in a very compact housing, but also an infrared pattern projector. It projects a random pattern of dots onto the object to be captured, allowing structures that are not visible or only faintly visible on the surface to be enhanced or highlighted. This is necessary because stereo matching requires the identification of "interest points" in an image. The object is then captured by the two image sensors in accordance with the stereo vision principle. Finally, 3D coordinates are reconstructed or calculated for each and every pixel using geometric relations based on the triangulation principle. This means that, even if parts with a relatively monotone surface are placed in the bin, a virtually seamless and detailed 3D image of the entire surface can be generated without additional technical effort. All of this happens in just a few milliseconds.

Two Ensenso N10 stereo 3D cameras are installed in a fixed location for each container.
Two Ensenso N10 stereo 3D cameras are installed in a fixed location for each container.

The speed offered by Ensenso has benefits for bsAutomatisierung when applied to its bin picking robot cells. The company, which has its headquarters in Rosenfeld, approx. 60 km south of Stuttgart, specializes in the development and construction of systems for fast and precise loading and unloading of production machinery and parts handling. Its bin-picking cells can automatically pick individual, randomly aligned parts out of a container and pass them on to downstream production processes. The cells achieve cycle times of less than 10 seconds, which would not be possible with conventional solutions such as the light section method. Depending on the application and on customer requirements, the cells can be supplied with a classic multi-axis industrial robot or with a linear portal robot. They have a modular structure, can be adjusted to suit various container sizes, and are designed for use with either a single bin or up to four bins simultaneously.

Each bin is monitored by two stationary mounted Ensenso cameras. This offers two key benefits over a camera installed directly on the robot arm. Firstly, the system can achieve shorter cycle times. While the robot is busy picking parts out of one bin, the process of object detection can already be started in another bin. Even if parts only need to be picked from a single bin, the process of searching for the next part can begin while the robot is placing the most recently picked part elsewhere.

As an additional benefit, the images from different Ensenso cameras can be very easily calibrated with each other, as the camera software has been specially designed to handle multi-camera applications. When two or more cameras are used simultaneously, a scene can be captured from different sides at the same time, thus reducing shadows and extending the picture field. When working with multiple cameras, the N10 software similarly delivers a single 3D point cloud containing the data from all cameras used. The resulting precision can also be scaled to meet virtually any requirements by adjusting the number of cameras installed. In addition, the Ensenso software not only allows you to combine several stereo cameras, but also enables the use of these cameras in conjunction with conventional industrial cameras, for example, to capture color information or barcodes.

Finally, the Ensenso software assumes control of the two CMOS sensors and pattern projector, and takes care of the capture and pre-processing of the 3D data. As a result, both frame rate and image quality are optimized, and so significantly fewer resources are required on the PC used for analysis. The camera is designed for working distances of 260 mm to 1,400 mm and for variable picture fields. The available focal lengths of 3.6 to 16 mm can cover a wide range of distances and sizes. Unlike other 3D imaging processes, this camera can capture both stationary and moving objects with a frame rate of up to 30 fps. Despite hav-ing two sensors and an integrated projector, the camera only measures approx. 150 x 45 x 45 mm, and weighs just 400 g. Thanks to its robust aluminum housing and GPIO connector for 12-24 V hardware triggers, input and output, the Ensen-so is perfectly equipped for industrial applications. Accordingly, the 3-pin M8 sen-sor/actuator connector and USB cable have a screw-mountable design.

The Ensenso industrial camera integrates two CMOS sensors and an infrared pattern projector in a compact housing.
The Ensenso industrial camera integrates two CMOS sensors and an infrared pattern projector in a compact housing.