In the beginning of 1980 industrial vision systems did not exist. Universities experimented with image analysis using, at that time, big computers. There was no way this research could end up on the factory floor.
Matrox in Canada launched a graphics module with 256 X 256 pixels by 4 bit. They also made an Analogue to Digital converter so binary video signals could be fed to the frame store. Thereby it was possible to interface a camera and store one image in the computer.
Cameras at that time were made with a vacuum tube where electronic scanning swept through the image projected onto the front glass. These cameras were prone to drift with temperature and were highly sensitive to external magnetic fields. The images were awful and not suited for applications where you wanted to measure with any precision.
After a visit to Matrox in Montreal in 1980, I borrowed a computer with frame store and interface to a tube camera. This was exhibited at a computer fair in Copenhagen in 1981. It attracted a lot of attention. The mere fact that you could store a picture in the computer was astonishing.
Ole Hansen from NKT nail factory came by, and was fascinated by the concept of storing an image in a computer. If the image was in the computer it should be possible to place a nail under the camera and measure if it was straight. This was a production problem this new technology may solve.
After the show Ole Hansen contacted me and asked if we could work on the idea. We agreed to start a paid preliminary study on the application.
We experimented with a tube camera and discovered how inaccurate the image was. It was distorted in all sorts of ways making it impossible to get any precise measurements.
But we spend a lot of time finding ways to compensate. We placed a straight piece of steel beside the nail and could then measure relative to the reference. We placed the optical centre right between the reference and the nail we could then measure the difference between these two lines. It was all done with backlighting and contrast was no problem.
After researching this, we made a report assuming it was possible to measure the straightness of a 13 mm long nail with a precision of 0.1 mm. An incredible accuracy when the pixel size was 0.8 mm. It was all done with compensation and a smart way of counting pixel steps over a set distance.
Ole Hansen came back and asked for a proposal for a complete machine to be used in NKT’s laboratory. We made the proposal – it was very expensive, but we got the order.
Now it became real. We had to develop the system. I was not a programmer so we hired an engineer who could program the Z80 in assembler. We found just the right guy Jesper Nilausen.
I had put a lot of thoughts into how the programme should work. Jesper looked at the design, and two weeks later he had programmed a working system. It was the first industrial vision system in Denmark and maybe in Europe.
The first system was delivered in 1981 and It was very good business, probably the best vision project ever
The next project was for Danish Steel where we should measure the dimensions of a red hot slab just before rolling. At that time the very first CCD cameras came out of the laboratories, and we managed to get two cameras ser. no. 002 and 004 from Fairchild. They had a lot of dead pixels and lines, but we guessed we could program our way around it.
To achieve the accuracy we used integration techniques and all sort of tricks. It was the first system with sub pixel accuracy. If you integrate over a lot of measurement points in the image, you improve the resolution with the square root of the number of measurements. We achieved an accuracy far better than demanded, and by using CCD cameras we had overcome most of the drift and un-linearity compared to tube cameras.
Risø the nuclear research centre in Denmark was asked to evaluate our design and concluded that pixel integration did not improve the accuracy. However, we build the system and it worked with tremendous accuracy. 10 years later at the SCIA conference in Aalborg a paper was presented proving pixel integration. I was pleased but also astonished that it could take 10 years before the methods we developed surfaced from the universities as a proven concept.
All the systems we have made since the eighties build on pixel integration, but even today new engineers from the universities may never have heard of it.
With these successes we thought we had invented the computer with vision
We took on many projects which were not successful. Some of them are just becoming possible today 35 years later, with the Machine Learning and AI.
My experience goes way back to tube cameras and Z80 computers with 64 Kb RAM. This experience is invaluable when we take on really tough projects, where we must measure with great accuracy.
Today we can achieve up to 500 times better accuracy than the pixel size, and have developed systems capable of measuring down to 4 nm.
This is astonishing as we use light with a 570 nm wavelength.
Optics and illumination is of great importance. In the eighties we had to develop and build illumination and optics ourselves. Today as the vision market has exploded you can buy virtually everything as catalogue items.
Without the history and previous projects we could not make the advanced systems we do today. All the tricks and methods, compensations and calibrations learned through hundreds of projects are invaluable.
An important part of successful vision systems is craftsmanship. You may have patented vision algorithms and all sorts of advanced functions, but when it comes down to it, the system must often work in hostile environments, be integrated in complex production equipment and it must be simple to understand for all the staffs.
The robustness is also important. We have systems installed in before yr. 2000 still in operating without being serviced.
The latest technology Artificial Intelligence and Machine Learning is matured and can be used in many projects previously programmed the traditional way.
We now develop systems with a combination of traditional vision and ML we call it “Hybrid Vision”. We believe this is the way to go forward.
Learn how we used Hybrid vision to solve a challenging inspection task in the wood industry right here: Download case story
