In the beginning of 1980 industrial vision systems did not exist. Universities experimented with image analysis using, at that time, big computers. There was no way this research could end up on the factory floor.
Matrox in Canada launched a graphics module with 256 X 256 pixels by 4 bit. They also made an Analogue to Digital converter so binary video signals could be fed to the frame store. Thereby it was possible to interface a camera and store one image in the compute
Cameras at that time were made with a vacuum tube where electronic scanning swept through the image projected onto the front glass. These cameras were prone to drift with temperature and were highly sensitive to external magnetic fields. The images were awful and not suited for applications where you wanted to measure with any precision.
After a visit to Matrox in Montreal in 1980, I borrowed a computer with frame store and interface to a tube camera. This was exhibited at a computer fair in Copenhagen in 1981. It attracted a lot of attention. The mere fact that you could store a picture in the computer was astonishing.
Check out this case story about the impact of industrial machine vision today
Ole Hansen from NKT nail factory came by, and was fascinated by the concept of storing an image in a computer. If the image was in the computer it should be possible to place a nail under the camera and measure if it was straight. This was a production problem this new technology may solve.
After the show Ole Hansen contacted me and asked if we could work on the idea. We agreed to start a paid preliminary study on the application.
We experimented with a tube camera and discovered how inaccurate the image was. It was distorted in all sorts of ways making it impossible to get any precise measurements.
Camera picture tube from the time before CCD and CMOS cameras.
But we spent a lot of time finding ways to compensate. We placed a straight piece of steel beside the nail and could then measure relative to the reference. We placed the optical centre right between the reference and the nail and we could then measure the difference between these two lines. It was all done with backlighting and contrast was no problem.
After researching this, we made a report assuming it was possible to measure the straightness of a 13 mm long nail with a precision of 0.1 mm. An incredible accuracy when the pixel size was 0.8 mm. It was all done with compensation and a smart way of counting pixel steps over a set distance.
Ole Hansen came back and asked for a proposal for a complete machine to be used in NKT’s laboratory. We made the proposal – it was very expensive, but we got the order.
Now it became real ! We had to develop the system. I was not a programmer so we hired an engineer who could program the Z80 in assembler. We found just the right guy Jesper Nilausen.
I had put a lot of thoughts into how the programme should work. Jesper looked at the design, and two weeks later he had programmed a working system. It was the first industrial vision system in Denmark and maybe in Europe.
It was very good business probably the best vision project ever. The future for vision looked bright !
The next project was for Danish Steel where we should measure the dimensions of a red hot slab just before rolling. At that time the very first CCD cameras came out of the laboratories, and we managed to get two cameras ser. no. 002 and 004 from Fairchild. They had a lot of dead pixels and lines, but we guessed we could program our way around it.
To achieve the accuracy we used integration techniques and all sort of tricks. It was the first system with sub pixel accuracy. If you integrate over a lot of measurement points in the image, you improve the resolution with the square root of the number of measurements. We achieved an accuracy far better than demanded, and by using CCD cameras we had overcome most of the drift and un-linearity compared to tube cameras.
Risø the nuclear research centre in Denmark was asked to evaluate our design and concluded that pixel integration did not improve the accuracy. However, we build the system and it worked with tremendous accuracy. 10 years later at the SCIA conference in Aalborg a paper was presented proving pixel integration. I was pleased but also astonished that it could take 10 years before the methods we developed surfaced from the universities as a proven concept.
All the systems we have made since the eighties build on pixel integration, but even today new engineers from the universities may never have heard of it.
With these successes we thought we had invented the computer with vision. We took on many projects which were not successful. Some of them are just becoming possible today 35 years later, with the Machine Learning and AI.
My experience goes way back to tube cameras and Z80 computers. This experience is invaluable when we take on really tough projects, where we must measure with great accuracy. Today we can achieve up to 500 times better accuracy than the pixel size, and have developed systems capable of measuring down to 4 nm. This is astonishing as we use light with a 570 nm wavelength.
Optics and illumination is of great importance. In the eighties we had to develop and build illumination and optics ourselves. Today as the vision market has exploded, you can buy virtually everything as catalogue items. However, we still make special very powerful lighting for our glass tube inspection system. For tableware we build the Dynamic Light Box - a very bright strobed LCD screen backlight - generating all sorts of funny patterns.
Without the history and previous projects we could not make the advanced systems we do today. All the tricks and methods, compensations and calibrations learned through hundreds of projects are invaluable.
In the beginning we had severe constraints because of hardware. Making systems with a 10 Mhz Z80 and 64 Kbyte ram was a challenge. Today we rarely need to optimize the speed of the programmes. The computers are so powerful and cheap. If we need to execute faster we simply add another computer.
An important part of successful vision systems is craftsmanship. You may have patented vision algorithms and all sorts of advanced functions, but when it comes down to it, the systems must work in many different environments, be integrated in complex production equipment and it must be simple to understand for all the staffs. The robustness is also important. We have systems installed before year 2000. They have been running flawlessly and never need service.
The latest technology - Artificial Intelligence and Machine Learning - is matured and can be used in many projects previously programmed the traditional way. We now develop systems using a lot of craftsmanship, and a combination of traditional vision and ML. We call it “Hybrid Vision”. We believe this is the way to go forward.
