Computer vision has been used for product inspection and quality control across many industries, typically involving the automated recognition of components followed by an analysis of geometric measurements to detect flaws and defects.
Recently, scientists at the Singapore Polytechnic University developed an AI-based hybrid imaging system that merges computer vision with deep learning networks, taking this technology in a new direction: construction worker training certification. They aimed to minimize the mundane manual process of assessing apprentices, saving hours on labor and improving the accuracy of test results.
In the construction industry, apprentices are required to go through extensive training and certification to ensure they have the necessary skills for a highly specialized trade, such as carpentry or plumbing. To be certified, apprentices build intricate assemblies from blueprints that must adhere to modern building requirements and be finished in a set amount of time. Apprentices' work is painstakingly evaluated by senior tradesmen, a task that can take hours.
Corner Detection Algorithm
In the study, the AI-based computer vision system devised at Singapore Polytechnic employs a corner detector algorithm available in Matlab image processing libraries. This algorithm is combined with deep learning techniques, along with a high-resolution camera, to measure the dimensions, orientation, and position of a complex wall assembly consisting of electrical boxes, switches, steel conduits, fittings, raceways, and wiring. Once assembled by an apprentice, each part is analyzed by the algorithm to determine any discrepancies from the blueprint.
Deep learning can detect the positioning of the electrical components and their reference corner points in each acquired image. Using a pixel-based filter, such as the Minimum Eigen Value Algorithm, and further filtering the results with the backing of deep learning output, the Singapore Polytechnic system detected the corners of components very accurately.
System performance depends on the identification of individual components and the position of their corner points, making the imaging capabilities of the camera critical. After researching several cameras, scientists chose an SVS-Vistek EXO 540CU3 industrial camera featuring a 24.5-mp color sensor featuring a global shutter. With a 2-in. (50 mm lens), the camera was set at the midline of the wall assembly with a resolution set at 5320 × 4600 pixels. Images were transferred at 15 fps using the camera's USB3 Vision interface. The camera was calibrated with checkerboard images, along with Matlab's camera calibration tools, for accuracy.
Automated System Processing
Using the acquired image, the Singapore Polytechnic system first detects the type of wall assembly being assembled with its trained deep learning model so it can perform computations accordingly. Next, object corners are detected that act as reference points in the measurement of component dimensions, alignments, and angles.
Before testing, a template of the finished wall assembly is uploaded into the system as a CAD drawing. Detected features are compared against the template so discrepancies can be identified. Finally, the system generates a report detailing the inspection results that are used in certification.
The scientists acquired a collection of images with drastically varying lighting conditions and camera angles to further assess the accuracy and consistency of the trained network, keeping in mind that the inspection system will operate in dynamic real-world environments rather than controlled labs. Underscoring the image quality of the system, the majority of these test results were above 93% accuracy.
Future System Development
Certification of apprentices can be an arduous, time-consuming process requiring multiple experts to be on-site to monitor testing. This new computer vision solution can reduce certification time significantly compared to manual inspection. Additionally, it's possible to capture and process images remotely, eliminating the need for travel.
According to Singapore Polytechnic, the system opens up new avenues to explore other certification processes in the trades besides electricians, such as apprentices in plumbing, HVAC, and carpentry. The scientists intend to create a general-purpose system with multiple configurations that can be customized by instructors to analyze diverse apprentice work.
1. Computer Vision-Based Inspection System for Worker Training in Build and Construction Industry, Singapore Polytechnic, Fikret Ercan, Ricky Ben Wang, School of Electrical and Electronic Engineering, February 2022
SVS-Vistek GmbH was founded in 1987 for the sale of optoelectronic components. In 1999 it began designing, developing, and manufacturing its own CCD and CMOS cameras in Germany. Today, SVS-Vistek is one of the world's leading manufacturers of industrial cameras with an international presence. SVS-Vistek GmbH is a TKH Technology Company. Learn more at www.svs-vistek.com