Our Latest News

An article to explain the structured light principle of three-dimensional vision

3D vision of structured light principle explained

In general, the said structured light can be divided into two main categories

Line scan structured light.

Faceted structured light.

Generally when talking about structured light refers to the second category, and here also the main focus is on face-array structured light.

  1. Line scan structured light

Line scan structured light is simpler and more accurate than surface array structured light, and is widely used in industry for object volume measurement and 3D imaging.

1.1 Mathematical basis

Let’s first look at a simple case in two dimensions.

A basic structure of the line scan structured light device can be seen in the above figure. The active light source L slowly sweeps through the object to be measured, during which the camera records the corresponding scanning process, and finally, based on the parameters such as the relative poses of the camera and the light source in the process and the camera internal reference, the 3D structure of the object to be measured can be reconstructed. From the above figure, we can see that.

can be obtained

Among them.

is the orientation of the projection device. a

then you need to pass the pixel coordinates of the corresponding pixel B

and the focal length f to determine. The final three-dimensional coordinates of point P can be found as U

Extending this to the three-dimensional space.

From the small-aperture imaging model there are

From the triangulation principle there is again

When the two equations are combined, we have

The final result is

As you can see, the situation in 3D space is similar to the previous 2D space, and 02fcfeac-8e73-11ed-bfe3-dac502259ad0.svg, which is the pitch angle, does not appear in the formula.

1.2 Applications

As shown above, the relative positions of the camera and the projector, etc., are precisely calibrated and a corner of the measurement table is chosen as the origin to establish the object-squared coordinate system. Therefore, the line laser projected by the laser projector can be described in the object-squared coordinate system by a plane equation as follows.

And the locus of the camera optical center is also known by geometric correction, the ray of the optical center and the pixel can be reconstructed by finding the corresponding pixel of the line laser in the image, and the intersection of the ray and the laser plane is the 3D spatial point to be found. From the small-aperture imaging model there are

Substituting into the plane equation, we get

  1. Faceted structured light

Faceted structured light can be broadly divided into two categories: random structured light and coded structured light. Random structured light is simpler and more commonly used. By projecting unevenly bright and randomly distributed point structured light into the measured space through a projector, the resulting binocular image is imaged by a binocular camera, and the corresponding depth map can be reconstructed after polar alignment and then binocular dense matching. The following figure shows some kind of infrared structured light of a surface array.

Random structured light is not mentioned here anymore, because it is very similar to the normal binocular algorithm. Some additional considerations are whether to add a filter to the camera, how dense the spot should be, and other hardware and optical issues. The main discussion here is on coded structured light. Encoded structured light can be divided into two categories.

Time-series encoding.

Spatial encoding.

2.1 Time-series coding

As shown above, the temporally encoded structured light is a series of light and dark structured light projected into the measured space by a projector within a certain time frame, and each projection is imaged by the camera. Assuming that there are n images, and that the shaded part is coded as 1 and the uncovered part is coded as 0. At this point, each pixel corresponds to a unique binary code of length n. The problem of searching for matching pixels in binocular images becomes one of finding pixels with the same coding value. If the binocular image has been polar-corrected, then the projected structured light only needs to be non-repetitive in the x-direction.

In the figure above, the pixel in the red box is coded as 0110, which translates to 5 in decimal, so it is only necessary to retrieve the pixel with the coded value of 5 on the same line on the right. The above encoding method is called binary code, and each segment is continuously bifurcated until the projected encoding width is equal to the pixel width of the camera. For an image with a width of 1024, a minimum of 10 images are required for encoding.

A modification of Binary Code is Gray Code. Gray Code has better robustness than Binary Code, it makes two adjacent pixels differ by 1 bit.

Note the difference between the gray code and the binary code in the first few lines of pixels.

Naturally, in addition to using binary 0-1 encoding, more color layers can be used for encoding. Assuming that M different gray levels are used for encoding, taking N images will result in an image containing

The effect of the number of bands. There are 27 bands in the plot for M = 3 and N = 3 in the figure below.

The advantages and disadvantages of time-coded structured light can also be derived from the above presentation: Advantages.

High precision.

Disadvantages.

Only applicable to static scenes.

Need to capture a large number of images.

2.2 Spatial encoding

To meet the needs of dynamic scenes, spatially encoded structured light can be used. While random structured light, which does not carry coding information and projects random textures, the spatially coded structured light discussed here is specifically structured light that projects a mathematically coded, non-repetitive range of spots in the measured space. As a result, the encoded value of a point can be obtained through its prodomain. The number of pixels containing a complete spatial encoding (window size) determines the accuracy of the reconstruction.

2.2.1 De Bruijn Sequence (De Bruijn) Sequence

De Bruijn sequence (Wikipedia) B(k, n) denotes a cyclic encoding of length 044b6abe-8e73-11ed-bfe3-dac502259ad0.svg with k symbols (e.g., binary, k = 2), and n is the length of an encoded value. Example: In the simplest case, with k = 2, using binary notation (0, 1) and the length of the encoded value n = 2, we get a cyclic sequence of length 045a25ae-8e73-11ed-bfe3-dac502259ad0.svg: [0, 0, 1, 1]. At this point, we get 4 different codes of length 2: [0, 0], [0, 1], [1, 1], [1, 0]. Thus, a certain structured light can then be encoded according to that de Bruijn sequence. And the obtained structured light image has the above 4 pixels encoded as [0, 0, 1, 1], and the encoded value of each pixel can be obtained through a sliding window of size 2 (assuming that the width of a structured light spot or beam is one pixel). Similarly, in the case of a polar-corrected binocular image, it is sufficient to search for the corresponding row, which only requires that the encoding is not repetitive in the x-axis. In this case, the structured light is a vertical strip. Of course, in order to improve the encoding efficiency, it is also possible to use projection methods such as grayscale maps and color images that have more possible encoding values than 0-1 encoding. For example, for an RGB image with binary encoding (i.e., a certain color has only two states, yes and no), there are 046985b2-8e73-11ed-bfe3-dac502259ad0.svg color combinations, and removing (0, 0, 0) leaves 7 colors. Thus k = 7, n = 3, which results in a strip sequence of length 343. For this sequence, the only constraint is that the adjacent strips cannot be of the same color. Otherwise it is easy to cause errors for the decoding algorithm. The following figure shows a structured light sequence using only 5 colors (k = 5, n = 3).

2.2.2 Two-dimensional space encoding

A de Bruijn sequence is a one-dimensional encoding that can be extended to a two-dimensional space such that for a two-dimensional space of size x * y, the encoded value contained in one of the sub-windows of size w * h appears only once in this entire two-dimensional encoding sequence.

As in the above sequence of 4 * 6 M-arrays, the encoded values contained in each window of size 2 * 2 are unique. It is also possible to use RGB information for 2D encoding, and there are related algorithms to generate some pseudo-random 2D encoding. For example, in the figure below, a 2-D matrix of size 6 * 6 is shown on the left, with sub-windows of size 3 * 3. The algorithm first fills the 3 * 3 sub-windows in the upper left corner with random colors; then a sliding window of size 3 * 1 is moved to the first blank space at the right end and filled with 3 random colors; before filling the generated random colors, the algorithm verifies that the uniqueness of the encoding of the sub-windows can be Before filling the generated random colors, the algorithm verifies that the uniqueness of the encoding of the sub-window can be guaranteed, and if not, it regenerates the 3 random colors; and so on, only the size of the sliding window in the vertical direction becomes 1 * 3, until the whole 6 * 6 matrix is filled. The figure on the right shows an example of some pseudo-random two-dimensional encoding produced by this algorithm.

The above discussion of spatial coding also reveals some advantages and disadvantages of spatially coded structured light.

Pros.

No need for multiple photos, only a pair of images is needed for 3D reconstruction. It can satisfy real-time processing and be used in dynamic environments.

Disadvantages

Vulnerable to noise interference: may result in missing coded information such as some areas during imaging due to reflections, illumination, etc.

More sensitive to occlusion in space.

Lower accuracy compared to time-series coded structured light.

The above is an introduction to some of the various commonly used structured light techniques. In fact, the most commonly used structured light in 3D reconstruction is random surface array structured light. By projecting such random structured light into space and combining it with binocular dense reconstruction, more reliable and accurate results can be obtained than simply using RGB images for 3D reconstruction.

Finally, for students interested in structured light and 3D reconstruction, we recommend a project: build your own 3D scanner. as the name suggests, the website provides everything you need to DIY a 3D scanner with a device that is within your reach and low cost, including tutorials, ppt, code, data and other people’s works, so interested students are welcome to try it.

    GET A FREE QUOTE

    FPGA IC & FULL BOM LIST

    We'd love to

    hear from you

    Highlight multiple sections with this eye-catching call to action style.

      Contact Us

      Exhibition Bay South Squre, Fuhai Bao’an Shenzhen China

      • Sales@ebics.com
      • +86.755.27389663