Generation and Display System of Measurement Matrix Based on DMD

Wenzhao Gu， Fu Zheng and Guangjie Zhai,

(1.National Space Science Center, Chinese Academy of Sciences, Beijing 100190, China; 2.University of Chinese Academy of Sciences, Beijing 100190, China)

Abstract: A measurement matrix is the key to sampling and signal reconstruction during the process of compressed sensing. On the basis of digital light processing (DLP) technology, a generation and display system of measurement matrix based on digital micro-mirror device (DMD) is proposed and well designed. In this system, the generation and controlling of measurement matrix are implemented on a PC, which reduces the hardware requirement to generate a random matrix and overcomes the difficulty of the hardware implementation for the random matrix. It can set up the display number of the measurement matrix, the mode of display and display time according to the requirements from users. The display information can be designed to complete the display of measurement matrix with a better adaptability. The system can be easily embedded into a variety of compressed sensing applications, which can be used to generate and display the corresponding measurement matrice with strong portability. In addition, the DMD of this system will be used as a spatial optical modulator to manipulate near-infrared light in a fast, accurate and efficient way in several applications such as in 3D scanning devices and spectrometers.

Key words: measurement matrix; digital micro-mirror device (DMD); compressed sensing; Gaussian random matrix

With the advent of the information age, the pressure of information demand, signal sampling rate, transmission and storage is accordingly increasing. When the sampling rate reaches more than twice the signal bandwidth, the original signal can be accurately reconstructed by the sample results based on the traditional Nyquist sampling theorem. But the sampling rate is too high so that many sampling results must be compressed before the storage and transmission[1-2]. In order to make the storage and transmission of information more effectively, Donoho, Candès and Tao et al. proposed compressive sensing (CS) in 2006[3-7]. The theory points out that a small amount of non-adaptive linear measurements of sparse or compressible signals are sufficient to reconstruct the original signal, which breaks the linear Nyquist/Shannon sampling pattern[5,8]. Based on this theory, a sparse or compressed image can be recovered accurately through a sample set of sub-sampling data[5,8]. This means that according to this theory the number of samples can be much lower than the Nyquist/Shannon sampling limit[3,8].Since the compression sensing has the characteristics of sub-sampling, the CS theory has been widely used in optical/remote sensing imaging[9-10], image processing[11-15]and other fields immediately since it was proposed.

The theory of compressed sensing[16-17]is a theory of sampling and reconstruction of sparse signals. Candès and others have proved that as long as the signal in a certain orthogonal space is sparse, the frequency of sampled signal can be reduced, and the signal can be reconstructed with a high probability. The process of compression perception is divided into three steps. The first step is the sparse representation of the signal, that is, to find a sparse base so that the signal can be sparse by it. The second step is the selection of measurement matrix, that is, to select a measurement matrix which is not related to the sparse base so that it does not discard the original information when it is used to sub-sample of signal in the process of dimension reduction. The last step is the reconstruction of signal, that is, how to design an algorithm to recover the original signal rapidly from the sub-sampled data[18]. From these three steps, it is obvious that the compressed sensing theory contains three key factors: sparseness, non-correlation measurement and non-linear optimization reconstruction. The sparseness of signals is a priori condition of compressed sensing. The non-correlation measurement is the key to the compression perception, and the non-linear optimization is the means to reconstruct signals by compressed sensing.

The key to compression perception is the construction of a measurement matrix. At the beginning of perception, the measurement system is physically easy to implement and it has a smaller RIC with the CS information operator matrix formed by the representation system. The two core contents in the design of measurement matrix are observation waveform and sampling mode. The design is mainly based on the three principles of the optimal performance, the universality and practicability of the observation waveform. At present, the main sampling methods are uniform sampling and random sampling in the sampling way.

It can be concluded that constructing reasonable and effective measurement matrix is the core problem in the compression perception research. Baraniuk and others proposed a single-element camera model[19-20]based on the theory of CS, which uses a digital micro-mirror device (DMD) to perform random light modulation of the image, and finally converges on the point detector. In this paper, DMD is used to perform random light modulation of the image and then a generation and display system of measurement matrix based on DMD is designed. The use and performance of this system are illustrated and analyzed. In this system, the measurement matrix is a completely random measurement matrix, but it is difficult to be implemented on a traditional hardware module, since the storage capacity of matrix is large and the actual operation of it has the high complexity and other shortcomings[21]. This system transfers the generation of measurement matrix from hardware to algorithm, which weakens the high demand for hardware. There are a single-element camera and other models using DMD for random modulation of the image. However, this system can be applied to different models, so it has good expansibility. In addition, the DMD of this system can also be used as a spatial light modulator (SLM) to manipulate near-infrared light fast, accurately and efficiently, so that it can produce the desired image. And the system can be combined with a unit detector to replace the expensive InGaAs array detector, so as to obtain a high-performance, cost-effective and portable solution. It has a great potential for application.

1 Measurement Matrix Processing

1.1 Measurement matrix generation algorithm

1.2 DLP display images conversion

At last, the measurement matrix data is displayed in BMP format images. The images in the BMP format have the characteristics of uncompressed storage and preserve the integrity of real image information, so that it can speed up the real-time processing of images. The generation of images to be displayed is divided into two steps.

Step 1: to generate multiple 912×1 140 size of binary 0/1 measurement matrix data according to the algorithm;

Step 2: each 912×1 140 size of the binary matrix data generates 912×1 140 size of 1-bit grayscale information of the image (Fig.1).

Fig.1 912×1 140 size of BMP images which is 1-bit grayscale

When the DLP works, the image and DMD have a mapping relationship, that is, the image pixels are one-to-one mapped to the micro lenses. The DMD chip used is DLP4500FQE, which is named as that its micro lens structure is arranged in a diamond array (shown in Fig.2, which is from TI manuals). And its distinguishability is 912×1 140.

Fig.2 Structure diagram of DMD

2 System and Information Flow

2.1 Information flow

The generation and display system of measurement matrix based on DMD consists of hardware and software. The flow of information is the basis for interaction of hardware and software. And the logical structure of the flow of information is shown in Fig.3. The arrows represent the direction of the information flow. When it is used, the software part reads the matrix data to be displayed according to the user’s requirement. Then it generates the firmware and the corresponding pattern sequence to be displayed. At last, it sends the firmware and pattern sequence to the hardware through USB. Meanwhile, the hardware part receives and updates the firmware and pattern sequences which are used to initialize the matrix data source. And then, according to the start and stop instructions of the software part, the start and stop of the matrix data displayed on the DLP4500 are controlled on the hardware, accordingly, when the pattern sequences are displayed, the hardware running states feedback to the control software and display via USB.

Fig.3 Information flow of generation and display system of measurement matrix based on DMD

2.2 System structure

According to the requirement of the system information processing and information flow, the system is composed of software control module and matrix display module (shown in Fig.4). The software control module is divided into control unit, connection and transceiver unit. On the other hand, the matrix display module is divided into connection and transceiver unit, processing unit and display unit. Every division has a different task and a function, but the different parts work together to complete the generation and display function of the measurement matrix.

Fig.4 System structure of generation and display control system of measurement matrix based on DMD

In the system, the software control module and the matrix display module have two connection and transceiver units, which can complete the connection and communication between the software and hardware, as well as the instruction and data receiving and transmitting function. The control unit realizes the functions of reading the date of measuring matrix, generating and updating firmware, generating and transmitting the pattern sequences and displaying the running states information. The processing unit realizes the storage and updating of the firmware information, the acquisition and feedback of the running states and the extraction and transmission of measurement matrix to the display unit, and so on. The display unit is responsible for starting and stopping the measurement matrix’s display according to the measurement matrix or the control instruction transmitted by the processing unit.

3 Working Procedure

After the images are stored in the system, it is necessary to configure the system to display the measurement matrix images by internal triggers or external triggers. This system displays two modes of operation, video mode and Flash mode. In order to achieve high speed and high precision, this system chooses the Flash mode. This paper describes its workflow in detail, and each step of the work will read the status of equipment and feedback it to the control software.

Fig.5 is the working process of the software control module, the basic procedure is listed as follows:

① To determine whether the USB connection is successful;

② To select an operation mode, image display or firmware generation. If for image display, turn ③, if for firmware generation, turn ⑤;

③ To select the single-graph display or multi-graph display, and configure the display information;

④ To download the configuration information and select start/stop/pause instruction to control the display of images, and turn ⑦;

⑤ To generate the firmware based on the measurement matrix data;

⑥ To download the firmware information;

⑦ End.

Fig.5 Working process of software control module

Fig.6 is the working process of the matrix display module, the basic procedure is listed as follows:

① To determine whether the USB connection is successful;

② To receive firmware information or configuration information or instructions, if the firmware information, turn ③, if the information is configured, turn ④, and if it’s instruction, turn ⑤;

③ To update firmware information, and then turn ⑥;

④ To update configuration information, and then turn ⑥;

⑤ To execute the start/stop/pause instructions to control the displays of measurement matrix;

⑥ To determine whether the USB channel is disconnected, if broken, the operation is over. Otherwise, repeat ②-⑤ until the end of operation or the USB channel is disconnected.

Fig.6 Working process of matrix display module

Fig.7 is the configuration process of measurement matrix display which takes a multi-graph display as an example. And the basic procedure is as follows:

① To set the mode of multi-graph display;

② To set the mode of trigger control;

③ To set the number of matrix displayed, exposure time, bit depth of image, and so on;

④ To save and download the configuration information.

Fig.7 Configuration process of measurement matrix display

In this system, the generation process of firmware is shown in Fig.8. And the basic procedure is shown as follows:

Fig.8 Process of generation firmware

① To open the source firmware and add the firmware tag;

② To open the binary files of measurement matrix;

Fig.9 Interface of firmware generation and update

③ To set the display area;

④ To add the 24-bit depth BMP images which generated by the measurement matrix;

⑤ To generate, download and update the new firmware.

4 Development and Implementation

4.1 Software environment

The generation and display system of measurement matrix based on DMD is divided into software control module and matrix display module. The development environment of the software control module is QT5.7.0, and the programming language is C++. The matrix display module adopts the DLPC350 control code which is packaged by TI (Texas Instruments), and it is not open source.

Fig.10 Interface of configuration information generation and download

Fig.11 Control interface of Start/Pause/Stop

Fig.9 is the generation and updating firmware interface of software control module. It completes the conversion of measurement matrix to firmware, and generates, downloads and updates new firmware. Fig.10 is the interface of configuration information generation and download, which completes the configuration information settings during the matrix display. When the measurement matrix is available, the control interface of start/pause/stop is shown in Fig.11, which complete the display control of the measurement matrix.

4.2 Main control programs

4.2.1USB connection program

This program is the key part of communication procedure. When the program starts running, the software control module and matrix display module will connect with each other via USB. After the USB channel is established, all of the new matrix firmware, configuration information and control instructions will be transmitted form the software control module to the matrix display module through the USB channel.

4.2.2BMP images generation program

This program is the core part of this system, and it is responsible for converting the measurement matrix into BMP images that can be displayed on the DMD. When the firmware production is performed, the binary file of measurement matrix is selected and the measurement matrix with 912×1 140 size is converted into 1-bit depth BMP images. And then 24 1-bit depth BMP images are turned into a 24-bit BMP image. At last, the 24-bit depth BMP images are added into the source firmware, and the measurement matrix can be displayed in the way of BMP images.

4.2.3Display configuration program

This program is responsible for setting the measurement matrix display time, trigger modes and other information. When it is in the operation of single or multi displays, it should be configured as the display mode. It is necessary to set the display time, the number of matrix which displayed, trigger mode and the flipping mode. Once the configuration is completed, the configuration information will be send to the matrix display module to make the display patterns effective. And then the existing firmware in the matrix display module will be displayed according to the display patterns.

4.3 Display example

4.3.1DLP display images with RGB

The images need to be displayed by the loading of the processing unit. And the DMD chip only reads a 24-bit RGB image at a time. So it needs to deal with the 1-bit depth grayscale image to compound 24-bit depth image with BMP format. In a 24-bit RGB full-color BMP format image, there are three channels with the 8-bit, 256-level grayscale map. Every 24-bit BMP image can store 24 grayscale map of 1-bit depth. And it places 24 grayscale images of 1-bit depth into each of the three channels of R, G, and B in turn. If the number of 1-bit depth grayscale images is not the integer multiple of 24, then the final position of less than 24 place into the all black 1-bit depth BMP image. When a measurement matrix needs to be displayed, it can be displayed by selecting the index number of the 24-bit RGB images in which it is stored, and selecting a channel number in the RGB diagram. The two images of 12 each shown in Fig.3 synthesized a 24-bit BMP image shown in Fig.12.

4.3.2DMD display example

The matrix display module is one of the more important hardware modules. And Fig.13 is the matrix display module, and it completes the display function of measurement matrix. Fig.14 shows the display of the measurement matrix of “ke” and “yuan”.

Fig.12 12 ABC and 12 OMG synthesize a 24-bit depth BMP image

Fig.13 Matrix display module

Fig.14 Display of the measurement matrix of “ke” and “yuan”

5 Conclusion and Future

In this paper, the generation and display system of measurement matrix based on DMD is designed, and the use and performance of this system are described and analyzed. The measurement matrix implemented by this system is a completely random measurement matrix. The characteristics of such a matrix are illustrated as follows. On the one hand, the matrix elements are independent and obey the same random distribution. It is not related to the most of sparse signals, and the reconstructed signals have great accuracy. And the random measurement matrix which generated in the control unit reduces the demands for hardware and overcomes the difficulties of hardware implementation. On the other hand, it can set up the display number of the measurement matrix, the mode of display and display time according to the users’ requirements. Because the display information can be custom-built, it has a better adaptability.

In addition the measurement matrix of this system not only has better randomness, but also has good expansibility. One of these extensions is to embed this generation and display system into more application models such as the use of near infrared light. And the other extension is that this system can control multiple DMDs to work at the same time and the using efficiency of measurement matrix can be improved. Thus, the system has excellent expansibility. It can be seen that the generation and display system of measurement matrix based on DMD has good adaptability, is easy to extend and has great application value.

Acknowledgement

We would like to thank National Space Science Center,CAS and University of Science and Technology of China for providing us with the experimental facilities and equipments.