Contactless Human Sensing with MIMO Radar and End-to-End Deep Learning

Contactless human sensing encompasses the collection of health or physiological informatics provided by reflections from the surface of the human body. Enabling contactless collection of information such as respiratory and pulse waveform reconstructions and precise body landmark tracking is important for many critical applications, such as long-term patient health monitoring and rehabilitation-targeted gait analysis. There has been increasing interest in using millimeter-wave multiple-input-multiple-output (MIMO) frequency-modulated continuous-wave (FMCW) radar for this task due to its robustness, compact size, privacy-preserving features, and ability to discern reflections with respect to both range and angle. However, due to low signal-to-noise ratios and limited numbers of transmit-receive elements, state-of-the-art solutions that rely on complex traditional radar signal processing chains tend to be more susceptible to real-world non-idealities. In this work, an end-to-end deep learning framework is developed to operate on unprocessed in-phase and quadrature MIMO-FMCW radar data and output signals of interest for several applications. By doing this, unreliable and computationally expensive traditional preprocessing steps such as beamforming, subject localization, clustering, arctangent demodulation, and filtering are replaced with a convolutional neural network-based encoder-decoder neural network. For the first time, accurate multi-subject respiration waveform reconstruction, precise upper-body 3D pose estimation and respiration reconstruction of exercising subjects, and direct ECG waveform reconstruction are achieved using variations of the developed methodology.