,2

(1.College of Physics and Information Engineering,Fuzhou University,Fuzhou 350108,China;2.Department of Electrical and Computer Engineering,Dalhousie University,Halifax B3H 4R2,Canada)

Abstract: A review of signal processing algorithms employing Wi-Fi signals for position?ing and recognition of human activities is presented.The principles of how channel state information (CSI) is used and how the Wi-Fi sensing systems operate are reviewed.It pro?vides a brief introduction to the algorithms that perform signal processing,feature extrac?tion and recognitions,including location,activity recognition,physiological signal detec?tion and personal identification.Challenges and future trends of Wi-Fi sensing are also discussed in the end.

Keywords: Wi-Fi sensing; channel state information; signal processing; classifications; fea?ture extraction;positioning;location;recognitions

1 Introduction

With the fast progress and development of artificial intelligence,smart environment sensing and detec?tion has attracted much attention recently,since it provides indispensable information and data for ar?tificial intelligence.Sensing and detecting humans and their activities is an important branch and a vital component of many artificial intelligence applications[1].

Device-free wireless sensing using electromagnetic signals is an emerging technology with its advantages over other wired or non-electromagnetic sensing technologies,such as good penetration ability,relatively unrestricted availability and ac?cessibility in space and time.With the increasing popularity and prevalent uses of Wi-Fi devices,Wi-Fi signals,occupying a part of the electromagnetic spectrum,are available in almost any place where humans live and work.As a result,it can be used for sensing and detecting humans and their activities,such as positioning,activity detection,gesture recognition and personal identification[2].

Recent developments have shown that the use of the chan?nel state information (CSI) of Wi-Fi signals can be a viable method for human sensing and detection,in addition to the well-known received signal strength indication (RSSI) meth?od.A time series of CSI contains the information of how wire?less signals travel through and around objects and humans，and the information is embedded in time,frequency and space[3].In other words,different human beings and their dif?ferent activities can be sensed,detected,and recognized through analyzing the amplitude and phase of CSI.As a re?sult,various signal processing and recognition algorithms have been developed to process CSI and relate it to human activities.This paper presents an overview of the algorithms and applications of human sensing with Wi-Fi signals that have been reported so far.

The outline of this paper is as follows.In section 2,the prin?ciple and the basic architecture of a Wi-Fi sensing system are introduced.In section 3,Wi-Fi sensing applications and the relevant technologies are presented.In section 4,the chal?lenge and future trend of Wi-Fi sensing are discussed.In sec?tion 5,we conclude the paper.

2 Background

2.1 Channel State Information

A Wi-Fi channel with multiple-input multiple-output (MI?MO) usually employs the orthogonal frequency division multi?plexing (OFDM) scheme that uses multiple subcarriers to form multiple links between transmitters and receivers.It can pro?vide CSI for each transmission link with its carrier frequency.CSI characterizes how wireless signals propagate from the transmitter to the receiver along multiple paths.When a per?son appears in a channel and acts,such as walking,drinking,waving or even just breathing,CSI will change accordingly,which presents how wireless signals travel through and how they are affected by the body shape or the movement of a per?son.As a result,information about a person may be extracted and captured from the CSI.

To detect the CSI and its changes,the IEEE 802.11n CSI Tool[4]has been developed and widely used through Intel 5 300 Wi-Fi cards.One set of CSI can be obtained from each re?ceived data packet,and it represents the amplitude and the phase of each OFDM subcarrier,as shown in Eq.(1).

where ‖H(k) ‖and ∠H(k) are,respectively,the amplitude and the phase of thek-th subcarrier.Both amplitude‖H(k)‖and phase ∠H(k) are affected by the body shape or the move?ment of a person.The CSI can be measured and therefore pro?cessed for information extraction,forming the basic principle of the Wi-Fi sensing and detection.

2.2 Wi-Fi CSI Processing System

A general processing flowchart of a Wi-Fi CSI sensing sys?tem is depicted in Fig.1.CSI measurements are first collect?ed by the receivers and then sent to a processor or a computer for extracting the CSI amplitude,phase and Doppler frequen?cy shift (DFS) that are related to the environments.First,sig?nal segmentation and noise elimination are applied to obtain clean and usable signals,which are conducted with Hampel filter,wavelet transformation,moving averaging and median filters.Differential techniques and linear regression are also used to remove raw CSI phase offsets.Next,the signals are processed to obtain key features and classifications in either one of the following two manners:(1) statistical and spectrum feature extractions first and then classifications with machine learning algorithms such as support vector machines (SVM)[5],k-nearest neighbors (KNN)[6]or Bayes[7]; (2) simultaneous fea?ture extractions and classifications directly with all-inclusive machine learning algorithms such as the convolutional neural network (CNN)[8],recurrent neural network (RNN)[9],and long short-term memory (LSTM)[10].They are based on deep learn?ing and have been developed in recent years.

3 Algorithms and Applications

3.1 Wi-Fi Positioning(or Locating)

Wi-Fi sensing technology was first used for positioning ap?plications in 2000[11].It can be categorized into two types:the fingerprint-based method and the model-based method.With the fingerprint-based method,target locations are determined with radio signal fingerprints constructed by RSSI or CSI.The position of an object is estimated with the fingerprint database which is acquired beforehand.Developing and updating the fingerprint database is,however,time-consuming and labor-in?tensive.In the model based methods,Wi-Fi signal propaga?tion models are developed and used to estimate distances be?tween the target and the known access points,and triangula?tion is applied for positioning.The accuracy of the propaga?tion model parameters directly affects positioning accuracy.

▲Figure 1.Processing flow chart of a Wi-Fi sensing system.

The propagation models include angle of arrival (AoA),time of flight (ToF) and DFS.However,the accuracy still needs to be improved,especially in complex indoor multi-path environ?ment,in spite that some work has been claimed to achieve decimeter-level or even centimetre-level positioning accuracy.To increase the accuracy and resolution of AoA estimation,SpotFi[12]utilizes OFDM subcarriers as sensors and applies the standard multiple signal classification (MUSIC) algorithm[13]to the constructed measurement matrix to estimate both the AoA and ToF parameters of multiple signal paths.The joint AoA and ToF estimations are made from the same signal path but in different data packets.Gaussian mean clustering algorithm is used to process clustered signals and associated parameters first,and then the highest likelihood metric is used to identify the direct path.The target location is determined based on the best match of the estimated AoA value of the direct path and its RSSI.SpotFi achieves a median location accuracy of 40 cm in a multipath-rich indoor environment.In general,AoA just provides the arrival direction of the signal,not the distance of a target.Therefore,AOAs measured from multiple access points are used,and interactions of their projection lines give the locations of a target.Three to four access points are used in SpotFi for a home environment.

In Ref.[14],a system that uses a single Wi-Fi access point,is proposed to locate targets to within tens of centimetres.By eliminating packet detection delay,resolving phase offsets,and mitigating multi-path,it can compute the absolute ToF us?ing commodity Wi-Fi cards at sub-nanosecond accuracy.In Ref.[15],by quantifying the relationships between CSI and a target’s location and velocity,a theoretical model is devel?oped and has achieved the decimeter-level passive tracking.It extracts the path length change rate (PLCR) from noisy CSI measurements and derives moving velocity from PLCR through at least two links,as shown in Fig.2.At least two links are required to determine the moving direction of a tar?get,and the distance between the receivers needs to be sepa?rated by a certain distance.In Ref.[16],joint estimations of AoA,ToF and DFS are exploited to achieve passive human lo?cation and tracking using a single link of Wi-Fi devices.Re?cent research efforts focused more on the use of multi-model sensing data such as computer vision to achieve positioning and tracking of multiple people at high accuracy[17].

3.2 Wi-Fi Recognitions of Human Activities

As mentioned before,the information on how wireless sig?nals propagate from a transmitter to a receiver at carrier fre?quencies along multiple paths is contained in CSI.The CSI amplitude and phase vary differently with different human ac?tivities,and they can be used for human behavior recognition.Many activity recognition algorithms have been developed,and they usually include three parts:signal preprocessing,fea?ture extraction and classification,as shown in Fig.1.

▲Figure 2.Illustration of the Widar tracking results presented in Ref.[15].

Figs.3a and 3b show the typical signals before and after the signal preprocessing with Butterworth filter and wavelet transformation.As can be seen in Fig.3b,noises are mitigated or even removed.After the noise mitigation,statistical fea?tures such as average,variance and energy are extracted for classification in activity recognition,as shown in Figs.3c and 3d[18].Deep learning methods based on neural networks are al?so used in activity recognition in recent years[19-20].In the field of image recognition,many deep learning models have already been developed.These models can be modified and used in Wi-Fi activity recognition by transforming 1D CSI signals to 2D images.

▲Figure 3.Typical signal variations and features of activity recognitions.

The main challenge in the activity recognition is that the ac?curacy of multiple-people recognition is not high enough.Re?cently,we have proposed a new algorithm to overcome the multiple-people recognition problem.A multi-dimensional fea?ture configuration to make full use of CSI information,includ?ing spatial correlation and temporal correlation is developed.The experimental results verify the robustness of our algo?rithm in multiple scenarios.

3.3 Wi-Fi Extraction of Physiological Signals

When a person is stationary (such as standing,sitting and sleeping),a Wi-Fi sensing system can sense his respiration and heart rate through measurements of the CSI.In Ref.[21],the Fresnel model is introduced to investigate the impact of respiration on the wireless signals and the theoretical model is developed to relate respiration depth,heart rate and positions to signal patterns (or classifications).Using the Fresnel model,location-dependent processing algorithms can be applied,which in turn helps improve the sensing accuracy.

The existing respiration estimation techniques are mainly applicable to the condition that a person is not moving.When a person moves,information extractions from weak heart rate and breathing are challenging since signal variations due to the movements can overwhelm the signals desired.We have researched breathing estimation of a person in the standing or sitting positions with body swing.Since different subcarriers have different sensitivities to the movements in different direc?tions,we select the subcarrier most sensitive to breathing.To do so,the recurrence plot (RP) method is used to analyze the periodicity of CSI measurements of each subcarrier,as shown in Fig.4.A peak detection algorithm is developed in the fre?quency-time domain to extract the respiration rate.Our algo?rithm can eliminate the disturbance of walking to the CSI mea?surements and extract human respiration rate and heart rate during walking,as shown in Fig.5.

3.4 Wi-Fi Personal Identification

Personal identification is one of the most convenient and important applications with Wi-Fi signals.Since gestures and gaits of a person are unique,they can be used to identify peo?ple.Many algorithms have been developed so far.

WiWho is developed to identify a person from a small group of people over a short walking distance of 2–3 meters on a straight-line path[22].It extracts steps and walking gaits of a person from CSI.Step and walking segment features are com?bined to develop a complete gait pattern profile and a deci?sion-tree-based machine learning classifier is then developed and used to complete the personal identification.However,WiWho can only be used in the single-person situation in cor?ridors or hallways.In addition,it has been tested only for the age group of 25–30 years old,while gaits of different ages are different.

▲Figure 4.Recurrence plot of different subcarriers.

The WiFi-ID system proposed in Ref.[23]analyses CSI,ex?tracts unique features of walking gaits and achieves the identi?fication of average accuracy of 93% to 77% for a group of 2 to 6 persons.In Ref.[24],the authors present a WFID system,propose a novel feature of subcarrier-amplitude frequency(SAF),and extract it from the subcarriers using the SVM algo?rithm,which achieves identification with the accuracies of 93.1% and 91.9% for the group of 6 and 9 people in two in?door scenarios,respectively.

Although WiFi-ID and WFID can recognize multiple per?sons,they are applicable to the situations where the walking trajectory passes through the line of sight (LOS) path between the transceivers or is in a straight line.Such a restriction makes the two algorithms not suitable for general home or of?fice environments

▲Figure 5.Respiration and heartbeat rate sensing of human in walking.

In Ref.[25],a system called WifiU is developed to detect the gaits of a person.It exploits signal processing technology to make spectrograms from CSI measurements and extract gait patterns that include walking speed,gait cycle time and footstep lengths.An SVM classifier is developed and used to achieve identification.The limitation of WifiU is that the per?son must walk on a predefined path in a predefined walking direction.Training for a new classification model is needed for a different walking path and directions.

In addition to the above problems or limitations,many oth?er factors may also affect recognition accuracies,such as clothes and shoes.Furthermore,temporary gait pattern changes of a person due to the carrying of a heavy backpack or a heavy object,or injury,may result in recognition fail?ure.Therefore,it is necessary to develop robust means to achieve effective and accurate identification,including com?binations of gait and physiological recognition.We are de?veloping a system,named Wi-GAH,which uses gestures to achieve personal identification.Wi-GAH can not only iden?tify persons based on gestures but also recognize the mean?ings of the gestures at the same time so that more sensing da?ta are available for automatic system adaption to the varying home environments.

4 Challenges and Future Trends of Wi-Fi Sensing

Most of the Wi-Fi based sensing systems presented so far assume a single person scenario.Sensing in multi-people sce?narios faces many challenges,such as how to determine the number of people in a group and how to distinguish among dif?ferent people.New models and new algorithms need to be de?veloped to mitigate the shadowing effects,especially when multiple people are close to each other[26].The multi-path propagation models that can quantitatively correlate CSI dy?namics and multi-person activities are also needed[27].

Furthermore,it is very challenging to deal with changes of the environments and to develop a universal Wi-Fi sensing system that can automatically adapt to the new environments.The CSI amplitude,CSI phase and DFS are very much affect?ed by the environments[28].When an environment changes,for example the distance and direction between the target and Wi-Fi transceiver alter,the accuracy of Wi-Fi sensing is affected.With the existing algorithms,new data are needed for training to ensure accuracy every time when an environment is changed.Along this line,body-coordinate velocity profile(BVP) is proposed for gesture recognition[29],which appears to be a potentially feasible solution.The emerging deep-learningbased methods,such as cross-domain methods and transfer learning techniques,show some potentials too.Deep similarity evaluation networks and deep generative adversarial networks(GAN) reduce training efforts significantly[30].Cross-device and cross-sensor methods are also considered helpful.Finally,Wi-Fi based sensing can be integrated with 5G based mobile communication networks to provide more comprehensive data and services for the sensing processing in the industries such as smart museums,smart hospitals,smart shopping malls,and smart factories.

5 Conclusions

In this paper,we discuss Wi-Fi sensing technologies in in?door multi-path environments and present the principle and basic methods of Wi-Fi sensing.The Wi-Fi sensing applica?tions and the relevant algorithms in location,activity recogni?tion,physiological signal extraction and personal identifica?tion are briefly described.A few classic sensing algorithms are discussed and analyzed in detail.In the end,we present the challenges in multi-people sensing and cross-scenario sensing.Future trends of the research and development in the area are briefly discussed.