Similar articles

Abstract: In this paper, a novel criterion is proposed to determine the retained principal components (PCs) that capture the dominant variability of online monitored data. The variations of PCs were calculated according to their mean and covariance changes between the modeling sample and the online monitored data. The retained PCs containing dominant variations were selected and defined as correlative PCs (CPCs). The new Hotelling’s T ² statistic based on CPCs was then employed to monitor the process. Case studies on the simulated continuous stirred tank reactor and the well-known tennessee Eastman process demonstrated the feasibility and effectiveness of the CPCs-based fault detection methods.

1.  Introduction

 Multivariate statistical methods, such as principal component analysis (PCA) and partial least squares (PLS), are widely used in industry for process monitoring (Nomikos and MacGregor, 1995; Qin, 2003; Ge and Song, 2008; Garcia-Alvarez et al., 2012). Other complementary multivariate statistical process monitoring methods, including canonical variate analysis, kernel PCA, dynamic PCA, and independent component analysis, have been proposed to address the limitations of PCA- or PLS-based monitoring strategies (Russell et al., 2000; Juricek et al., 2004; Lee et al., 2004a; 2006). PCA-based and related monitoring methods, which build statistical models from normal operation data and partition the measurements into a principal component subspace (PCS) and a residual subspace (RS), are among the most widely used multivariate statistical methods. In this methodology, the dimension of the PCA model, which is the estimation of the optimal number of principal components (PCs) to retain, must be determined and has an important role on the process monitoring performance. However, the determination of the number of PCs is not unique, given that the sensor outputs are generally disturbed by noise (Tamura and Tsujita, 2007).

 The choice of PCs is a crucial step for the interpretation of monitoring results or subsequent analysis because it could lead to the loss of important information or the inclusion of undesirable interference. To tackle this challenge, a large number of well-known techniques for selecting the number of PCs have been proposed. A simple approach is to choose the number of PCs for the variance to achieve a predetermined percentage, such as 85%, termed as cumulative percent variance (CPV) (Jackson, 1991). Other methods for determination, including cross validation, average eigenvalue approach, variance of reconstruction error (VRE) criterion, and fault signal-to-noise ratio (fault SNR), have been proposed to determine the number of the retained PCs (Wold, 1978; Dunia and Qin, 1998; Valle et al., 1999; Tamura and Tsujita, 2007). The cross validation method uses part of the training samples for the model construction, while the rest are compared with the prediction by the model. However, this criterion needs to build multi-PCA models, which is tedious. The average eigenvalue approach accepts all the eigenvalues with values of above the average eigenvalue and rejects those below the average. This criterion is simple, but comparatively robust. The VRE criterion assumes that the corresponding PCs are deemed as the optimal PCs when the error of fault reconstruction comes to minimum. Fault SNR shows the relationship between the number of PCs and the fault detection sensitivity, which maximizes the sensitivity of fault detection. When a priori knowledge of the fault direction is available, both methods can determine the optimal number of PCs for the fault detection from different perspectives.

 Several comparative studies have been conducted on these methods for determining the number of PCs for the fault detection. Valle et al. (1999) tested 11 methods and concluded that the VRE criterion is preferable. Tamura and Tsujita (2007) compared fault SNR with VRE criterion, and concluded that these two methods calculate different characteristics depending on the analysis objective. In addition, both methods need a fault direction vector as a priori process knowledge, which is hardly obtained in practical applications, especially for complex chemical processes. Nevertheless, the determination of the retained PCs of these traditional approaches is rather subjective or requires a priori knowledge, and none of these methods considers the performance of the fault detection in the absence of a priori knowledge. Apart from these comparative studies, the fault detection ability depends on the PCs retained in the PCA model (Kano et al., 2002). Togkalidou et al. (2001) indicated that including components with smaller eigenvalues in the PCA model and excluding those with larger eigenvalues can improve the prediction quality. Motivated by this perspective, the present work deals with several of the limitations inherently associated with most of the traditional criterions for determining the retained PCs.

 When the PCA model is employed for monitoring industrial processes, Hotelling’s T ² and squared prediction error (squared prediction error (SPE) or Q) statistics are usually used for the fault detection. T ² index is the squared Mahalanobis distance of the projections in PCS, designed to measure the variability of the mean and covariance within the PCS. Given each PC is the linear combination of original data, the variability of the original data may be submerged in several projection directions with larger variance considering small coefficients corresponding to the measured variables. In the following section, the monitoring components with relatively small eigenvalues have been shown to result in a better fault detection performance. Similar to the statement presented by Togkalidou et al. (2001), the basis of the proposed method is that the first l PCs that capture the dominant variance of the modeling data may not best reflect the variability of the online monitored data samples, and thus a number of special faults cannot be well detected.

 In this work, a novel criterion is introduced to determine the retained PCs objectively and dynamically. This method differs from the traditional approaches mentioned above, in which the selected PCs can capture the dominant variability (mainly the changing of mean and covariance) of the online observations against the modeling data, instead of capturing cumulative variance. The online measurements lead to variation within PCS, and the degree of variability captured by each component can be evaluated according to a predefined criterion through a moving-window technique. The PCs that contain the dominant variability is selected for the fault detection, which is termed as correlative PCs (CPCs) in this paper. With the embedding of the moving window, no prior knowledge of the abnormal situations is needed, and the selected components can expose the variability of the current samples to maximum. The contributions of the proposed method are as follows.

1. Unlike the traditional criterion, the proposed approach determines the retained PCs objectively, without the restriction that the components corresponding to larger eigenvalues should be selected. CPCs are similar to PCs containing the dominant variance of the modeling data when the current samples are normal. For potential abnormal observations, the CPCs can be automatically determined to be those that capture the main variability. Therefore, the PCA model parameter that accounts for the fault detection can be chosen, and the monitoring performance can subsequently be improved.

2. Although fault SNR and VRE criterion can determine the number of PCs objectively, a prior knowledge of the fault direction is required, which may induce redundant information for the fault detection inevitably. In the proposed method, prior knowledge is not necessary and CPCs can be determined dynamically according to the variations caused by the current window.

3. Compared with the conventional PCA-based method, SPE statistic is eliminated in the improved PCA-based approach considering that the variability of online observations is mainly contained in CPCs and the one-fold T ² index is enough for monitoring the industrial processes. Moreover, a single index can provide the operators more precise information without any confusion.

2.  PCA-based process monitoring

2.1.  Preliminaries of PCA

 One of the most popular methods for dealing with multivariate data is PCA, which transforms a correlated original data to uncorrelated data set, while preserving the most important information of the original data set (Jackson, 1991). PCA includes the decomposition of data matrix X ∈ which contains n regular-sampled observations of m process variables and is scaled to zero mean and unit variance, into a transformed subspace of reduced dimension. The decomposition is expressed as follows: (1) , where X ∈ and P ∈ are the score matrix and the loading matrix, respectively. The matrices and E represent the estimation of X and the residual part of the PCA model, respectively, which are defined as follows: (2) , (3) .

 The principal component projection reduces the original set of variables to l PCs. The decomposition assumes that is orthonormal and is orthogonal. The columns of P are actually the eigenvectors of the covariance or correlation matrix, R , associated with m eigenvalues [λ ₁, λ ₂, …, λ_m ] (the eigenvalues are listed in descending order). Alternatively, the matrix X can be decomposed using singular value decomposition to build the PCA model. The number of PCs, l, is a key parameter in the PCA model and is commonly determined using the CPV method, cross validation, and average eigenvalue approach, as mentioned above.

 The procedures of the conventional PCA to perform process monitoring are introduced in this subsection. A new sample vector x (after scaling with the mean and variance obtained from the normal data) becomes available and is projected with the help of the PCA model to either PCS or RS. Two statistics, namely, Hotelling’s T ² and SPE (or Q), have been developed for process monitoring (Chiang et al., 2001). T ² is the Mahalanobis distance of a score vector, t , in the PCS, and Q is the Euclidean distance of a residual vector in the RS, which are given by (4) , (5) , where Λ =diag{λ ₁, λ ₂, …, λ_l }, e is the residual vector, and I ∈ denotes identity matrix. Therefore, the scalar thresholds can qualify the process status. The approximated control limits of T ² and Q statistics, with a confidence level α, can be determined from the normal operating data in several ways by applying the probability distribution assumptions (Kourti and MacGregor, 1995; Qin, 2003). The control limits can be calculated as follows: (6) , (7) , where F_α (l, n−l) is the upper limit of α percentile of the F-distribution with the degree of freedoms l and n−1, c_α is the normal distribution value with the level of significance α, and (8) , (9) .

 In general, T ² statistic measures the significant variability within the PCs, while Q indicates how well the reduced dimensional PCA model can describe the significant process variation. In most cases, Q is considered as an assistant statistic for T ² considering that the value of Q can be significantly affected by the process noise. The process condition is considered abnormal if the statistics of a new observation exceed the control limit.

2.2.  Motivation analysis

 In this subsection, a simple multivariate process (Lee et al., 2004b), which is a modified version of the system suggested by Ku et al. (1995), is considered to illustrate the motivation analyses: (10) , (11) , where u is a correlated input expressed as (12) .

 The input w is a random vector, in which each element is uniformly distributed on the interval (−2, 2). The output y is equal to z plus a random noise, v . Each element of v has zero mean and 0.1 variance. Both input u and output y are measured, but not z and w . The basic data vector for the monitoring at step k consists of x (k)=[u ₁, u ₂, y ₁, y ₂, y ₃]^T. The following abnormal case is considered to reveal the defect of the traditional criterions.

 In PCA, a three-principal component model has been developed on 300 normal samples, which captured about 87% of the process variance. For testing purpose, a step change of 2 is introduced to w ₁ starting from sample 100 to the end of operating duration. The T ² and Q charts of the process with this disturbance are shown in Fig. 1, as well as the 99% confidence limits. However, PCA cannot detect the disturbance and captures only the dominant randomness. By contrast, the occurrence of abnormality is mainly captured by the first and the fifth PCs based on the plots of the five PCs of the online samples given in Fig. 2. With the inclusion of unnecessary information and the elimination of several components corresponding to relatively small eigenvalues, the T ² and Q derived from the first three PCs did not show considerable departure from the normal status. Therefore, fault detection must be considered in determining the retained PCs.

3.  CPCs for the fault detection

 Chemical process is often characterized by large scale and process data projected onto the loading matrix often marked with irrelevant information for the fault detection. Although the T ² statistic of the first l PCs can show deviation from the normal values in PCS, it still suffers from irrelevant PCs and low computational efficiency. Thus, selecting several PCs that indicate significant variation of the online observations in reduced subspace can improve the monitoring performance of the conventional PCA method. The selected PC is defined as CPCs, as mentioned above. The definition of CPCs is presented in the following subsection.

3.1.  Definition of CPCs

 For Hotelling’s T ², the variations of the mean and the covariance matrix are generally two essential factors that represent the process changes from a normal situation to a situation with several faults (Kresta et al., 1991). Each PC’s contribution to the T ² index comes from two parts, namely, the mean or variance change of the PC itself and the relationship with other PCs responsible for the process change. Recently, cumulative percent variation (He et al., 2009) based on each variable’s equivalent variation has been proposed to determine the candidate variables as follows: (13) , where x_j (i) is the ith element of the jth observation, b ₀(i) is the ith element of the normal mean vector b ₀, δ ₀(i) is the standard deviation of the ith variables of the normal data set X ₀, and r _i is the ith column of the correlation matrix of the fault data set X . The index shows that each variable’s maximum variation comes from the mean or variance change of either the variable itself or the other variables related with the variable.

 In this work, a similar index is defined to calculate the maximum variation of ith PC for online samples. In addition, a moving window is adopted to update the current data matrix, that is, the newest sample was augmented to the data matrix, whereas the oldest sample is discarded to keep a fixed number of samples in the data matrix. Let the kth data matrix with window length d be X =[ x _{k −d+1}, x _{k −d+2}, …, x _k ]^T∈, and then T _k =[ t _{k −d+1}, t _{k −d+2}, …, t _k ]^T∈ is obtained by projecting X _k onto the loading matrix P . For normal modeling data set X ₀, the corresponding score matrix T ₀=[ t ₁, t ₂, …, t _d ]^T is used as reference matrix. The following index has been adopted to calculate each PC’s contribution to the process change: (14) , where t_j (i) is the ith element of the jth score vector in T _k , and is the standard deviation of the ith PCs of X ₀, which is equal to the ith eigenvalue. r _i is the ith column of the correlation matrix, R , of T _k . S ₀ and S _k are the covariance matrices of T ₀ and T _k , respectively, which are given by (15) , (16) . S _k utilizes the mean vector b ₀ of the reference score matrix T ₀, which is a zero vector, and indicates the change in the covariance matrix more clearly. ( S _k − S ₀) _i is the ith column of ( S _k − S ₀). Eq. (14) has two parts. The first part shows the mean change and the mean change induced by other variables. The other part represents the equivalent covariance change. After calculating all the PCs’ maximum variations, they are ranked in a decreasing order. The first several PCs sufficient to express the dominant change information of the current data matrix from the normal data set are determined as CPCs. In this step, the CPCs can be obtained by calculating the cumulative percent variation as follows: (17) . Cumulative percent variation is a measurement of the percent variation captured by the first l ordered PCs, similar to CPV. We use D to denote the cumulative percent variation to avoid the confusion of this index with CPV. These two indices calculate different characteristics depending on the analysis objective. The first l PCs are defined as CPCs. The number of CPCs can be determined when D reaches a predetermined limit η, and we suggest that 65%≤η ≤70%. Moreover, when the data set X _k contains no abnormal sample data and the PCs have insignificant mean or variance change, they share similar variation along all PCs.

3.2.  Process monitoring procedures

 Given that the CPCs are determined by computing the correlation and covariance matrices of the score matrix of the current data set X _k , their adaptation suffer from low computational efficiency. Wang et al. (2005) proposed a fast-moving-window algorithm for adaptive process monitoring, which incorporates the adaptation technique in a recursive PCA algorithm (Li et al., 2000) and performs through a two-step procedure to calculate the correlative matrix efficiently. In this study, the same idea is introduced to update the correlation and covariance matrices of the current score matrix to increase computational efficiency. The full details of the calculation are described in the Appendix.

 A real-time monitoring scheme can be implemented efficiently using the presented two-step algorithms. First, a sufficient large number of normal observations are necessary to obtain the PCA model. When the initial size of online observations reaches the prescribed window length, they are projected onto the loading matrix P to obtain the score matrix. ΔT(i) is then calculated for each PC, and the CPCs for the process monitoring are determined using the index D. Subsequently, the time-window is moved to the next step when a new sample is measured. The fast algorithm is applied to update the covariance and correlative matrices efficiently, and the CPCs for the next data matrix are determined. The consistent threshold for monitoring statistics is prescribed according to the number of CPCs for each time-window. The monitoring statistic used is Hotelling’s T ² statistic. Given that all the PCs of the significant variation are incorporated into T ² and the remainders have little deviation from the normal data, calculating the SPE statistic is unnecessary.

 The complete monitoring procedures of the improved PCA with CPCs are summarized as follows.

1. Off-line modeling:

   (1) Sufficient data are acquired when a process is operated under a normal condition. Each column (variables) of the data matrix is normalized, i.e., scaled to zero mean and unit variance.

   (2) PCA is applied to the data matrix, and the loading matrix P ∈ and the eigenvalue matrix D =diag{λ ₁, λ ₂, …, λ_m } are obtained.

   (3) The size of time-window, d, and the threshold of cumulative percent variation, η are determined.

2. Online monitoring:

   When the online observations become available, the monitoring task is initiated after the number of the new process data reaches d. The CPCs for the first time-window are used for monitoring all of the d observations, whereas the CPCs for the subsequent data matrix are only used for monitoring the new added data point.

   (1) For online monitoring, the data matrix X _k representing the current operating conditions is updated by moving the time-window step by step and is scaled based on the mean and the variance obtained at Step 1. Subsequently, they are projected onto P to obtain the score matrix T _k , and ΔT(i) is computed for each PC using Eq. (14).

   (2) The values in the vector Δ T are ranked, and the first l PCs that capture the dominant variability of the current score matrix T _k are set as the CPCs according to Eq. (17). The Hotelling’s T ² statistic of the CPCs is computed. The control limit is then calculated according to the number of CPCs dynamically.

   (3) If the T ² statistic is under the control limit, the process is judged as normal and the time-window is moved to the next step. Conversely, if the T ² statistic is outside the control limit, the new added data point is considered as abnormal, and the fault diagnosis method is used to analyze the fault roots.

4.  Case study

4.1.  Continuous stirred tank reactor (CSTR) simulation

 In this section, a continuous stirred tank reactor (CSTR) is simulated. The process model is similar to that provided by Yoon and MacGregor (2001). A diagram of the process is shown in Fig. 3. The simulation is performed according to the following model: (18) , (19) , where k ₀, E, and ΔH_rxn are the pre-exponential constant, activation energy, and enthalpy of the reaction, R is the gas constant, c _p and ρ are the heat capacity and density of the fluid in the reactor. The process is monitored to measure the cooling water temperature T _C, inlet temperature T ₀, inlet concentrations C _AA and C _AS, solvent flows F _S, cooling water flow F _C, outlet concentration C _A, temperature T, and reactant flow F _A. The detailed model information and simulation parameters are provided in (Alcala and Qin, 2010). The measurements are sampled every minute, and the 1000 samples taken under normal conditions are used as the training data set. For fault cases, the test data sets also comprise 1000 samples, with the fault introduced after the 200th sample.




 Two different faults are studied in this work. The first simulated fault, Fault 1, is a bias in the sensor of the output temperature T, with a bias magnitude of 0.05. Given that T is a controlled variable, the effect of the fault will be removed by the proportional-integral controller, and its effect will propagate to other variables. This fault is considered as a complex fault because it affects several variables. The monitoring results of the conventional PCA-based method and the proposed method are illustrated in Figs. 4a and 4b, respectively. The retained number of PCs for PCA is determined using the CPV criterion, with the cutoff value of 85%. The superiority of the proposed method for fault detection can be easily observed. Fault 2 is a drift in the sensor of C _AA and its magnitude is kmol/(m³·min); thus, this is a simple fault. The conventional PCA-based method detects this fault with a delay of 190 min, as shown in Fig. 5. The improved PCA-based method can successfully and consistently detect the fault.

4.2.  Tennessee Eastman (TE) process

 TE process is a well-known benchmark for testing the performance of various fault detection methods (Lyman and Georgakist, 1995; Yu and Qin, 2008; Liu et al., 2010; Chen and Yan, 2012; Stubbs et al., 2012). A flowchart of the TE process is shown schematically in Fig. 6. There are five major unit operations in the process: a reactor, a condenser, a recycle compressor, a separator, and a stripper. Four reactants A, C, D, E, and inert B are fed to the reactor where the products G and H are formed and a by-product F is also produced. The process has 22 continuous process measurements, 12 manipulated variables, and 19 composition measurements sampled less frequently. Details on the process description are well explained in (Chiang et al., 2001). A total of 33 variables are used for monitoring in this study as shown in Table 1. All composition measurements are excluded since they are hard to measure online in practice.







 The detailed description of the programmed faults (Faults 1–21) is listed in Table 2. The data are generated at a sampling interval of 3 min and can be downloaded from http://brahms.scs.uiuc.edu. For comparison, 960 normal samples are used to build the model. Another 500 normal samples are used for the validation. Each fault data set contains 960 samples, with the fault introduced after sample 160.




 The proposed method is applied to the TE process simulation data and its fault detection performance is compared with conventional PCA monitoring strategy that utilizes the subjective method (i.e., CPV, cross-validation) to select PCs. A total of 18 faults have been tested in the TE process. Faults 3, 9, and 15 have been suggested to be difficult to detect (Russell et al., 2000; Lee et al., 2006; Wang and He, 2010), which is also confirmed in this study.

 Therefore, they are not considered in this study. The normal training data set has 960 observations, which is used for PCA modeling. The loading matrix P ∈ is obtained. The dimension of the reduced subspace is determined using the CPV method, in which the CPV value is larger than 85% when the number of PCs is 14.

 In this study, the time-window length is d=50. The discussion of different window length is illustrated later. The limit of the D index has been selected as 70%, that is, the CPCs can capture 70% of all the variability of the current score matrix. For the normal data set, the monitoring charts for the two methods are shown in Fig. 7. The false alarm rate (Type I error) of normal data are almost at the same degree. The fault detection results of all the 18 faults are listed in Table 3. The conventional PCA-based methods have difficulties in consistently detecting the five faults (Faults 5, 10, 16, 19, and 20), with detection rates of less than 40% in most cases. By contrast, the improved PCA can detect almost all of the 18 faults consistently, with detection rates higher than 80%. The detail fault detection results of Faults 10, 16, and 19 are shown in Figs. 8 to 10, respectively. The results clearly show that the fault detection performance has been greatly improved by the proposed monitoring scheme.

5.  Discussion

5.1.  Advantages of utilizing CPCs

 Based on the case study described earlier, several points are discussed in this section. First, the feature of the CPCs for statistic T ² is considered. For the PCA-based monitoring method, the number of PCs greatly affects the ability of the fault detection (Tamura and Tsujita, 2007). In addition, a publication of PCA applications indicates that including components with smaller eigenvalues in the PCA model and excluding those with larger eigenvalues can improve the prediction quality (Togkalidou et al., 2001). One advantage of CPCs, as mentioned earlier, is choosing the PCs with significant variation from the reference PCs and excluding those PCs with small variation. This approach focuses on the variation rather than on the variance, thereby leading to better monitoring performance. This advantage is well illustrated by a specific example discussed below.

 For example, the detection rate of Fault 4 for the improved PCA is higher than that of the conventional PCA. Given that CPCs include the 16th, 21st, and 22nd PCs after the fault occurred, they are ignored by the CPV criterion because they maintain smaller variance compared with the first 14 PCs. Meanwhile, the 4th, 5th, 6th, and other PCs are excluded in CPCs because they present small deviations after the fault occurs. The plots of the above PCs are illustrated in Fig. 11. Retaining PCs with a larger eigenvalue in the PCA model may present a small variability of the abnormal data. As a result, including PCs with smaller eigenvalues for the fault detection can improve the monitoring performance because they show much more significant variation than the other PCs, thereby directly improving the monitoring performance.




 Second, the false alarm rate of the proposed method is considered. False alarms are expected to increase when numerous PCs are retained because the model may include noise, which yields a large prediction error. Another 500 normal observations are used for analyzing the occurrence frequency of Type I error. The result is shown in Fig. 7. The frequency of Type I error is about 0.4%, almost the same with the conventional PCA. In practice, this significantly low false alarm rates can be neglected. Given the similar variability for all PCs, any PC can be selected as CPC. When we check the number of CPCs, we find that it is less than 14 and lessens when fault occurs. Given that the number of selected PCs is less than that of the CPV criterion, the false alarm rate can be confirmed.

5.2.  Detection delay of improved PCA with CPCs

 The size of the time-window is a predetermined parameter in the proposed method, which probably results in large delay in the fault detection. The detection delay of the proposed method is discussed in this subsection. Table 4 shows the detection delay result of the 18 faults. Considering an out-of-control value of a statistic can trigger a fault alarm, the fault is indicated only when six consecutive statistic values exceed the control limit to decrease the rate of false alarms (Detroja et al., 2007). The window lengths of d=50, 100, and 150 have little impact on the fault detection delay, and the time delay is not greater than that of the conventional PCA with CPV criterion, and sometimes even better. The index ΔT(i) in Eq. (14) is composed of the correlation and covariance matrices of the current score matrix, which incorporates the 1-norm of the ith column in correlation and covariance matrices. Although the newly added data point is abnormal even if the rest of the data matrix is normal, the degree that the ith element of its score vector changes can be captured by ΔT(i) immediately. Therefore, the detection delay is unlikely to be affected by the size of the time-window.

6.  Conclusions

 In this work, a novel objective method for determining PCs used for fault detection is developed. Instead of initially selecting several PCs subjectively, the PCs that capture the dominant variability of the current time-window are determined online. Therefore, the selected PCs, termed as CPCs, can capture the dominant process variation dynamically and objectively. Considering the fault detection performance, the novel CPC-based method can be more sensitive to faults.

 The proposed method has been applied in the TE process and compared with the conventional PCA to examine its performance. The case study has demonstrated that the improved PCA with CPC-based method detects various faults more efficiently than the conventional PCA method. In particular, the detection delay is superior to the conventional PCA, even with a time-window incorporated into the monitoring scheme. However, the proposed method focuses on the selection of PCs, and the other procedures are similar to the conventional PCA. Therefore, the proposed monitoring procedures suffer from the defects of the conventional PCA and need further investigation.

---

* Project supported by the National Basic Research Program of China (973 Program) (No. 2013CB733600), the National Natural Science Foundation of China (No. 21176073), the Program for New Century Excellent Talents in University (No. NCET-09-0346), and the Fundamental Research Funds for the Central Universities, China

Appendix: Fast algorithm for updating correlation matrix and covariance matrix

 Let the kth data matrix projected onto P with window length d be R _k+1=[ t _{k −d+1}, t _{k −d+2}, …, t _k ]^T, similarly, the next data matrix projected onto P would be T _k+1=[ t _{k −d+2}, t _{k −d+3}, …, t _k+1 ]^T. A two-step adaptation, as shown in Figs. FA1, is used to recursively update covariance matrix S and correlative matrix R .




 Firstly, with eliminating the oldest sample from T _k, the covariance matrix and correlation matrix associated with can be computed recursively. Eqs. (A1) and (A2) describe the variable mean. Eqs. (A3) and (A4) describe the variable variance, while the scaling of the data point is defined in Eq. (A5). (A1) , (A2) , (A3) , (A4) , (A5) .

 A new matrix R ^* is now introduced to simplify the formation of the equations: (A6) .

  R ^*is further derived into: (A7) . Then the correlative matrix of Matrix II is expressed as (A8) .

 Secondly, the next window of selected data (Matrix III) produced by adding the new data point t _k+1 . With , S _k+1 and R _k+1 associated with T _k+1 are obtained similar to those mentioned above, i.e., (A9) , (A10) , (A11) , (A12) , (A13) , (A14) , where the updated mean vector and the change in the mean vectors are computed from Eqs. (A8) and (A9), and the adaptation of the standard deviations follows from Eqs. (A11) and (A12). The scaling of the new projected data and the updating of covariance and correlation matrix are described in Eqs. (A13), (A10), and (A14), respectively. The adaptation of the standard deviations follows from substituting Eqs. (A2) into (A11) to yield: (A15) . In fact, the above two steps can be combined to pro-vide a routine that derives Matrix III directly from Matrix I. R _k+1 is expressed as (A16) . Because of the special formulation of covariance matrix, it is easier to update the covariance matrix. The updated S _k+1 is (A17) , where b ₀ is the mean vector of T ₀ as described above.