Condition Based Monitoring using Nonparametric Similarity Based Modeling
公開日:
カテゴリ: 第3回
Stephan WEGERICH Technology Development, SmartSignal Corporation
1. Introduction
Nonparametric modeling techniques are utilized extensively in applications involving the analysis of complex data sets. Nonparametric approaches have the desirable characteristic that they do not require a priori knowledge of the analytical form of the model meant to characterize a dataset. Instead, they rely exclusively on the available data to establish relationships present in the data. Many techniques fall into this class, including Kernel Regression (KR) [1, 2], Support Vector Machines (SVM) [3, 4], some Neural Networks [15, 16] and Similarity Based Modeling (SBM) (5, 6, and 7]. Principle Component Analysis (PCA), although not typically described as a nonparametric technique, can be thought of as nonparametric as well.Many real-world applications are well suited for nonparametric modeling techniques especially when the driving phenomena are unknown or too complicated for classical methods to be easily utilized. For example, A KR approach based on local polynomial fitting is utilized in [8] to build a model of the salinity levels in rivers in the Western US to better identify mitigation needs. A computer aided diagnosis system for sonography is proposed in [9] that incorporates SVMs to automatically classify breast tumors as benign or malignant. VariousPOC: Stephan WEGERICH, SmartSignal Corp. 901 Warrenville Rd., Suite 300, Lisle, Illinois, U.S.A., Tel: 630-829-4013, e-mail:swegerich@smartsignal.comneural-fuzzy techniques are used to tackle a wide variety of business-related applications such as bankruptcy prediction, money laundering detection and consumer risk assessment in [10].In the context of condition based monitoring and fault detection, a number of nonparametric approaches have been applied successfully. The Multivariate State Estimation Technique (MSET), which is itself a type of SBM, was originally developed for monitoring applications in Nuclear Plants in the US [11, 12] and has since been applied to computer systems as well [13]. KR is used for novelty detection in an asynchronous motor in [14]. Unlike most applications of KR in which an inferential framework is employed, KR is applied in an autoassociative mode in this application. An autoassociative NN framework is utilized in [15] and [16] for monitoring applications in nuclear plants. In [17], a PCA-based method is presented for detecting faults in a chemical process. Finally, in [6], SVM, KR and SBM are examined theoretically and practically based on two realworld condition-monitoring applications.SBM in particular has been successfully applied to a number of diverse condition based monitoring applications including power plant monitoring [18, 19], jet engine monitoring [7], steel production plant monitoring [20], helicopter gearbox monitoring [21] and variable speed motor vibration monitoring [5]. SBM has also been demonstrated commercially in fleet-wide asset monitoring applications. In [7], the jet engines for all of the fleets of a commercial airliner are monitored automatically using
out that the use of the term ““nonparametric““ does not necessarily imply that there are no free parameters to be estimated or specified for a nonparametric model.Methods for ModelingDo131SBM and in [18] a centralized monitoring station based on SBM is described for monitoring all of the power production units of a power company.Nonparametric modeling approaches are well suited for condition monitoring and fault detection applications because most, if not all, systems that are suitably instrumented and for which historical data exist, can be modeled and hence monitored. Similarity Based SBM and in [18] a centralized monitoring station based on SBM is described for monitoring all of the power production units of a power company.Nonparametric modeling approaches are well suited for condition monitoring and fault detection applications because most, if not all, systems that are suitably instrumented and for which historical data exist, can be modeled and hence monitored. Similarity BasedEmpiricaEmpiricalout that the use of the term ““nonparametric““ does not necessarily imply that there are no free parameters to be estimated or specified for a nonparametric model.Methods for Modeling1st Principles ModelingEmpirical ModelingSBM and in [18] a centralized monitoring station based on SBM is described for monitoring all of the power production units of a power company.Nonparametric modeling approaches are well suited for condition monitoring and fault detection applications because most, if not all, systems that are suitably instrumented and for which historical data exist, can be modeled and hence monitored. Similarity Based Modeling turns out to be a particularly effective nonparametric condition based monitoring technique. Unlike most nonparametric approaches, SBM does not require the use of complicated optimization algorithms in order to be deployed and is routinely used to model systems with large numbers of variables [5, 18,19 and 21] (in [5], for example, the modeled variables are the many frequency components of a spectral signal). This paper describes the mathematics behind SBM and compares the performance of SBM (specifically local SBM) to MSET, a PCA-based approach and a KR-based approach using actual plant data for a power plant in the US.Design-based Physics-basedParametricNonparametricFSBM- Kalman Filter FARMASVM L Linear regression L KRFigure 1 - Modeling Techniques Taxonomy3. Similarity Based Modeling (SBM) SBM is a pattern reconstruction technique based on multidimensional interpolation that is designed to exactly fit training data. The philosophy behind this design in the2. Nonparametric Modeling Condition monitoring modeling techniques essentially fall into two main classes: first principles models andNonparametric modeling approaches are well suited for condition monitoring and fault detection applications because most, if not all, systems that are suitably instrumented and for which historical data exist, can be modeled and hence monitored. Similarity Based Modeling turns out to be a particularly effective nonparametric condition based monitoring technique. Unlike most nonparametric approaches, SBM does not require the use of complicated optimization algorithms in order to be deployed and is routinely used to model systems with large numbers of variables (5, 18,19 and 21] (in (5), for example, the modeled variables are the many frequency components of a spectral signal). This paper describes the mathematics behind SBM and compares the performance of SBM (specifically local SBM) to MSET, a PCA-based approach and a KR-based approach using actual plant data for a power plant in the US. Condition monitoring modeling techniques essentially fall into two main classes: first principles models and empirical models. The former requires deep knowledge of the system or process being modeling to be effectively utilized. The latter is driven almost entirely by available data that characterize the behavior of the system (e.g., sensor data). If we focus on the empirical modeling class, we can further break down the methodologies as shown in figure 1 into nonparametric and parametric techniques.The difference between a parametric model and nonparametric model can be described as follows. A parametric model fits the model:Y; = f(b, x;) +E; where B = (B1, B2,...,B) is a vector of parameters to be estimated for the function f(““) that is specified in advance. Here x; is a set of input variables and ￡; is a random component that accounts for the noise in the data and the-1ofle5S,adA3. Similarity Based Modeling (SBM) SBM is a pattern reconstruction technique based on multidimensional interpolation that is designed to exactly fit training data. The philosophy behind this design in the context of fault detection is that in real-world applications it is extremely rare to be able to make reliable assumptions about the statistical distribution and level of smoothness in most datasets encountered so instead assume that the training data are pristine. In practice, this assumption is just as valid (if not more valid) then any typical smoothness and statistical distribution assumptions about the data. This is particularly apparent in applications in which the data are sampled at low sampling rates (as is typical in fleet wide condition monitoring applications) and in which the data path includes sensor signal filters, A/D converters, data historians that use compression, and other system that effect the data in some way before the data are made available to the modeling technique.The SBM approach is based on the application of a “similarity operation” on pairs of observation vectors and the manipulation of a ““state““ matrix (D) containing a set of historical training vectors (data points). The number of columns is equal to the number of representative training vectors (M) and the number of rows is equal to the number of data sources contained in each vector (L). Defining thebehehenered- 309 ?The object of a nonparametric model is to estimate 1) directly rather than to estimate parameters. In other words, f(-) is treated as a ““black box““. It should be pointed - 309 ?SBM is a pattern reconstruction technique based on multidimensional interpolation that is designed to exactly fit training data. The philosophy behind this design in the context of fault detection is that in real-world applications it is extremely rare to be able to make reliable assumptions about the statistical distribution and level of smoothness in most datasets encountered so instead assume that the training data are pristine. In practice, this assumption is just as valid (if not more valid) then any typical smoothness and statistical distribution assumptions about the data. This is particularly apparent in applications in which the data are sampled at low sampling rates (as is typical in fleet wide condition monitoring applications) and in which the data path includes sensor signal filters, A/D converters, data historians that use compression, and other system that effect the data in some way before the data are made effect the data in some way before the data are made available to the modeling technique.The SBM approach is based on the application of a ““similarity operation” on pairs of observation vectors and the manipulation of a ““state““ matrix (D) containing a set of historical training vectors (data points). The number of columns is equal to the number of representative training vectors (M) and the number of rows is equal to the number of data sources contained in each vector (L). Defining the set of measurements taken at a given time n; as a training vector, x(ni),x(n;)=[x(N) x2(n) x3(n;)...87(n)]-3where xi(n;) is the measurement from data source i at time nj, then the state matrix D is given by:D=[x(n) x(m2) x(nz)... X(nm)]. (4) The result of the similarity operation for two observation vectors is a similarity score (a scalar). The similarity operation is nonlinear but can be extended to matrix operations, where a scalar similarity score is rendered for each combination of two vectors stored in two matrices. Accordingly, given an input vector Xin containing single readings from each of the L data sources, a vector of corresponding estimated data source values, Xest. is determined from (5)-(7).(5) The estimate is composed of a linear combination of the vectors stored in D. Here, w is a set of weighting factors derived from the following.Xest = D.wR =(PC 8D).(DT 8x,,)=G-a (7) The similarity operation is indicated by the symbol R. Equation (7) can be interpreted as a two-step transformation:1. Measure the amount of similarity between theinput vector and each of the corresponding training vectors to produce a similarity scorevector, (a), 2. Transform (pre-multiply by G-?) the score vectorinto a set of weighting factors that are then used to compose an estimate based on a linearcombination of the vectors in D. The D matrix is typically created from a larger set of historic reference data covering the full dynamic range of the monitored equipment. The selection of vectors is accomplished such that the number of them is minimized while their data span the full dynamic range of the monitored equipment. The number of vectors required is dependent on both the number of sensors in the model, as well as the dynamic variability of the equipment in normal operation. In situations in which the modeled data sources-3are highly linearly correlated, few vectors are needed. However, a general rule of thumb provides for the selection of a number of vectors no less than 2 times the number of data sources. It is not uncommon to model a 50-60 variable system with 200-300 vectors.The contributions of each training vector to an individual estimate can be either positive or negative. Figure 2 shows an example of a fictitious three variable system. The surface represents the ideal behavior of the system. The gray triangle data points represent the training vectors. Estimates are generated that lie on the ideal surface using a weighted combination of the training vectors. In this example, an estimate (shown by the black circle) is calculated using the weighted linear combination of training vectors with the contributions shown in black near the origin. The example in Figure 2 depicts an inferential modeling application, which means that the independent inputs (x and y values) are used to determine the contributions needed to generate the corresponding-6-7stepOss01 0507 06 054321atheFigure 2 ? Inferential estimate generation using response variable (z value) estimate. Advantageously, the SBM technique is also capable of generating estimates in an autoassociative mode, where all outputs are also inputs. This is a critical advantage for the situation where there are hundreds or even thousands of monitored data sources and accurately determining cause and effect relationships is nearly impossible.The above description of SBM is referred to as ““fixed““ SBM (FSBM). Meaning that the state matrix D is generated up front and remains static (except for intermittent adaptation, in which case a new fixed state matrix is defined). However, SBM can also be applied in a local learning fashion. In this way, a state matrix is generated for each new input vector. This approach is called Local SBM (or LSBM). LSBM has been shown in - 310 -practice to provide better model performance characteristics then fixed models in a variety of real-world applications (SBM or otherwise). The basic idea is to select a subset of data to define a state matrix D(1) that is most relevant to the current input vector from a larger superset H that encompasses the full operating range of the modeled system. The estimate is then generated based on a linear combination of only these selected, currently relevant vectors. The process is repeated for each new input vector. The estimate thus becomes,Dt)={IF(H, xint))} W = (D(0)““ D(1))““:(D(0)““ Rxn(t))=G(1)-4.a (9) where F(-;-) the vector selection process given an input vector Xin(t) and a reference data matrix H.robustness metric for each variable in a model is then given by the following equation.mean (Sao ? al) Robustness =-10Here, perfect robustness is achieved when (10) is equal to 0. That is, when the unperturbed and perturbed estimates are identical. A larger value indicates more over-fitting and hence less model robustness.The spillover metric measures the relative amount that variables in a model deviate from normality when another variable is perturbed. In contrast to robustness, spillover measures the robustness on all other variables when one variable is perturbed. The spillover measurement for each variable is calculated using a similar calculation, which is given bySpillover, - w-12-Imean(\ &io ? fina, 1)““(11)where to is the estimate for variable i when no variables are perturbed, cili is the estimate of variable i when variable j is perturbed by Aj, and A; is the perturbation amount used when variable i is itself perturbed.The error metric is simply the root mean squared error of the difference between the actual value and its estimate divided by the standard deviation of the actual value, or equivalently the residual RMS divided by the actual value standard deviation (equation 12). Error = rms(x ? m)_rms (residual)-12Equations (10, 11 and 12) define the metrics for each variable in a model. The overall performance metrics for a model are calculated by averaging the results for each variable in each case.4. Model Performance MetricsThe performance of a model can be measured using a variety of techniques. In this paper, a nonparametric perturbation based-approach is used that is particularly well suited for fault detection applications. The performance of a model is assessed using three metrics: 1) robustness, 2) spillover and 3) error. The robustness metric is a measurement of the likelihood that a model will follow (or over-fit) a perturbation introduced into the data. To measure robustness, first estimates for all of then| --.toy,0.50.60.70.8variable in each case.6789Figure 3 -Robustness Metric5. Performance Comparison To compare the performance of a number of nonparametric modeling techniques, data from the Detroit Edison, Belle River Power Plant were utilized. The Belle River facility is a 600 MW coal plant located in Midwestern US. The data were collected over a 6-month period at 10-minute intervals and consisted of 13 variables. The 13 variables are related to the Primary Air Fan (PA Fan) system, which provides the primary air supply to the boiler. Included in the set of model variables are motor current, motor temperatures, motora model are made based on a test data set normal data (to in figure 3). Next, aA is added one at a time to each variable in as shown in figure 3 (xs). Finally, estimates ed for each of perturbed variable (s). The- 311 - variables in a model are made based on a test data set containing normal data (& in figure 3). Next, a perturbation A is added one at a time to each variable in the model as shown in figure 3 (xs). Finally, estimates are generated for each of perturbed variable (). The? 311 -AVERAGE PERFORMANCE METRICS COMPARED TO SOMROBUSTNESS SPILLOVER ERRORvibrations, and air pressures and flows. Table 1 shows of list of the actual variables used in the models.SEMMSETPCAGRNNTable 1 - Variables used in the P.A Fan ModelSTO OF PERFORMANCE METRICS COMPARED TO SOMVar #DescriptionUnitsROBUSTNESS SPILLOVER ERRORAMPPA FAN V-1004 'OUTSIDE AIR TEMPERATURE 'TOTAL PRIMARY AIR FLOW 'PA FAN V-1004 DISCH AIR TEMP' 'PA FAN V-1004 DISCH PRESSDEGF KPPH DEGE INWCSAMMSETPCAGRNN5Figure 4-Performance Metric Results ATable 1 - Variables used in the PA Fan ModelVar #DescriptionUnitsPA FAN V-1004AMP 'OUTSIDE AIR TEMPERATURE DEGF 'TOTAL PRIMARY AIR FLOWKPPH 'PA FAN V-1004 DISCH AIR TEMP' DEGF 'PA FAN V-1004 DISCH PRESS INWC 'PA FAN V-1004 INBD BRG'DEGE 'PA FAN V-1004 INBD BRG RADL' MIL 'PA FAN V-1004 OUTBD BRG'DEGF 'PRY AIR FAN V-1004 MOTOR TEMP'DEGC11 |12 | 13PA FAN V-1004 MTR INBD BRG' DEGF 'PA FAN V-1004 MTR INBD BRG | MIL 'PA FAN V-1004 MTR OUTBD BRG' DEGF 'PA FAN V-1004 OUTBD BRG RADL | MILSeveral modeling techniques were used in the comparison study: the Multivariate State Estimation Technique (MSET), Generalized Regression Neural Network (GRNN) and Principle Component Analysis (PCA). The version of SBM used in this comparison was of the local SBM (LSBM). The GRNN used is equivalent to Nadaraya Watson Kernel Regression (NWKR) with a Gaussian kernel. In all cases, the models are applied in autoassociative mode (all inputs are also outputs).All models were trained using the same data from the beginning of the PA Fan data set (September 1, 2005 through November 16, 2005). To assess the each model's performance, 3000 samples were used from November 23 through December 14, 2005 to form a test data set. The training data and test data were assumed to represent normal behavior (or at least a baseline condition). Figure 4 shows the overall model performance metric results for the four modeling approaches using SBM as the baseline (i.e. the percent increase or decrease in each metric with respect to the SBM metric values). The top plot shows the ratio of the average of each metric for each technique, to the corresponding SBM metric (which is why all of the metric values for SBM are equal to 1). The bottom plot shows the analogous plots for the standard deviation of each metric. The change in average error is fairly small in all cases. In fact, MSET shows a smaller error. However, the robustness metric is at least three times larger then that of SBM in all cases and the spillover metric is over fourtimes that of SBM in all cases. This shows that even though all of the techniques produce estimates with similar levels of accuracy, they would not all be effective for detecting faults. When the robustness metric is high, any drift within the range of the test data is followed by the model (over-fitted) masking any indication of the drift (or fault). In the context of condition based monitoring, this renders the technique useless because all drifts essentially ““look““ normal to the model. Nearly as bad is the presence of a large spillover metric. In this case, if a drift occurs in one of the variables in the model, the model responds by indicating the presence of drifts in many variables. This behavior is undesirable because determining exactly what caused the drift in the first place is nearly impossible to deduce.6. ConclusionIn this paper, the mathematics of the SBM technique is described. The SBM approach is a nonparametric modeling technique that can operate in both a fixed mode and localizing mode. The performance of the local SBM techniques is compared to four other nonparametric modeling approaches: MSET, GRNN and PCA. A nonparametric performance comparison approach is utilized as well that measures three distinct characteristics of a model (all three extremely important in the context of fault detection). These characteristics are model error, model robustness and model spillover.Actual power plant data are utilized in the performance comparison. The results show that all of the modeling techniques exhibit very similar error characteristics but vary dramatically in terms of robustness and spillover characteristics. In the context of fault detection, a model must have good robustness and spillover characteristics to enable an effective condition based monitoring system.? 312 ?SBM performed at least twice as well in terms of robustness and at least four times as well in terms of spillover than any other of the techniques examined in this study. Of the techniques tested, SBM is the most suitable for this condition monitoring application.ReferencesBockhorst, ““MSET Modeling of Crystal River-3 Venturi Flow Meters““, Proc. ASME/JSME/SFEN 6th Intnl. Conf. on Nuclear Engineering, San Diego, CA,USA, May 10-15, 1998. [13] Gross, K., Vaidyana, K., “MSET PerformanceOptimization for Detection of Software Aging““, Proc. 14th IEEE International. Symposium on Software Reliability Engineering. (ISSRE’03), Denver, CO,USA, Nov. 2003. [14] Diaz, I. and Hollmen, J., ““Residual Generation andVisualization for Understanding Novel Process Conditions““, Proc. of IEEE Int. Joint Conf. on NN (IJCNN2002), Vol. 3, pp. 2070-2075, IEEE Press,May, 2002. [15] Wrest, D., Hines, J., and Uhrig, R., ““SensorCalibration and Monitoring Using Autoassociative Neural Networks““, Proc. of XI ENFIR International Conference on Nuclear Energy, Rio Janerio, Brazil,August 18-22, 1997. [16] Hines, W, Uhrig, R., Black, C., and Xu, X., ““AnEvaluation of Instrument Calibration Monitoring Using Artificial Neural Networks““, Proc. American Nuclear Society Winter Meeting, Albuquerque, NM,USA, Nov. 16-20, 1997. [17] Kano, M., Hasebe, S., Hashimoto, I. Strauss, R.,Bakshi, B. and Ohno, H., ““Contribution Plots for Fault Identification Based on the Dissimilarity of Process Data““, Proc. AICHE Annual Meeting, Los Angeles,CA, USA, Nov. 12-17, 2000. [18] Flesch, P., Stoecker, C., ““Power Plant CentralizedMonitoring Using SBM““, Proceedings of PWR2006,ASME Power, Atlanta, GA, USA, May 2-4, 2006. [19] Holtan, T. and Wheeler, T., ““Using Real-TimePredictive Condition Monitoring to Increase Coal Plant Asset Availablility,““ Coal-Gen, August 6-8,2003. [20] Nieman, W. and Olson, R., ““Early Detection ofSignal or Process Variation in the Co-Generation Plant at U.S. Steel, Gary Works,““ Proceedings of TurboExpo: Land, Sea, Air, June 4-7, 2001. [21] Wegerich, S., ““Similarity Based Modeling of TimeSynchronous Averaged Vibration Signals for Machinery Health Monitoring,““ Proceedings, IEEE Aerospace Conference, Big Sky, MT, USA, March 613, 2004.[1] Hardle, W., Applied Nonparametric Regression,Cambridge University Press, Cambridge, UK, 1999. [2] Schimek, G., Smoothing and Regression, John Wiley &Sons, New York, USA, 2000. [3] Vapnik, V., The Nature of Statistical Learning,Springer, New York, USA, 2000. [4] Cristianini, N. and Shawe-Taylor, J., An Introductionto Support Vector Machines, Cambridge UniversityPress, Cambridge, UK, 2000. [5] Wegerich, S., ““Similarity Based Modeling of VibrationFeatures for Fault Detection and Identification““,Sensor Review, Vol. 25, Issue 2, 2005. [6] Wegerich X. Xu, ““A Performance Comparison ofSimilarity-Based and Kernel Modeling Techniques““, Proc. of MARCON 2003, Knoxville, TN, USA, May5-7, 2003. [7] Herzog, P., Hanlin, J., Wegerich, S. and Wilks, A.,““High Performance Condition Monitoring of Aircraft Engines““, Proceedings of GT2005, ASME Turbo,Reno-Tahoe, NV, USA, June 6-9, 2005. [8] Prairie, J., Rajagopalan, B., Fulp, T., and Zagona, A.,““Statistical Nonparametric Model for Natural Salt Estimation““, Journal of Environmental Engineering,Vol. 131, No. 1, pp. 130-138, Jan. 1, 2005. [9] Chang, R., Wu, W., Moon, W. and Chen, D.,““Improvement in Breast Tumor Discrimination by Support Vector Machines and Speckle-Emphasis Texture Analysis““, Ultrasound in Med. & Biol., Vol.29, No. 5, pp. 679-686, 2003. [10] Lisboa, P., et al., Business Applications of NeuralNetworks, World Scientific Publishing Co., RiverEdge, NJ, USA, 2000. [11] Gross, K., Singer, R., Wegerich, S., Herzog, J.,VanAlstine, R. and Bockhorst, F., “Application of a Model-based Fault Detection System to Nuclear Plant Signals”, Proc. 9th International Conf. On Intelligent Systems Applications to Power Systems, Seoul, Korea,July 6-10, 1997. [12] Herzog, J., Wegerich, S., K. C. Gross, and F. K.- 313 ?“ “Condition Based Monitoring using Nonparametric Similarity Based Modeling“ “Stephan WEGERICH
1. Introduction
Nonparametric modeling techniques are utilized extensively in applications involving the analysis of complex data sets. Nonparametric approaches have the desirable characteristic that they do not require a priori knowledge of the analytical form of the model meant to characterize a dataset. Instead, they rely exclusively on the available data to establish relationships present in the data. Many techniques fall into this class, including Kernel Regression (KR) [1, 2], Support Vector Machines (SVM) [3, 4], some Neural Networks [15, 16] and Similarity Based Modeling (SBM) (5, 6, and 7]. Principle Component Analysis (PCA), although not typically described as a nonparametric technique, can be thought of as nonparametric as well.Many real-world applications are well suited for nonparametric modeling techniques especially when the driving phenomena are unknown or too complicated for classical methods to be easily utilized. For example, A KR approach based on local polynomial fitting is utilized in [8] to build a model of the salinity levels in rivers in the Western US to better identify mitigation needs. A computer aided diagnosis system for sonography is proposed in [9] that incorporates SVMs to automatically classify breast tumors as benign or malignant. VariousPOC: Stephan WEGERICH, SmartSignal Corp. 901 Warrenville Rd., Suite 300, Lisle, Illinois, U.S.A., Tel: 630-829-4013, e-mail:swegerich@smartsignal.comneural-fuzzy techniques are used to tackle a wide variety of business-related applications such as bankruptcy prediction, money laundering detection and consumer risk assessment in [10].In the context of condition based monitoring and fault detection, a number of nonparametric approaches have been applied successfully. The Multivariate State Estimation Technique (MSET), which is itself a type of SBM, was originally developed for monitoring applications in Nuclear Plants in the US [11, 12] and has since been applied to computer systems as well [13]. KR is used for novelty detection in an asynchronous motor in [14]. Unlike most applications of KR in which an inferential framework is employed, KR is applied in an autoassociative mode in this application. An autoassociative NN framework is utilized in [15] and [16] for monitoring applications in nuclear plants. In [17], a PCA-based method is presented for detecting faults in a chemical process. Finally, in [6], SVM, KR and SBM are examined theoretically and practically based on two realworld condition-monitoring applications.SBM in particular has been successfully applied to a number of diverse condition based monitoring applications including power plant monitoring [18, 19], jet engine monitoring [7], steel production plant monitoring [20], helicopter gearbox monitoring [21] and variable speed motor vibration monitoring [5]. SBM has also been demonstrated commercially in fleet-wide asset monitoring applications. In [7], the jet engines for all of the fleets of a commercial airliner are monitored automatically using
out that the use of the term ““nonparametric““ does not necessarily imply that there are no free parameters to be estimated or specified for a nonparametric model.Methods for ModelingDo131SBM and in [18] a centralized monitoring station based on SBM is described for monitoring all of the power production units of a power company.Nonparametric modeling approaches are well suited for condition monitoring and fault detection applications because most, if not all, systems that are suitably instrumented and for which historical data exist, can be modeled and hence monitored. Similarity Based SBM and in [18] a centralized monitoring station based on SBM is described for monitoring all of the power production units of a power company.Nonparametric modeling approaches are well suited for condition monitoring and fault detection applications because most, if not all, systems that are suitably instrumented and for which historical data exist, can be modeled and hence monitored. Similarity BasedEmpiricaEmpiricalout that the use of the term ““nonparametric““ does not necessarily imply that there are no free parameters to be estimated or specified for a nonparametric model.Methods for Modeling1st Principles ModelingEmpirical ModelingSBM and in [18] a centralized monitoring station based on SBM is described for monitoring all of the power production units of a power company.Nonparametric modeling approaches are well suited for condition monitoring and fault detection applications because most, if not all, systems that are suitably instrumented and for which historical data exist, can be modeled and hence monitored. Similarity Based Modeling turns out to be a particularly effective nonparametric condition based monitoring technique. Unlike most nonparametric approaches, SBM does not require the use of complicated optimization algorithms in order to be deployed and is routinely used to model systems with large numbers of variables [5, 18,19 and 21] (in [5], for example, the modeled variables are the many frequency components of a spectral signal). This paper describes the mathematics behind SBM and compares the performance of SBM (specifically local SBM) to MSET, a PCA-based approach and a KR-based approach using actual plant data for a power plant in the US.Design-based Physics-basedParametricNonparametricFSBM- Kalman Filter FARMASVM L Linear regression L KRFigure 1 - Modeling Techniques Taxonomy3. Similarity Based Modeling (SBM) SBM is a pattern reconstruction technique based on multidimensional interpolation that is designed to exactly fit training data. The philosophy behind this design in the2. Nonparametric Modeling Condition monitoring modeling techniques essentially fall into two main classes: first principles models andNonparametric modeling approaches are well suited for condition monitoring and fault detection applications because most, if not all, systems that are suitably instrumented and for which historical data exist, can be modeled and hence monitored. Similarity Based Modeling turns out to be a particularly effective nonparametric condition based monitoring technique. Unlike most nonparametric approaches, SBM does not require the use of complicated optimization algorithms in order to be deployed and is routinely used to model systems with large numbers of variables (5, 18,19 and 21] (in (5), for example, the modeled variables are the many frequency components of a spectral signal). This paper describes the mathematics behind SBM and compares the performance of SBM (specifically local SBM) to MSET, a PCA-based approach and a KR-based approach using actual plant data for a power plant in the US. Condition monitoring modeling techniques essentially fall into two main classes: first principles models and empirical models. The former requires deep knowledge of the system or process being modeling to be effectively utilized. The latter is driven almost entirely by available data that characterize the behavior of the system (e.g., sensor data). If we focus on the empirical modeling class, we can further break down the methodologies as shown in figure 1 into nonparametric and parametric techniques.The difference between a parametric model and nonparametric model can be described as follows. A parametric model fits the model:Y; = f(b, x;) +E; where B = (B1, B2,...,B) is a vector of parameters to be estimated for the function f(““) that is specified in advance. Here x; is a set of input variables and ￡; is a random component that accounts for the noise in the data and the-1ofle5S,adA3. Similarity Based Modeling (SBM) SBM is a pattern reconstruction technique based on multidimensional interpolation that is designed to exactly fit training data. The philosophy behind this design in the context of fault detection is that in real-world applications it is extremely rare to be able to make reliable assumptions about the statistical distribution and level of smoothness in most datasets encountered so instead assume that the training data are pristine. In practice, this assumption is just as valid (if not more valid) then any typical smoothness and statistical distribution assumptions about the data. This is particularly apparent in applications in which the data are sampled at low sampling rates (as is typical in fleet wide condition monitoring applications) and in which the data path includes sensor signal filters, A/D converters, data historians that use compression, and other system that effect the data in some way before the data are made available to the modeling technique.The SBM approach is based on the application of a “similarity operation” on pairs of observation vectors and the manipulation of a ““state““ matrix (D) containing a set of historical training vectors (data points). The number of columns is equal to the number of representative training vectors (M) and the number of rows is equal to the number of data sources contained in each vector (L). Defining thebehehenered- 309 ?The object of a nonparametric model is to estimate 1) directly rather than to estimate parameters. In other words, f(-) is treated as a ““black box““. It should be pointed - 309 ?SBM is a pattern reconstruction technique based on multidimensional interpolation that is designed to exactly fit training data. The philosophy behind this design in the context of fault detection is that in real-world applications it is extremely rare to be able to make reliable assumptions about the statistical distribution and level of smoothness in most datasets encountered so instead assume that the training data are pristine. In practice, this assumption is just as valid (if not more valid) then any typical smoothness and statistical distribution assumptions about the data. This is particularly apparent in applications in which the data are sampled at low sampling rates (as is typical in fleet wide condition monitoring applications) and in which the data path includes sensor signal filters, A/D converters, data historians that use compression, and other system that effect the data in some way before the data are made effect the data in some way before the data are made available to the modeling technique.The SBM approach is based on the application of a ““similarity operation” on pairs of observation vectors and the manipulation of a ““state““ matrix (D) containing a set of historical training vectors (data points). The number of columns is equal to the number of representative training vectors (M) and the number of rows is equal to the number of data sources contained in each vector (L). Defining the set of measurements taken at a given time n; as a training vector, x(ni),x(n;)=[x(N) x2(n) x3(n;)...87(n)]-3where xi(n;) is the measurement from data source i at time nj, then the state matrix D is given by:D=[x(n) x(m2) x(nz)... X(nm)]. (4) The result of the similarity operation for two observation vectors is a similarity score (a scalar). The similarity operation is nonlinear but can be extended to matrix operations, where a scalar similarity score is rendered for each combination of two vectors stored in two matrices. Accordingly, given an input vector Xin containing single readings from each of the L data sources, a vector of corresponding estimated data source values, Xest. is determined from (5)-(7).(5) The estimate is composed of a linear combination of the vectors stored in D. Here, w is a set of weighting factors derived from the following.Xest = D.wR =(PC 8D).(DT 8x,,)=G-a (7) The similarity operation is indicated by the symbol R. Equation (7) can be interpreted as a two-step transformation:1. Measure the amount of similarity between theinput vector and each of the corresponding training vectors to produce a similarity scorevector, (a), 2. Transform (pre-multiply by G-?) the score vectorinto a set of weighting factors that are then used to compose an estimate based on a linearcombination of the vectors in D. The D matrix is typically created from a larger set of historic reference data covering the full dynamic range of the monitored equipment. The selection of vectors is accomplished such that the number of them is minimized while their data span the full dynamic range of the monitored equipment. The number of vectors required is dependent on both the number of sensors in the model, as well as the dynamic variability of the equipment in normal operation. In situations in which the modeled data sources-3are highly linearly correlated, few vectors are needed. However, a general rule of thumb provides for the selection of a number of vectors no less than 2 times the number of data sources. It is not uncommon to model a 50-60 variable system with 200-300 vectors.The contributions of each training vector to an individual estimate can be either positive or negative. Figure 2 shows an example of a fictitious three variable system. The surface represents the ideal behavior of the system. The gray triangle data points represent the training vectors. Estimates are generated that lie on the ideal surface using a weighted combination of the training vectors. In this example, an estimate (shown by the black circle) is calculated using the weighted linear combination of training vectors with the contributions shown in black near the origin. The example in Figure 2 depicts an inferential modeling application, which means that the independent inputs (x and y values) are used to determine the contributions needed to generate the corresponding-6-7stepOss01 0507 06 054321atheFigure 2 ? Inferential estimate generation using response variable (z value) estimate. Advantageously, the SBM technique is also capable of generating estimates in an autoassociative mode, where all outputs are also inputs. This is a critical advantage for the situation where there are hundreds or even thousands of monitored data sources and accurately determining cause and effect relationships is nearly impossible.The above description of SBM is referred to as ““fixed““ SBM (FSBM). Meaning that the state matrix D is generated up front and remains static (except for intermittent adaptation, in which case a new fixed state matrix is defined). However, SBM can also be applied in a local learning fashion. In this way, a state matrix is generated for each new input vector. This approach is called Local SBM (or LSBM). LSBM has been shown in - 310 -practice to provide better model performance characteristics then fixed models in a variety of real-world applications (SBM or otherwise). The basic idea is to select a subset of data to define a state matrix D(1) that is most relevant to the current input vector from a larger superset H that encompasses the full operating range of the modeled system. The estimate is then generated based on a linear combination of only these selected, currently relevant vectors. The process is repeated for each new input vector. The estimate thus becomes,Dt)={IF(H, xint))} W = (D(0)““ D(1))““:(D(0)““ Rxn(t))=G(1)-4.a (9) where F(-;-) the vector selection process given an input vector Xin(t) and a reference data matrix H.robustness metric for each variable in a model is then given by the following equation.mean (Sao ? al) Robustness =-10Here, perfect robustness is achieved when (10) is equal to 0. That is, when the unperturbed and perturbed estimates are identical. A larger value indicates more over-fitting and hence less model robustness.The spillover metric measures the relative amount that variables in a model deviate from normality when another variable is perturbed. In contrast to robustness, spillover measures the robustness on all other variables when one variable is perturbed. The spillover measurement for each variable is calculated using a similar calculation, which is given bySpillover, - w-12-Imean(\ &io ? fina, 1)““(11)where to is the estimate for variable i when no variables are perturbed, cili is the estimate of variable i when variable j is perturbed by Aj, and A; is the perturbation amount used when variable i is itself perturbed.The error metric is simply the root mean squared error of the difference between the actual value and its estimate divided by the standard deviation of the actual value, or equivalently the residual RMS divided by the actual value standard deviation (equation 12). Error = rms(x ? m)_rms (residual)-12Equations (10, 11 and 12) define the metrics for each variable in a model. The overall performance metrics for a model are calculated by averaging the results for each variable in each case.4. Model Performance MetricsThe performance of a model can be measured using a variety of techniques. In this paper, a nonparametric perturbation based-approach is used that is particularly well suited for fault detection applications. The performance of a model is assessed using three metrics: 1) robustness, 2) spillover and 3) error. The robustness metric is a measurement of the likelihood that a model will follow (or over-fit) a perturbation introduced into the data. To measure robustness, first estimates for all of then| --.toy,0.50.60.70.8variable in each case.6789Figure 3 -Robustness Metric5. Performance Comparison To compare the performance of a number of nonparametric modeling techniques, data from the Detroit Edison, Belle River Power Plant were utilized. The Belle River facility is a 600 MW coal plant located in Midwestern US. The data were collected over a 6-month period at 10-minute intervals and consisted of 13 variables. The 13 variables are related to the Primary Air Fan (PA Fan) system, which provides the primary air supply to the boiler. Included in the set of model variables are motor current, motor temperatures, motora model are made based on a test data set normal data (to in figure 3). Next, aA is added one at a time to each variable in as shown in figure 3 (xs). Finally, estimates ed for each of perturbed variable (s). The- 311 - variables in a model are made based on a test data set containing normal data (& in figure 3). Next, a perturbation A is added one at a time to each variable in the model as shown in figure 3 (xs). Finally, estimates are generated for each of perturbed variable (). The? 311 -AVERAGE PERFORMANCE METRICS COMPARED TO SOMROBUSTNESS SPILLOVER ERRORvibrations, and air pressures and flows. Table 1 shows of list of the actual variables used in the models.SEMMSETPCAGRNNTable 1 - Variables used in the P.A Fan ModelSTO OF PERFORMANCE METRICS COMPARED TO SOMVar #DescriptionUnitsROBUSTNESS SPILLOVER ERRORAMPPA FAN V-1004 'OUTSIDE AIR TEMPERATURE 'TOTAL PRIMARY AIR FLOW 'PA FAN V-1004 DISCH AIR TEMP' 'PA FAN V-1004 DISCH PRESSDEGF KPPH DEGE INWCSAMMSETPCAGRNN5Figure 4-Performance Metric Results ATable 1 - Variables used in the PA Fan ModelVar #DescriptionUnitsPA FAN V-1004AMP 'OUTSIDE AIR TEMPERATURE DEGF 'TOTAL PRIMARY AIR FLOWKPPH 'PA FAN V-1004 DISCH AIR TEMP' DEGF 'PA FAN V-1004 DISCH PRESS INWC 'PA FAN V-1004 INBD BRG'DEGE 'PA FAN V-1004 INBD BRG RADL' MIL 'PA FAN V-1004 OUTBD BRG'DEGF 'PRY AIR FAN V-1004 MOTOR TEMP'DEGC11 |12 | 13PA FAN V-1004 MTR INBD BRG' DEGF 'PA FAN V-1004 MTR INBD BRG | MIL 'PA FAN V-1004 MTR OUTBD BRG' DEGF 'PA FAN V-1004 OUTBD BRG RADL | MILSeveral modeling techniques were used in the comparison study: the Multivariate State Estimation Technique (MSET), Generalized Regression Neural Network (GRNN) and Principle Component Analysis (PCA). The version of SBM used in this comparison was of the local SBM (LSBM). The GRNN used is equivalent to Nadaraya Watson Kernel Regression (NWKR) with a Gaussian kernel. In all cases, the models are applied in autoassociative mode (all inputs are also outputs).All models were trained using the same data from the beginning of the PA Fan data set (September 1, 2005 through November 16, 2005). To assess the each model's performance, 3000 samples were used from November 23 through December 14, 2005 to form a test data set. The training data and test data were assumed to represent normal behavior (or at least a baseline condition). Figure 4 shows the overall model performance metric results for the four modeling approaches using SBM as the baseline (i.e. the percent increase or decrease in each metric with respect to the SBM metric values). The top plot shows the ratio of the average of each metric for each technique, to the corresponding SBM metric (which is why all of the metric values for SBM are equal to 1). The bottom plot shows the analogous plots for the standard deviation of each metric. The change in average error is fairly small in all cases. In fact, MSET shows a smaller error. However, the robustness metric is at least three times larger then that of SBM in all cases and the spillover metric is over fourtimes that of SBM in all cases. This shows that even though all of the techniques produce estimates with similar levels of accuracy, they would not all be effective for detecting faults. When the robustness metric is high, any drift within the range of the test data is followed by the model (over-fitted) masking any indication of the drift (or fault). In the context of condition based monitoring, this renders the technique useless because all drifts essentially ““look““ normal to the model. Nearly as bad is the presence of a large spillover metric. In this case, if a drift occurs in one of the variables in the model, the model responds by indicating the presence of drifts in many variables. This behavior is undesirable because determining exactly what caused the drift in the first place is nearly impossible to deduce.6. ConclusionIn this paper, the mathematics of the SBM technique is described. The SBM approach is a nonparametric modeling technique that can operate in both a fixed mode and localizing mode. The performance of the local SBM techniques is compared to four other nonparametric modeling approaches: MSET, GRNN and PCA. A nonparametric performance comparison approach is utilized as well that measures three distinct characteristics of a model (all three extremely important in the context of fault detection). These characteristics are model error, model robustness and model spillover.Actual power plant data are utilized in the performance comparison. The results show that all of the modeling techniques exhibit very similar error characteristics but vary dramatically in terms of robustness and spillover characteristics. In the context of fault detection, a model must have good robustness and spillover characteristics to enable an effective condition based monitoring system.? 312 ?SBM performed at least twice as well in terms of robustness and at least four times as well in terms of spillover than any other of the techniques examined in this study. Of the techniques tested, SBM is the most suitable for this condition monitoring application.ReferencesBockhorst, ““MSET Modeling of Crystal River-3 Venturi Flow Meters““, Proc. ASME/JSME/SFEN 6th Intnl. Conf. on Nuclear Engineering, San Diego, CA,USA, May 10-15, 1998. [13] Gross, K., Vaidyana, K., “MSET PerformanceOptimization for Detection of Software Aging““, Proc. 14th IEEE International. Symposium on Software Reliability Engineering. (ISSRE’03), Denver, CO,USA, Nov. 2003. [14] Diaz, I. and Hollmen, J., ““Residual Generation andVisualization for Understanding Novel Process Conditions““, Proc. of IEEE Int. Joint Conf. on NN (IJCNN2002), Vol. 3, pp. 2070-2075, IEEE Press,May, 2002. [15] Wrest, D., Hines, J., and Uhrig, R., ““SensorCalibration and Monitoring Using Autoassociative Neural Networks““, Proc. of XI ENFIR International Conference on Nuclear Energy, Rio Janerio, Brazil,August 18-22, 1997. [16] Hines, W, Uhrig, R., Black, C., and Xu, X., ““AnEvaluation of Instrument Calibration Monitoring Using Artificial Neural Networks““, Proc. American Nuclear Society Winter Meeting, Albuquerque, NM,USA, Nov. 16-20, 1997. [17] Kano, M., Hasebe, S., Hashimoto, I. Strauss, R.,Bakshi, B. and Ohno, H., ““Contribution Plots for Fault Identification Based on the Dissimilarity of Process Data““, Proc. AICHE Annual Meeting, Los Angeles,CA, USA, Nov. 12-17, 2000. [18] Flesch, P., Stoecker, C., ““Power Plant CentralizedMonitoring Using SBM““, Proceedings of PWR2006,ASME Power, Atlanta, GA, USA, May 2-4, 2006. [19] Holtan, T. and Wheeler, T., ““Using Real-TimePredictive Condition Monitoring to Increase Coal Plant Asset Availablility,““ Coal-Gen, August 6-8,2003. [20] Nieman, W. and Olson, R., ““Early Detection ofSignal or Process Variation in the Co-Generation Plant at U.S. Steel, Gary Works,““ Proceedings of TurboExpo: Land, Sea, Air, June 4-7, 2001. [21] Wegerich, S., ““Similarity Based Modeling of TimeSynchronous Averaged Vibration Signals for Machinery Health Monitoring,““ Proceedings, IEEE Aerospace Conference, Big Sky, MT, USA, March 613, 2004.[1] Hardle, W., Applied Nonparametric Regression,Cambridge University Press, Cambridge, UK, 1999. [2] Schimek, G., Smoothing and Regression, John Wiley &Sons, New York, USA, 2000. [3] Vapnik, V., The Nature of Statistical Learning,Springer, New York, USA, 2000. [4] Cristianini, N. and Shawe-Taylor, J., An Introductionto Support Vector Machines, Cambridge UniversityPress, Cambridge, UK, 2000. [5] Wegerich, S., ““Similarity Based Modeling of VibrationFeatures for Fault Detection and Identification““,Sensor Review, Vol. 25, Issue 2, 2005. [6] Wegerich X. Xu, ““A Performance Comparison ofSimilarity-Based and Kernel Modeling Techniques““, Proc. of MARCON 2003, Knoxville, TN, USA, May5-7, 2003. [7] Herzog, P., Hanlin, J., Wegerich, S. and Wilks, A.,““High Performance Condition Monitoring of Aircraft Engines““, Proceedings of GT2005, ASME Turbo,Reno-Tahoe, NV, USA, June 6-9, 2005. [8] Prairie, J., Rajagopalan, B., Fulp, T., and Zagona, A.,““Statistical Nonparametric Model for Natural Salt Estimation““, Journal of Environmental Engineering,Vol. 131, No. 1, pp. 130-138, Jan. 1, 2005. [9] Chang, R., Wu, W., Moon, W. and Chen, D.,““Improvement in Breast Tumor Discrimination by Support Vector Machines and Speckle-Emphasis Texture Analysis““, Ultrasound in Med. & Biol., Vol.29, No. 5, pp. 679-686, 2003. [10] Lisboa, P., et al., Business Applications of NeuralNetworks, World Scientific Publishing Co., RiverEdge, NJ, USA, 2000. [11] Gross, K., Singer, R., Wegerich, S., Herzog, J.,VanAlstine, R. and Bockhorst, F., “Application of a Model-based Fault Detection System to Nuclear Plant Signals”, Proc. 9th International Conf. On Intelligent Systems Applications to Power Systems, Seoul, Korea,July 6-10, 1997. [12] Herzog, J., Wegerich, S., K. C. Gross, and F. K.- 313 ?“ “Condition Based Monitoring using Nonparametric Similarity Based Modeling“ “Stephan WEGERICH