Coefficient of determination in nonlinear signal processing

Edward R. Dougherty, Seungchan Kim, Yidong Chen

Research output: Contribution to journalArticle

151 Citations (Scopus)

Abstract

For statistical design of an optimal filter, it is probabilistically advantageous to employ a large number of observation random variables; however, estimation error increases with the number of variables, so that variables not contributing to the determination of the target variable can have a detrimental effect. In linear filtering, determination involves the correlation coefficients among the input and target variables. This paper discusses use of the more general coefficient of determination in nonlinear filtering. The determination coefficient is defined in accordance with the degree to which a filter estimates a target variable beyond the degree to which the target variable is estimated by its mean. Filter constraint decreases the coefficient, but it also decreases estimation error in filter design. Because situations in which the sample is relatively small in comparison with the number of observation variables are of salient interest, estimation of the determination coefficient is considered in detail. One may be unable to obtain a good estimate of an optimal filter, but can nonetheless use rough estimates of the coefficient to find useful sets of observation variables. Since minimal-error estimation underlies determination, this material is at the interface of signal processing, computational learning, and pattern recognition. Several signal-processing factors impact application: the signal model, morphological operator representation, and desirable operator properties. In particular, the paper addresses the VC dimension of increasing operators in terms of their morphological kernel/basis representations. Two applications are considered: window size for restoring degraded binary images; finding sets of genes that have significant predictive capability relative to target genes in genomic regulation.

Original languageEnglish (US)
Pages (from-to)2219-2235
Number of pages17
JournalSignal Processing
Volume80
Issue number10
DOIs
StatePublished - Oct 2000
Externally publishedYes

Fingerprint

Error analysis
Signal processing
Genes
Nonlinear filtering
Binary images
Random variables
Pattern recognition

ASJC Scopus subject areas

  • Signal Processing
  • Electrical and Electronic Engineering

Cite this

Coefficient of determination in nonlinear signal processing. / Dougherty, Edward R.; Kim, Seungchan; Chen, Yidong.

In: Signal Processing, Vol. 80, No. 10, 10.2000, p. 2219-2235.

Research output: Contribution to journalArticle

Dougherty, Edward R. ; Kim, Seungchan ; Chen, Yidong. / Coefficient of determination in nonlinear signal processing. In: Signal Processing. 2000 ; Vol. 80, No. 10. pp. 2219-2235.
@article{b20c4d9f964f40b68cb6b63e8916af85,
title = "Coefficient of determination in nonlinear signal processing",
abstract = "For statistical design of an optimal filter, it is probabilistically advantageous to employ a large number of observation random variables; however, estimation error increases with the number of variables, so that variables not contributing to the determination of the target variable can have a detrimental effect. In linear filtering, determination involves the correlation coefficients among the input and target variables. This paper discusses use of the more general coefficient of determination in nonlinear filtering. The determination coefficient is defined in accordance with the degree to which a filter estimates a target variable beyond the degree to which the target variable is estimated by its mean. Filter constraint decreases the coefficient, but it also decreases estimation error in filter design. Because situations in which the sample is relatively small in comparison with the number of observation variables are of salient interest, estimation of the determination coefficient is considered in detail. One may be unable to obtain a good estimate of an optimal filter, but can nonetheless use rough estimates of the coefficient to find useful sets of observation variables. Since minimal-error estimation underlies determination, this material is at the interface of signal processing, computational learning, and pattern recognition. Several signal-processing factors impact application: the signal model, morphological operator representation, and desirable operator properties. In particular, the paper addresses the VC dimension of increasing operators in terms of their morphological kernel/basis representations. Two applications are considered: window size for restoring degraded binary images; finding sets of genes that have significant predictive capability relative to target genes in genomic regulation.",
author = "Dougherty, {Edward R.} and Seungchan Kim and Yidong Chen",
year = "2000",
month = "10",
doi = "10.1016/S0165-1684(00)00079-7",
language = "English (US)",
volume = "80",
pages = "2219--2235",
journal = "Signal Processing",
issn = "0165-1684",
publisher = "Elsevier",
number = "10",

}

TY - JOUR

T1 - Coefficient of determination in nonlinear signal processing

AU - Dougherty, Edward R.

AU - Kim, Seungchan

AU - Chen, Yidong

PY - 2000/10

Y1 - 2000/10

N2 - For statistical design of an optimal filter, it is probabilistically advantageous to employ a large number of observation random variables; however, estimation error increases with the number of variables, so that variables not contributing to the determination of the target variable can have a detrimental effect. In linear filtering, determination involves the correlation coefficients among the input and target variables. This paper discusses use of the more general coefficient of determination in nonlinear filtering. The determination coefficient is defined in accordance with the degree to which a filter estimates a target variable beyond the degree to which the target variable is estimated by its mean. Filter constraint decreases the coefficient, but it also decreases estimation error in filter design. Because situations in which the sample is relatively small in comparison with the number of observation variables are of salient interest, estimation of the determination coefficient is considered in detail. One may be unable to obtain a good estimate of an optimal filter, but can nonetheless use rough estimates of the coefficient to find useful sets of observation variables. Since minimal-error estimation underlies determination, this material is at the interface of signal processing, computational learning, and pattern recognition. Several signal-processing factors impact application: the signal model, morphological operator representation, and desirable operator properties. In particular, the paper addresses the VC dimension of increasing operators in terms of their morphological kernel/basis representations. Two applications are considered: window size for restoring degraded binary images; finding sets of genes that have significant predictive capability relative to target genes in genomic regulation.

AB - For statistical design of an optimal filter, it is probabilistically advantageous to employ a large number of observation random variables; however, estimation error increases with the number of variables, so that variables not contributing to the determination of the target variable can have a detrimental effect. In linear filtering, determination involves the correlation coefficients among the input and target variables. This paper discusses use of the more general coefficient of determination in nonlinear filtering. The determination coefficient is defined in accordance with the degree to which a filter estimates a target variable beyond the degree to which the target variable is estimated by its mean. Filter constraint decreases the coefficient, but it also decreases estimation error in filter design. Because situations in which the sample is relatively small in comparison with the number of observation variables are of salient interest, estimation of the determination coefficient is considered in detail. One may be unable to obtain a good estimate of an optimal filter, but can nonetheless use rough estimates of the coefficient to find useful sets of observation variables. Since minimal-error estimation underlies determination, this material is at the interface of signal processing, computational learning, and pattern recognition. Several signal-processing factors impact application: the signal model, morphological operator representation, and desirable operator properties. In particular, the paper addresses the VC dimension of increasing operators in terms of their morphological kernel/basis representations. Two applications are considered: window size for restoring degraded binary images; finding sets of genes that have significant predictive capability relative to target genes in genomic regulation.

UR - http://www.scopus.com/inward/record.url?scp=0034302007&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=0034302007&partnerID=8YFLogxK

U2 - 10.1016/S0165-1684(00)00079-7

DO - 10.1016/S0165-1684(00)00079-7

M3 - Article

AN - SCOPUS:0034302007

VL - 80

SP - 2219

EP - 2235

JO - Signal Processing

JF - Signal Processing

SN - 0165-1684

IS - 10

ER -