Strong feature sets from small samples

Seungchan Kim, Edward R. Dougherty, Junior Barrera, Yidong Chen, Michael L. Bittner, Jeffrey M. Trent

Research output: Contribution to journalArticlepeer-review

79 Scopus citations


For small samples, classifier design algorithms typically suffer from overfitting. Given a set of features, a classifier must be designed and its error estimated. For small samples, an error estimator may be unbiased but, owing to a large variance, often give very optimistic estimates. This paper proposes mitigating the small-sample problem by designing classifiers from a probability distribution resulting from spreading the mass of the sample points to make classification more difficult, while maintaining sample geometry. The algorithm is parameterized by the variance of the spreading distribution. By increasing the spread, the algorithm finds gene sets whose classification accuracy remains strong relative to greater spreading of the sample. The error gives a measure of the strength of the feature set as a function of the spread. The algorithm yields feature sets that can distinguish the two classes, not only for the sample data, but for distributions spread beyond the sample data. For linear classifiers, the topic of the present paper, the classifiers are derived analytically from the model, thereby providing an enormous savings in computation time. The algorithm is applied to cancer classification via cDNA microarrays. In particular, the genes BRCA1 and BRCA2 are associated with a hereditary disposition to breast cancer, and the algorithm is used to find gene sets whose expressions can be used to classify BRCA1 and BRCA2 tumors.

Original languageEnglish (US)
Pages (from-to)127-146
Number of pages20
JournalJournal of Computational Biology
Issue number1
StatePublished - 2002
Externally publishedYes


  • Cancer
  • Classification
  • Gene expression
  • Perceptron

ASJC Scopus subject areas

  • Computational Mathematics
  • Genetics
  • Molecular Biology
  • Computational Theory and Mathematics
  • Modeling and Simulation


Dive into the research topics of 'Strong feature sets from small samples'. Together they form a unique fingerprint.

Cite this