Isabelle GUYON
Clopinet Consulting
955 Creston road
Berkeley, CA 94708
(510) 524 62 11
isabelle@clopinet.com
http://www.clopinet.com
2001:
BIOwulf Technologies, Berkeley, California.
Vice President.
Assembled a team of world recognized experts in Machine Learning and
SVMs. Initiated several research projects with Universities and Industry
to analyze genomics and proteomics data, including DNA microarray data
and mass spectrometric data.
1995:
AT&T Bell Laboratories, Holmdel, New Jersey.
Information services department (headed by Dr. John Denker).
Research on search engines for World Wide Web applications.
1989 to 1994:
AT&T Bell Laboratories, Holmdel, New Jersey.
Adaptive systems department (headed by Dr. Larry Jackel).
Research on artificial neural networks and the theory of learning,
with application to pattern recognition and in particular on-line handwriting
recognition.
1981-1985:
Ecole Supérieure de Physique et Chimie, Paris, France.
Engineering diploma in physics and chemistry in June 1985.
Master degree in electrical engineering in June 1985.
Consulting project, 1996-1997: Signal filtering of an electronic
pen using accelerometers and gyroscopes.
David Stork
Ricoh Silicon Valley
2882 Sand Hill Road, Suite 115
Menlo Park, CA 94025-7022, USA
1 +(415) 496-5720
stork@rsv.ricoh.com
Research on Neural Networks and Handwriting Recognition, 1989-1994.
Department head:
Doctor Larry Jackel
AT&T Labs Research
Room 3-140
100 Schltz Dr
Red Bank, NJ 07701-7033, USA
1 + (732) 345-3370
ldj@research.att.com
Research on Neural Networks, 1985-1988.
Thesis director:
Professor Gérard Dreyfus
Laboratoire d'Electronique
ESPCI
10, rue Vauquelin
75005 Paris
France
33 + (1) 47-07-13-93
dreyfus@neurones.espci.fr
From a practical standpoint, I demonstrated that classical recognition algorithms such as "nearest neighbors'' outperform the best Hopfield networks in a handwritten digit recognition benchmark. I proposed a novel perceptron architecture (one layer neural network) based on pairwise separations between classes. My architecture outperformed nearest neighbors and all other classical statistical techniques. It would today be called an error correcting code method: the outputs of the perceptron being redundantly encoded, the error rate is reduced substantially with some simple postprocessing.
I pursued the last part of my thesis work, devoted to learning sequences of data, when I joined Bell Laboratories in 1989. I explored various applications in speech recognition, handwriting recognition and control. It became apparent that an "all neural network'' solution is not viable. I started working on hybrids of neural networks and hidden Markov models.
The first system I developed, which is described in "Design of a Neural Network Character Recognizer for a Touch Terminal'' (1991) has inspired many presently used systems and was granted a patent in 1992. I won in 1992 an AT&T award for leadership in pen computing. In collaboration with other people from my department, we won in 1993 a benchmark organized by AT&T GIS (ex-NCR) in which we were compared with the best commercially available systems for on-line handprinted sentences. The commercialization of our system was started by AT&T but abandoned when AT&T phased out of the pen computing business.
Following a project that I initiated with a summer student at AT&T in 1992, a collaborator following one of my original ideas implemented a signature verification system that won a best paper award at the NIPS conference1993.
In my consulting practice, since 1996, my customers have recognized me as a world expert in pen computing by hiring my services. I have conducted several projects of pen computer interfaces (Ricoh, 1996; Baron, 1996; University of Dublin, 1997). One of my recent projects was the design and implementation of a program to teach cursive handwriting to children (Penmanship, 1997-1998). The resemblance of signature verification and fingerprint verification problems allowed me to design and implement a fingerprint verification system (WhoVision, 1999). I contributed to the design of an XML format for storing and annotating handwriting pieces of evidence to be analyzed by forensic experts (Wanda, 2003).
In a paper entitled "Structural Risk Minimization for Character Recognition'' (1992), we developed a practical method to control the VC dimension.
We went further and proposed, in a paper entitled "A training algorithm for optimal margin classifiers"(1992), an novel algorithm based on the principles of learning theory. This algorithm now called "Support Vector Machines" or SVMs has known considerable developments in the past few years (see http://www.support-vector.com/). It is considered to be the successor of Neural Networks and has dozens of successful applications (see http://www.clopinet.com/isabelle/Projects/SVM/applist.html). In a recent article entitled "Comparison of Classifier Methods: a Case Study in Handwriting Digit Recognition'', this algorithm reaches, without any a priori knowledge about the task at hand (i.e. working directly on pixel images), the same performance as the best system, a neural network with sophisticated architecture which has been optimized over several years of human effort. Our invention was granted a patent in 1997. It has become a standard textbook technique that is described in the new edition of the classical textbook "Pattern Classification'' by Duda, Hart and Stork, 2001.
In a paper published in 1998, ``What Size Test Set Gives Good Error Rate Estimates'', we address the difficult problem of finding an optimum split of the data into training set and test set. My present research addresses the difficult problem of input variable selection.
The classical measure of performance of linguistic engines is the cross-entropy on a large test set, that is the per character length of the shortest code to encode the prediction errors. I ran experiments to compare our model with a model designed at IBM which is the best model reported in the literature. Using a standard benchmark (the Brown corpus) our system reached approximately 2 bits per character, compared to 1.75 for the IBM system and 1 for humans (as estimated by Shannon in 1950). While our performance are slightly worse, our model has the tremendous advantage that it is 200 times smaller than the IBM system (160,000 parameters).
In our design, we controlled the tradeoffs among memory, speed and accuracy by varying a single information-theoretic parameter. Finite state automata composition allowed us to reconfigure flexibly the language model according to the needs of different applications, without having to retrain the core of the model.
Combining the model with on-line neural network handwriting recognizers designed in our group showed substantial reduction in error rate (up to a factor of 2 to 3 depending on the writer).
While working with BIOwulf (1999 to 2002), I have been confronted with a wide variety of problems of medical diagnosis and prognosis and have been collaborating with a number of universities, biotech companies and pharmaceutical companies. Central to this research is the problem of variable or feature selection: determining which input variables contribute most to making correct predictions. Applied to diagnosis or prognosis, variable selection is called "biomarker discovery". Drug target discovery is also a related problem that benefits from the identification of valid biomarkers.
The initial results of gene selection using SVM Recursive Feature Elimination, a technique of feature selection that I invented, demonstrated that predictors using only a few genes can be built to diagnose Leukemia, colon cancer, lymphoma, and protate cancer ("Gene Selection for Cancer Classification using Support Vector Machines", Machine Learning, 2002). The SVM prediction performance also compare favorably to other methods and our group won several benchmarks organized by our customers.
I am presently consulting for several companies designing new biomedical assays. In this way, I have direct access to good quality fresh data by having some control over experimental design and data collection. One of my main charters has been instrument characterization and quality control. At Pointilliste (2002 to present) I have been confronted with patterns of activities of antibody arrays. At Biospect (2002 to present) I am working on two dimensional patterns resulting form the combination of capillary electrophoresis and mass-spectrometry (CE-MS).
In the course of the pen computing project, I co-advised two PhD students
and one Masters student:
- Nada Matic graduated with honors in 1993 from the City University
of New York for a work entitled "Exploration in On-Line Handwriting Recognition''.
Her thesis comprises two parts: (1) innovative methods of computer aided
data cleaning and (2) writer adaptation.
- Markus Schenkel graduated with honors in 1994 from the Swiss Federal
Institute of Technology of Zürich. His work entitled "Handwriting
Recognition using Neural Networks and Hidden Markov Models'' presents the
design and study of a hybrid neural network and hidden Markov models for
cursive handwriting recognition.
- Claudia Medina obtained in 1995 a Masters degree from "Mills College,
California'' for her work entitled "Demographic and Linguistic Study of
Handwriting Recognition''. For her thesis she conducted a survey of the
uses of handwriting and a linguistic analysis of handwritten notes.
- Asa Ben Hur spend a year with me as a post-doc working on the applications
of clustering to DNA microarray data.
I have considerable experience in participating to and leading teleconference calls and even giving presentations over the telephone. I have scientific collaborators all over the world with whom I interact by electronic mail. Some of them I have never met in person. At BIOwulf, I have conducted a reading group meeting regularly to discuss scientific papers over the phone, with participants living on 3 continents and accross 4 different time zones.
In my consulting practice, I mostly work from home. My customers appreciate
that they do not need to provide me with office space and computer equipment.
They get a very clear understanding of the project progress because I thoroughly
document everything I do. They occasionally use me to work with remotely
located associates or to coordinate efforts of several people located at
different places.
(This is a partial list, total number of publications is 30, for a full list, see http://clopinet.com/isabelle/Papers/index.html)
Book and journal edited:
9. Advances in Pattern Recognition Systems using
Neural Network Technologies ,volume 7.
I. Guyon and P.S.P.
Wang, editors.
World Scientific, Series
in Machine Perception and Artificial Intelligence, Singapore, 1994.
10. Variable and feature selection.
I. Guyon and A. Elisseeff, editors.
Journal of Machine Learning Research, Volume 3,
March 2003. http://www.jmlr.org/papers/special/feature.html.
Patents granted:
11. Time delay neural network for printed and
cursive handwritten character recognition.
I. Guyon, J. S. Denker,
and Y. Le Cun.
US Patent 5,105,468.
1992.
12. Pattern recognition system using support
vectors.
B. Boser, I. Guyon,
and V. Vapnik.
US Patent 5,649,068.
1997.