Top 10 Data Mining Mistakes

March 2004 Section Meeting

When: Tuesday, March 16, 2004 - 7-9 PM

Speaker: Dr. John Elder, Elder Research, Inc. Click here for his biographical information.

Click Here to view a PDF of the presentation (2765 K).

Abstract:

Data Mining is a promising interdisciplinary field whose practitioners discover key patterns in historical data by which to make effective decisions today. It gives one a "crystal ball" which has enhanced performance in noisy, data-rich fields ranging from the stock market to credit risk assessment, and marketing to fraud detection.

But, Data Mining is still as much an art as a science, providing one many convenient ways to do wrong things with one's data. Case studies of (often personal) errors -- both simple and complex -- will be drawn from real-world consulting engagements. "Best Practices" for Data Mining will be (accidentally) illuminated by their (rarely described) opposites. These common errors range from allowing "after-the-fact" variables into the pool of candidate inputs, to subtly inflating results, through early up-sampling.

You'll hear cautionary tales of endangered projects and embarrassed teams -- and, thereby perhaps, avoid such a fate yourself.

Biographical Information:

Dr. John Elder heads a Data Mining consulting firm with offices in Charlottesville, Virginia, and Washington, DC. Elder Research, Inc. (ERI) focuses on investment and commercial applications of pattern discovery, including cross-selling, stock selection, image recognition, biometrics, drug efficacy, credit scoring, market timing, and fraud detection.

John earned Electrical Engineering degrees from Rice University, and a PhD in Systems Engineering from the University of Virginia, where he is currently an adjunct professor, teaching Optimization. He spent five years in high-tech defense consulting, four heading research at an investment management firm, two in Rice's Computational & Applied Mathematics department, and has led ERI since 1995.

Dr. Elder is active in Statistics and Engineering conferences and boards and is a Program co-chair of the 2004 Knowledge Discovery and Data Mining conference. Since Fall of 2001, he's been honored to serve on a panel guiding technology for the National Security Agency.

Click Here for more information on Elder Research, Inc.