Linear discriminant analysis : a detailed tutorial

Gaber, T; Tharwat, A; Ibrahim, A; Hassanien, AE

Abstract

Linear Discriminant Analysis (LDA) is a very common
technique for dimensionality reduction problems as a preprocessing
step for machine learning and pattern classification
applications. At the same time, it is usually used as a
black box, but (sometimes) not well understood. The aim of
this paper is to build a solid intuition for what is LDA, and
how LDA works, thus enabling readers of all levels be able
to get a better understanding of the LDA and to know how to
apply this technique in different applications. The paper first
gave the basic definitions and steps of how LDA technique
works supported with visual explanations of these steps.
Moreover, the two methods of computing the LDA space, i.e.
class-dependent and class-independent methods, were explained
in details. Then, in a step-by-step approach, two numerical
examples are demonstrated to show how the LDA
space can be calculated in case of the class-dependent and
class-independent methods. Furthermore, two of the most
common LDA problems (i.e. Small Sample Size (SSS) and
non-linearity problems) were highlighted and illustrated, and
state-of-the-art solutions to these problems were investigated and explained. Finally, a number of experiments was conducted
with different datasets to (1) investigate the effect of
the eigenvectors that used in the LDA space on the robustness
of the extracted feature for the classification accuracy,
and (2) to show when the SSS problem occurs and how it can
be addressed.

Citation

Gaber, T., Tharwat, A., Ibrahim, A., & Hassanien, A. (2017). Linear discriminant analysis : a detailed tutorial. Ai communications : the European journal on artificial intelligence, 30(2), 169-190. https://doi.org/10.3233/AIC-170729

• Journal Article Type: Article
• Publication Date: Jan 1, 2017
• Deposit Date: Aug 19, 2019
• Publicly Available Date: Aug 19, 2019
• Journal: AI Communications
• Print ISSN: 0921-7126
• Publisher: IOS Press
• Volume: 30
• Issue: 2
• Pages: 169-190
• DOI: https://doi.org/10.3233/AIC-170729
• Publisher URL: http://dx.doi.org/10.3233/AIC-170729
• Related Public URLs: https://content.iospress.com/journals/ai-communications/Pre-press/Pre-press