Stability of Model Explanations in Interpretable Prototype-based Classification Learning

Introduction

Learning vector quantization (LVQ) as originally proposed by Kohonen  and mathematically justified as the Generalized LVQ (GLVQ)  constitutes a classifier for multiple class learning of vector data $\bold{x} \in \R^n$ based on the nearest prototype principle (NPP). GLVQ is known to be a robust classifier maximizing inherently the hypothesis margin during training.  GLVQ can be combined with class related data embedding learning by a linear transformation $\bold{\Omega} \bold{x} \in \R^m$ proposed as Generalized Matrix LVQ (GMLVQ).  The basic prototype learning scheme distributes the class dependent prototype vectors $\mathcal{P} = \{ \bold{p}_1, \ldots, \bold{p}_M \} \subset \R^m$, where the prototypes are equipped with class labels $c (\bold{p}_k) \in \mathcal{C}$ indicating their class responsibilities such that each class is represented by at least one prototype. Both, prototype and embedding learning usually is realized by stochastic gradient descent learning using an approximated overall classification error. 

Model interpretation is an important aspect to prefer the shallow GMLVQ over deep networks.  Thus, model certainty and stability are strongly desired properties such that general causal implication from model inspection could be drawn about model decisions in terms of feature contributions and relations.  Yet, from a machine learning perspective there is some evidence that there are many close-to-optimum-solutions (CTOS).  Those different CTOS may realize their model performance according to substantially varying decision strategies. Hence, resulting generalizations regarding causality are crucial, which are, however, frequently demanded and expected from a user perspective.  We present respective qualitative considerations for GMLVQ. 

While the resulting performance remains almost similar, it is close to the optimal solution (COST). This is important for understanding how to interpret causal models. What is more, this research is also related to a better understanding of how machines learn, which can be very different from how humans learn and understand things.

Description of the Experiments and Results

Let $d_\bold{\Omega} (\bold{x}, \bold{p}_k) = (\bold{\Omega} \bold{x} − \bold{p}_k)^\intercal \, (\bold{\Omega} \bold{x} − \bold{p}_k)$ be the prototype-to-data dissimilarity used for NPP in GMLVQ using an embedding matrix $\bold{\Omega} \in \R^{m \times n}$ with $m \le n$.  From a trained model we can derive the classification correlation matrix (CCM) $\bold{\varLambda_\Omega} = \bold{\Omega}^\intercal$ $\bold{\Omega}$ describing feature correlations supporting the classification and the corresponding classification information profile (CIP) $\bold{\gamma_\Omega} = (\gamma_1^\bold{\Omega}, \ldots, \gamma_n^\bold{\Omega})^\intercal$ with $\gamma_k^\bold{\Omega} = \sum_j \big| [\bold{\varLambda_\Omega}]_{k,j} \big|$ estimating feature importance. Further, the relevance profile $\bold{\lambda_\Omega} = (\lambda_1^\bold{\Omega},\ldots, \lambda_n^\bold{\Omega})^\intercal$ with $\lambda_k^\bold{\Omega} = [\bold{\varLambda_\Omega}]_{k,k}$ is a simple measure for feature relevance.  Thus, these quantities give qualitative information about the decision process of the model. Further, we compare the CIP-profiles with corresponding Shapley-values (Shap) for feature relevance evaluations  whereas permutation feature invariance (PFI) values are easy to compute for feature sensitivity evaluation.  Shap is calculated with respect to the GMLVQ cost function (ShapCosts) and with respect to the output (ShapOut - changes for the predicted labels). Similarity is judged by the Spearman correlations. 

We trained GMLVQ on several data sets and present a typical result: We considered a 11-dimensional medical tumor metabolite (MTM) data set (235 samples / 3 classes) adopted and modidied from.  Five-fold cross validation yields high accuracies for each fold with only small deviations for each data set. Yet, the visual analysis of the $\bold{\varLambda_\Omega}$-matrices reveals substantial deviation within between the folds as depicted in Fig. 1. Thus, we obtained qualitatively different CTOS in the individual folds but with approximately the same performance. This underpins the above statement regarding high variability of CTOS. In consequence, model interpretation is only valid for the specific model in use. Moreover, the correlation analysis of the feature relevance profile yields the observation that Shapley-values seem to be approximable by easier to generate profiles like CIP.