Single

A gradient boosting decision tree model is a powerful machine learning method that iteratively constructs decision trees to form an additive ensemble model. The method uses the gradient of the loss function to improve the model at each iteration step.Inspired by the databaseliterature, we exploit bitset and bitslice data structures in order toimprove the run time efficiency of learning the trees. We can use thesestructures in two ways. First, they can represent the input data itself.Second, they can store the discretized gradient values used by the learningalgorithm to construct the trees in the boosting model. Using thesebit-level data structures reduces the problem of finding thebest split, which involves counting of instances and summing gradientvalues, to counting one-bits in bit strings.Modern CPUs can efficiently count one-bits using AVX2 SIMD instructions.Empirically, ourproposed improvements can result in speed-ups of 2 to up to 10 times ondatasets with a large number of categorical feature without sacrificingpredictive performance.

Before constructing a classifier, we should examine the data to gainan understanding of the relationships between the variables, toassist with the design of the classifier. Using multi-label datarequires us to examine the association between labels: its multi-labelness. We cannotdirectly measure association between two labels, since the labels'relationships are confounded with the set of observation variables.A better approach is to fit an analytical model to a label withrespect to the observations and remaining labels, but this mightpresent false relationships due to the problem of multicollinearitybetween the observations and labels.In this article, we examine the utility of regularised logisticregression and a new form of split logistic regression for assessingthe multi-labelness of data.We find that a split analytical model usingregularisation is able to provide fewer label relationships when norelationships exist, or if the labels can be partitioned. We alsofind that if label relationships do exist, logistic regression with l_1 regularisationprovides the better measurement of multi-labelness.

We present an approach to explain the decisions of black box models for image classification. While using the black box to label images, our explanation method exploits the latent feature space learned through an adversarial autoencoder. The proposed method first generates exemplar images in the latent feature space and learns a decision tree classifier. Then, it selects and decodes exemplars respecting local decision rules. Finally, it visualizes them in a manner that shows to the user how the exemplars can be modified to either stay within their class, or to become counter-factuals by "morphing" into another class. Since we focus on black box decision systems for image classification, the explanation obtained from the exemplars also provides a saliency map highlighting the areas of the image that contribute to its classification, and areas of the image that push it into another class. We present the results of an experimental evaluation on three datasets and two black box models. Besides providing the most useful and interpretable explanations, we show that the proposed method outperforms existing explainers in terms of fidelity, relevance, coherence, and stability.

Association rules are among the most important concepts in data mining. Rules of the form X -> Y are simple to understand, simple to act upon, yet can model important local dependencies in data. The problem is, however, that there are so many of them. Both traditional and state-of-the-art frameworkstypically yield millions of rules, rather than identifying a small set of rules that capture the most important dependencies of the data.In this paper, we define the problem of association rule mining in terms of the Minimum Description Length principle.That is, we identify the best set of rules as the one that most succinctly describes the data.We show that the resulting optimization problem does not lend itself for exact search, and hence propose Grab, a greedy heuristic to efficiently discover good sets of noise-resistant rules directly from data.Through extensive experiments we show that, unlike the state-of-the-art, Grab does reliably recover the ground truth.On real world data we show it finds reasonable numbers of rules, that upon close inspection give clear insight in the local distribution of the data.