Rough Set based ensemble classifier

Speaker: C. A. Murthy
Co-authors: Suman Saha and S. K. Pal

Abstract:

Combining the results of a number of individually trained classification systems to obtain a more accurate classifier is a widely used technique in pattern recognition. Here, we have introduced a rough set based meta classifier to classify web pages. The proposed method consists of two parts. In the first part, the output of every individual classifier is considered for constructing a decision table. In the second part, rough set attribute reduction and rule generation processes are used on the decision table to construct a meta classifier. It has been shown that (1) the performance of the meta classifier is better than the performance of every constituent classifier and, (2) the meta classifier is optimal with respect to a quality measure defined here. Some other theoretical results on this classifier and comparison with Bayes decision rule are also described here. There are several ensemble classifiers available in literature like Adaboost, Bagging, Stacking. Experimental studies show that the meta classifier improves accuracy of classification uniformly over some benchmark corpora and beats other ensemble approaches in accuracy by a decisive margin, thus demonstrating the theoretical results. Apart from this, it reduces the CPU load compared to other ensemble classification techniques by removing redundant classifiers from the combination.

An approach named RSM( Rough Set Meta classifier) is proposed, which is designed to extract decision rules from trained classifiers ensemble that perform classification tasks. RSM utilizes trained classifier ensembles to generate a number of instances containing prediction made by individual classifier as condition attribute values and actual class as decision attribute value. Then RSM constructs a decision table with one instance in each row. Once the decision table is constructed rough set attribute reduction is performed to determine core and minimal reduct. The combination of classifiers corresponding to the features of minimal reduct is then taken to form classifier ensemble for RSM classifier system. Now from the minimal reduct obtained in the previous step we compute decision rules by finding mapping between decision attribute and condition attributes. These decision rules obtained by rough set technique are then used to perform classification task. Our approach tries to solve the problem of representing less redundant ensemble of classifies and the problem of making reasonable decision from the predictions of ensemble classifiers, by using rough set attribute reduction and rough set decision rule generation on a granular meta data generated by base classifiers from input data.