iaeng international journal of computer science, 46:2, ijcs 46 2 … · 2019-06-01 · f measure =...

S

IAENG International Journal of Computer Science, 46:2, IJCS_46_2_21

(Advance online publication: 27 May 2019)

______________________________________________________________________________________

are needed to be designed specifically. In this paper,we study and review 12 states of the art techniques:SMOTE, AdaBoost, RUSBoost, EUSBoost, SMOTEBoost,MSMOTEBoost, DataBoost, Easy Ensemble, BalanceCas-cade, OverBagging, UnderBagging, SMOTEBagging formining imbalanced data and compare their performance on27 imbalanced datasets. In the rest of the paper has beenarranged as follows. In section II we have discussed thedifferent performance evaluator and their function. SectionIII presents different types of data balancing methods. Sec-tion IV presents the discussion on 12 imbalanced classclassification algorithms. In section V we illustrate the char-acteristics of the used real-world datasets that have beencollected from KEEL [9] data set repository. Section VIrepresents most significant outcome of our research. Wehave introduced some tables and figures to provide betterinsight of our experiments. The input parameters that hasbeen used in our experiments is given in table III. In tableVI, we have shown AUROC comparison. In figure 2 we haverepresented graphical view of table VI. Apart from that, wehave introduced another useful process that is G-mean intable VII to compare algorithms. In figure 3 represents theline charts of table VII. To prove the significance of theresearch, we have included some other important methodssuch as F-measure and the resulting value is mentioned intable VIII with graphical representation in figure 4. Differentexecution or processing time that is mentioned in table IXwith line charts in figure 5. Last but not least, the most crucialpart of this section is the independent AUCROC line chartsof algorithms and datasets (Randomly chosen 4 out of 27used dataset: PIMA, ecoli2, ecoli3, glass01) shows in figure6. Finally, we conclude this analysis in Section VIII.

II. THE CLASSIFIER PERFORMANCE EVALUATORS

The performance of machine learning algorithms is eval-uated by a standard evaluation matrix which is called confu-sion matrix, as illustrated in Figure 1 (for a 2 class problem).Here the columns are the predicted class and the rows arethe actual class. In the confusion matrix, TN is the numberof negative examples correctly classified (True Negatives),FP is the number of negative examples incorrectly classifiedas positive (False Positives), FN is the number of positiveexamples incorrectly classified as negative (False Negatives)and TP is the number of positive examples correctly classi-fied.

TABLE I: 2x2 Confusion Matrix

Confusion matrixPositive Prediction Negative Prediction

Positive class True Positive(TP) False Negative(FN)Negative class False Positive(FP) True Negative(TN)

The most common performance evaluation metrics relatedto imbalanced classes are: 1) Accuracy 2)Recall (sensitivity)3) Specificity 4) Precision 5) F-measure and 6) Geometricmean (g-mean)[4]. Accuracy is the ratio between truedecisions predict by a classifier. Sensitivity (also called TruePositive Rate) and specificity (also called True NegativeRate) are used to monitor the classification performance ofeach individual classifier and recognise positive and negativeexamples respectively. [10]. Precision is used in problems

interested in high performance in only one class and it isthe ratio between true positive examples. F- measure andG-mean are harmonic averages and geometric averages ofsensitivity and precision respectively. The geometric averageof sensitivity and specificity is known as G-mean 2 [11].

ROC curve introduces the true positive rate (TPR) alongthe y-axis and false positive rate (FPR) along the x-axis. Theclassifier which produces accurate result would have an areaunder the curve (AUROC) of 1, where TPR = 1 and FPR= 0. At various cut - off points, the curve displays the TPRand FPR of the classifier at a different range.

Accuracy =TP + TN

TP + FP + FN + TN(1)

TPRate(Sensitivity) =TP

TP + FN(2)

FPRate(specificity) =FP

FP + TN(3)

precision =TP

TP + FP(4)

G−mean =√TPrate× TNrate (5)

G−mean =√sensitivity × precision (6)

G−mean2 =√TPrate× TNrate (7)

G−mean2 =√sensitivity × specificity (8)

F −measure =2(precision.Sensitivity)

precision+ Sensitivity(9)

AUROC =TPrate+ TNrate

2(10)

The F1 score (also F-score or F-measure) is a weightedaverage evaluator that measure accuracy of a classifiers. Thisis a regular performance evaluation metrics interpreted asa better choice that combines precision and recall into asingle value, by giving equal weight on both class [12].The geometric average (G-mean) has been used by severalresearchers for evaluating classifiers on imbalanced datasetsespecially when performance of both classes is concernedand expected to be high at the same time. These measurespoint out the stability between classification performance onthe majority and minority class. Both G-mean and F1 scorereaches its best value at 1 (perfect precision and recall) andworst at 0 [13]. In the case of imbalanced classificationissues, insistence is established notably on the predictiveaccuracy of minority/positive class while having accuracyfor the majority/negative class. This would correlate to highTPR and low FPR, and thus reflected by a high AUROC.

III. DATA BALANCING METHODS

A. Sampling Technique

The customary evaluation procedures were unable tomeasure the exact model performance mostly when working



______________________________________________________________________________________

with real-life imbalanced datasets. Sampling techniquessolve the class imbalanced classification problem withthe redistribution of the imbalanced datasets in order toachieve an almost equal number of instances in two ways,which are called undersampling and oversampling methods.Sometimes a performance can be elevated by combiningboth the under and oversampling method which can beidentified as hybrid approaches such as SMOTE withTOMEK or ENN method [5].Various intellectual algorithmshave been proposed related to sampling technique [14],like SMOTE, borderline SMOTE, and Wilson’s editing[15]. Hulse [16] observes the performance of diverse datasampling methods (around seven) using learning algorithmsincluding investigational data sets, by discovering that thetwo most successful sampling algorithms are RUS andSMOTE.

1) Over-Sampling: The over-sampling method works withminority class instances by creating synthetic instanceswhich causes no lose of any potential information. It canbe done in two ways: (1) Random Over-sampling, and (2)Informative Over-sampling. Random over-sampling methodsbalance the datasets by randomly over-sampling the minorityclass instances.Whereas, informative over-sampling synthet-ically generates and adds the minority class instances intothe dataset. The disadvantage of this method is overfitting ofdatasets as it adds replicated data from the original dataset.Some well-known over-sampling Methods are:

SMOTE (Synthetic Minority Over-sampling Technique)- this sampling method combines under-sampling of themajority class instances with over-sampling of the minorityclass instances [4]. Inspired by the favorable outcome ofsome popular concepts of using artificial instances likeSMOTE [4], SMOTEBoost [4], and DataBoost [17], thistheory proposes an adaptive method called ADASYN(Adaptive Synthetic Sampling Approach for ImbalancedLearning) [18]. With Borderline-SMOTE-I, to attainexcellent prediction, the borderline instances of the minorityclass are over-sampled only. This sampling concept isdifferent from the existing over-sampling ideas where thenegligible instances or the arbitrary subset of the minorityclass are over-sampled [19] [20] [4]. Apart from that, Wewill mention another significant method called borderlineOver-sampling in the Feature Space (BOSFS), that manageover-sampling method by creating novel synthetic minorityinstances with the existing borderline instances. The SVMclassifier achieves higher recognition performance with thisBOSFS method using the Euclidean distance, especially forthe minority class instances [21].

2) Under-Sampling: The under-sampling method workswith majority class instances, which removes instances ran-domly from a majority class to make the dataset balanced.It is best to use when the dataset is too big. Under-samplingmethods are of two types: (1) Random under-sampling,and (2) Informative under-sampling. Random under-samplingmethod randomly chooses instances from the majority class,which later eliminates when the dataset gets balanced.Removing instances randomly may causes loss importantinformation. On the other hand, informative under-samplinguses a pre-specified selection criterion to remove the majority

class instances to balance the dataset [22].

Fig. 1: Sampling Techniques.

B. Cost-sensitive Learning Method

The cost-sensitive learning (CSL) takes the misclassifica-tion costs into account in the learning process to minimizetotal cost. It is also a common approach to solve the classimbalanced problem [23]. The objective of the cost-sensitivelearning is to achieve high accuracy to classify minority classinstances. However, it is difficult to find the misclassificationcost in different iterations as it depends on different types oferrors.

C. Ensemble Method

Ensemble learning is the process of combining multipleclassifiers to form a strong classifier to classify new instanceswith high prediction accuracy. Examples are RandomForest,Bagging, and Boosting. While constructing ensemble meth-ods the most challenging task to be faced by the classifiersare: (1) the coalescence of the classifiers to be used, (2)the base classifiers used for ensemble must be naive toovercome overfitting, and (3) the base learners used shouldbe as accurate as possible, and as distinct as possible.

IV. IMBALANCED CLASS CLASSIFICATION METHODS

In this section, we discuss the following machinelearning algorithms: SMOTE, AdaBoost, RUSBoost,EUSBoost, SMOTEBoost, MSMOTEBoost, DataBoost, EasyEnsemble, BalanceCascade, OverBagging, UnderBagging,SmoteBagging that are widely used in many real-worldimbalanced data classification applications [24].



______________________________________________________________________________________

1) SMOTE: This sampling method combines under-sampling of the majority class with a notable form ofover-sampling of the minority class [4]. The concept of themethod is developed with various investigational datasets,C4.5 as the base classifier, and Naive Bayes classifierincluding the AUROC curve [25]. This is an over-samplingapproach where not only the instances are over-sampled butalso the minority class builds some synthetic samples. Oneof the well-established examples of this approach is theidentification of the optical character [26], where the authorsproduced additional training data by performing particularoperations on real data. To perturb the training data theauthors used operations like rotation and skew. Assume that,the total adjacent instances of minority class is K. Accordingto the quantities of total instances needed, adjacent instancesamong the K (where k=5) instances are selected at random.For example, if the required over-sampling amount is 400%,then only 4 are selected from the 5 adjacent instancesand 1 instance is produced in each direction. Syntheticinstances are created by taking the difference between theadjacent instances and featured instances. Now assume 2random numbers between 0 and 1 and we multiply thisdifference by the random number. Then add the resultingvalue to the feature vector. The minority class become moregeneral when this method successfully forces the decisionregion. SMOTE algorithm which is the latest approach toover-sampling technique enhance the appropriateness of aclassifier for a minority class. The combination of SMOTEand under-sampling method performs better than ordinaryunder-sampling technique. The novel approaches withSMOTE are examined using different datasets, with varyinglevels of imbalance. Merging SMOTE and under-samplingtechnique also functions better, based on possession in theROC space. As SMOTE makes the setting territory largerand unspecified, it makes the classifier very much uniquethan that of other classifiers.

2) AdaBoost: AdaBoost is the most popular and effectiveensemble classifier, with key functions to improve theperformance of weak learners [27], [7]. During each ofits iteration, instance weights are changed to see how theclassification rate changes with focuses on misclassifiedinstances in the next iteration. AdaBoost uses the weightedvote strategy to classify the unlabeled instances. In the classimbalanced situation, the AdaBoost algorithm performsefficiently as each of its iterations. Minority class instancesare misclassified and given higher weights to considerin subsequent iterations. AdaBoost is a most desired andused boosting algorithm, which assumes a sequence ofclassifiers and incorporates the vote of each isolate classifierfor classifying an unknown or known instances [28]. Thealgorithm performs efficiently with each of its iterations,the misclassified minority class instances are given higherweights. During every iteration of this algorithm, an infirmpresumption is made by the foundation learner of theclassifier. If the instance accurately classified its weight willbe reduced and if failed to classify then the opposite willbe done which means the weight will be increased. Thesignificant characteristic of AdaBoost is that the approachmainly focuses on troublesome data points which have beenmisclassified by the weak classifier. The weak learner need

no advanced knowledge and an optimally weighted majorityvote uses by the classifier.

3) RUSBoost: RUSBoost is an under-sampling approach,which randomly removes the majority class instances tomake the dataset balanced [29], [19]. It is an ensembleclassifier as it combines the random under-samplingtechnique with a boosting approach (AdaBoost.M2algorithm). It creates a balanced dataset using a randomunder-sampling technique in each boosting iteration andconsiders the majority-voting technique to classify newinstances. RUSBoost is an ensemble classifier which is anextended version of SMOTEBoost that produces a quickerand easier approach with an achievement which is normallyperfect and sometimes more than that of SMOTEBoost.Seiffert et al. [29] say that RUSBoost is an unprecedentedmongrel data sampling technique, which has a strongoutline to enhance the performance of models. The accuracyof RUSBoost is measured with SMOTEBoost [19]. Thecommon thing among the classifiers is that these twoclassifiers RUSBoost and SMOTEBoost introduce datasampling with the popular AdaBoost classifier. RUS, asampling technique that arbitrarily dispels instances fromthe leading class [30] used by RUSBoost. On the otherhand, SMOTEBoost [4] which constructs new minority classinstances applies the SMOTE technique. The uses of RUStechnique into the boosting process are its intelligibility,fastness, and performance. On the other side, SMOTE isa puzzled and extended data sampling technique whichmakes the SMOTEBoost more complex and suffers fordisadvantages of prolonged model training time as comparedto RUSBoost. The main advantage of RUS classifier isthat it reduces the period necessary for constructing amodel, especially when constructing an ensemble of models.Though there is no universally conventional optimal classdistribution, a balanced (50: 50) distribution is often thoughtto be near optimal [20]. Moreover, when instances of theminority class are extremely uncommon, a ratio closer to35: 65 (minority: majority) may result in better classificationperformance.

4) EUSBoost: Enhancing ensembles for extremelyimbalanced datasets by the evolutionary under-samplingmethod is called as EUSBoost [31]. It is based on theRUSBoost method as it combines random under-samplingwith boosting techniques [5], [32]. EUSBoost starts withthe under-sampling technique that continued until thecurrently finest under-sampled uplifted. This method iscomputationally more expensive compared to the RUSBoostalgorithm as it executes EUS in all iterations of boosting. Inreal-world applications, the usefulness and aptness of EUShave been proved successfully [33]. But the main drawbackof EUSBoost is that it seeks perfect base learners that causesdiversity loss in the final output. EUSBoost training phaseis computationally more expensive than other methods. Thisis due to EUS, which is executed in all iterations of Boosting.

5) SMOTEBoost: SMOTEBoost combines the oversam-pling technique SMOTE with the AdaBoost algorithm, whichoutperforms on both the SMOTE and AdaBoost algorithmsfor learning from imbalanced data [34]. It acquires a balanced



______________________________________________________________________________________

dataset by applying SMOTE in each round of iteration of theboosting process [4].

This is an extremely potential sampling/boosting methodto learn from imbalanced data that outperform on bothfor SMOTE and AdaBoost algorithms. Before building theweak hypothesis during each round of iteration, a morebalanced training data set SMOTE is applied to the trainingdata. Hence, the algorithm for SMOTEBoost is analogous tothat of RUSBoost. At this point, the application of SMOTEhas two defects that RUSBoost is designed to overcome.First, it enhances the complexity of the algorithm. SMOTEmust find the k nearest neighbors of the minority classexamples and have to make an educated guess betweenthem to make new instances. RUS, contrariwise, simplyeliminate the majority class instances indiscriminately.Secondly, SMOTE requires an elongated time to train themodel, as it is an oversampling technique. When uses largertraining datasets, many models require to build for theaccurate output which produces elongated training time.Whereas, shorter model training times will be required forsmaller training data sets [4]. However, this causes thepossibility of creating redundancy. If only the borderlineand noisy majority instances are removed then the selectionprocedures try to find a representative subset of the majoritysamples. As a result, the complexity of the algorithm isincreased and the training time of the model have prolongedas well, which makes the algorithm more complex than RUS.

6) MSMOTEBoost: MSMOTEBoost combines MSMOTE(Modified Synthetic Minority Over-sampling Technique) andAdaBoost algorithm [7]. MSMOTE is the modified andimproved form of SMOTE method. MSMOTE categorizesthe minority class instances into three types depending on thedistances between the instances: border instances, securityinstances and noisy instances [35]. By using the nearestneighbors method MSMOTE algorithm generates syntheticinstances. The classification of MSMOTEBoost is moreaccurate than SMOTEBoost for classifying minority classinstances [36]. MSMOTE generates synthetic samples bycalculating the space between each minority samples usingthe nearest neighbor technique. MSMOTE Technique hasmaintained the following strategy: firstly, the algorithm arbi-trarily selects a data point from the nearest neighbor instancesfor the security instances. After that, the nearest neighbor forthe border instances is selected and doesn’t perform anythingfor latent noise instances. It is mandatory for this methodto compute the space among the instances of a negligibleclass and all the instances in order to judge the samplestype. Security instances can improve the function of theclassifier. Whereas, Noisy instances reduce the performanceof a model and border instances are very hard to classifyfor the model. The experimental analysis has proved that theprediction accuracy of MSMOTE model is outperformed bySMOTEBoost in case of negligible class [36].

7) DataBoost: The DataBoost-IM is an ensembleclassifier that amalgamates data creation and boostingprocesses to achieve high classification accuracy to classifyboth the majority and minority class instances withoutomitting the majority and minority classes [17]. It builtwith the following three phases. Step1, an equal weightis assigned to each training instances. Here, the original

training set is applied to build the initial classifier. Step 2,difficult instances are identified and for each of these coreinstances, a set of artificial instances is created. Finally, inStep 3, the artificial majority and minority class instancesare merged with the original training set to create a balanceddataset.

8) EasyEnsemble: EasyEnsemble is to build classifiersto lead the sampling process for subsequent classifiers andto further utilize the majority class instances disregardedby under-sampling [37]. It has the advantage of combiningboosting and bagging methods with data balancing method.The core concept of EasyEnsemble is straightforward andin some direction, it is analogous to the balanced RandomForest algorithm [38]. The final output of this method isan individual ensemble, though it acts like an ensembleof ensembles. Diverse researches [39], [40], [41], [42]amalgamate various ensemble techniques to attain strongergeneralization. MultiBoosting [40], [41] amalgamatesboosting and bagging [28] through applying boostedensembles as foundation learners. Different ensemblestrategies are combined to release a better solution likeStochastic Gradient Boosting [43] and Cocktail Ensemble[42].

9) BalanceCascade: BalanceCascade uses an under-sampling process to combine weak classifiers [44]. In thismethod, the sequential reliance among classifiers is pre-dominantly utilized for lessening the duplicate informationin the majority class. It conducts instance space for theunder-sampling method to pursue proper knowledge [23].To be successful in the rapid testing speed test usually thebalanced cascade classifier is an appropriate choice [37].BalanceCascade omits the duplicate instances in the majorityclass and confines instance difference to pursue convenientknowledge. Both BalanceCascade and EasyEnsemble areanalogous to their structures. The most popular technique iscalled stacking, [44] which combines weak classifiers withEasyEnsemble and BalanceCascade.

EasyEnsemble and BalanceCascade both share a commonstrategy where they intentionally enhance the weights ofthe instances by adding higher misclassification cost at thetime of processing a boosting approach. Both classifiersare designed to utilize the majority class instances whichare mostly ignored by under-sampling in keeping the paceof training speed with high quality. They almost performalike by sampling multiple subsets of the majority class.Training each of the subsets and finally combining all weakclassifiers as an output. As ensemble subsets comprisemore information than a single one, which makes bothalgorithms much better uses of the majority class insteadof under-sampling. The same training times and the samenumber of weak classifiers are used by both the classifiers.The principal distinction is that BalanceCascade works withtrained classifiers for subsequent classifiers. On the otherhand, EasyEnsemble samples work with independent subsets.

10) OverBagging: OverBagging is the ensemble processof combining over-sampling with bagging methods. Itrandomly over-samples minority classes in each baggingiteration. It uses majority-voting technique to classify new



______________________________________________________________________________________

instances, as each classifier gives its decision and finalclassification made by a majority of votes. If a tie appears,then the class with minor instances are returned [36]. Theentire over-bagging method can be done in 3 steps fromits training phase to testing phase i.e. i) re-sampling, ii)constructing ensemble and iii) voting. Because of multipleminorities and majority class, it is very complicated tochoose the re-sampling rate. The output of the algorithmshows that a variety of instances dominates recall valuenotably. The larger diversity consequences excellent recallfor a minority whereas poor recall for majority classes.

11) UnderBagging: UnderBagging method uses arandom under-sampling technique of majority classinstances with bagging to build an ensemble classifier[14]. It also considers majority voting of classifiers toclassify a new/ known instance. We may lose some valuableinformative majority class instances when randomlysampling majority class instances. The core idea of thismethod is to educate each of the single components of theensemble classifier with a balanced learning instance. Byreplacing an individual classification model, (Here the 1-NNrule) with an imbalanced training set, by an amalgamationof various classifiers, each model uses a balanced trainingset for its learning process. To earn this, as many trainingsub-samples as possible to get balanced subsets are beget.The number of sub-samples will be discovered by thedifference between the number of prototypes from themajority class and that of the minority class. This workflowmakes it possible to properly handle the difficulties of theimbalance and stays away from the implicit disadvantagesto both the over and under-sampling techniques.

12) SMOTEBagging: SMOTEBagging builds a model byemploying SMOTE with bagging methods to balance theclass distribution. It over-samples from the minority classinstances using SMOTE. In SMOTEBagging, we need to setthe size/ amount of over-sampling of minority class instancesand also the nearest-neighbors of instances [36]. SMOTE-Bagging is differed from UnderBagging and OverBagging,by involving in subset creation of synthetic instances. Asclaimed by the SMOTE algorithm, at first the (k) nearestneighbors and the volume of over-sampling from minorityclass (N) should be set. Here N=100, 200, 300, 400 and500 i.e K=5 nearest neighbours. The correlative class mustbe reviewed at the time of all minority class ordination. Anexample is illustrated below: assume that the minority classX consists of 10 instances and the majority class Y consistsof 60 instances. To over-sample, both X and Y through thesame N value are applied so that the two classes are inner-imbalanced. In SMOTEBagging, x% value is used to controlthe instances from each class that is used for propagatingnovel instances.

V. DATASET DESCRIPTION

A. Imbalanced Datasets

In this paper, we have considered 27 datasets from KEELdataset repository [9]. Table II and Table V gives an outlineof the features of the selected datasets.

TABLE II: Datasets description.(All are real world data)

SlNo.

Dataset Features

Ins tances

ImbalancedRatio

1 glass1 9 214 1.822 ecoli0 vs 1 7 220 1.863 wisconsin 9 683 1.864 pima 8 768 1.875 iris0 4 150 26 glass0 9 214 2.067 yeast1 8 1484 2.468 haberman 3 306 2.789 vehicle2 18 846 2.8810 vehicle1 18 846 2.911 vehicle3 18 846 2.9912 glass0123 vs 456 9 214 3.213 vehicle0 18 846 1.8214 ecoli1 7 336 1.8215 new-thyroid1 5 215 1.8216 new-thyroid2 5 215 1.8217 ecoli2 7 336 1.8218 segment0 19 2308 1.8219 glass6 9 214 1.8220 yeast3 8 1484 1.8221 ecoli3 7 336 1.8222 page-blocks0 10 5472 1.8223 yeast-2 vs 4 8 514 9.0824 glass-0-1-6 vs 2 9 192 10.2925 vowel0 13 988 10.1026 yeast-0-5-6-7-

9 vs 48 528 9.35

27 ecoli-0-1-3-7 vs 2-6

7 281 39.15

B. Algorithm Parameters

We have used the following parameters for our experi-ments:



______________________________________________________________________________________

TABLE III: Input Parameters for the experiment

SlNo.

Parameters Value SlNo.

Parameters Value

1. pruned TRUE 9 Classification rate by algo-rithm and fold

YES

2. InstancesPerLeaf 2 10 Header size in previous ta-ble

2

3. Number of classes 2 11 Data used in previous table TEST-TRAIN4. Folds 5 12 imbalanced Measure Area Under the ROCCurve

(AUROC)5. K Value 3 13 imbalanced Measure Area Under the ROCCurve

(AUROC)6. Distance Function Euclidean 14 OS used for this experi-

ments:Linux and Mac OS

7. Train Method NORESAMPLING 15 CPU speed for this experi-ment.

2.5 GHz

8. dataformat keel

TABLE IV: Strengths and weaknesses of 12 algorithms.

SlNo.

Algorithm Strengths Weaknesses

1 SMOTE The classifiers are very much unique and com-putationally simple [4].

It is a perplexed and prolonged procedure.

2 AdaBoost Prediction accuracy improve in case of minoritydata set and resolve the multi class imbalanceddata set problem [7]

Ignore overall performance of the classifier.

3 RUSBoost Simpler, easier, faster and less complex algo-rithm. RUS has been operated skilfully andswiftly [45].

The major handicap of this model is that im-portant data can be destroyed.

4 EUSBoost Easier, quicker and outperformed in highly im-balanced datasets.

Computationally more expensive than that ofthe other methods [32].

5 SMOTEBoost This method is a perfect selection when thetraining instance size is excessively huge [4].

The possibility of creating redundancy. Besidesmore complex and prolonged than RUS.

6 DataBoost Produce high accuracies of both classes (minor-ity and majority) [17].

The use of many classifiers makes them morecomplex and produces output that is very hardto analyze

7 MSMOTEBoost

This method has better prediction accuracy [46]. Time-consuming and perplexing task.

8 Easy Ensem-ble

Reduce information loss. Faster training speedof undersampling [4].

Examining only binary classification problems[37].

9 Balance Cas-cade

Reduces the rate of creating duplicate informa-tion. Most popular method is stacking [44] .

Performance standard reduces in multi class.

10 Over Bagging Performed masterly both in the binary andmulti-class data set

Performance standard degrades in minorityclass [36].

11 UnderBagging

This procedure makes it possible to properlyhandle the difficulties of both the over andundersampling techniques [14].

When the experimental data size is not large theperformance may be deteriorated.

12 SMOTE Bag-ging

Increased accuracy level [14]. Biased in favor of the majority class on high-dimensional data



______________________________________________________________________________________

TABLE V: Datasets detail description (Cont.)

SlNo.

Name Attributes Class

1. glass0, glass1, glass01-23 vs 456, glass6 andglass-0-1-6 vs 2.

RI, Na, Mg, Al, Si, K, Ca, Ba and Fe. Positive and Negative.

2. ecoli1, ecoli2, ecoli3,ecoli0 vs 1 and ecoli-0-1-3-7 vs 2-6.

Gvh, Lip, Chg, Aac, Alm1 and Alm2. Positive and Negative.

3. wisconsin ClumpThickness, CellSize, CellShape, MarginalAdhesion,EpithelialSize, BareNuclei, BlandChromatin, NormalNucleoliand Mitoses.

Positive and Negative.

4. pima Preg, Plas, Pres, Skin, Insu, Mass, Pedi and Age. Positive and Negative.

5. iris0 SepalLength, SepalWidth, PetalLength and PetalWidth. Positive and Negative.

6. yeast1, yeast3, yeast-2 vs 4 and yeast-0-5-6-7-9 vs 4.

Mcg, Gvh, Alm, Mit, Erl, Pox, Vac and Nuc. Positive and Negative.

7. haberman Age and Year. Positive and Negative.8. vehicle1, vehicle2, vehi-

cle3Compactness, Circularity, Distance circularity, Radius ratio,Praxis aspect ratio, Max length aspect ratio, Scatter ratio,Elongatedness, Praxis rectangular, Length rectangular,Major variance, Minor variance, Gyration radius,Major skewness, Minor skewness, Minor kurtosis,Major kurtosis and Hollows ratio.


9. new-thyroid1 and new-thyroid2

T3resin, Thyroxin, Triiodothyronine, Thyroidstimulating andTSH value.


10. segment 0 Region-centroid-col, Region-centroid-row, Region-pixel-count, Short-line-density-5, Short-line-density-2, Vedge-mean, Vegde-sd, Hedge-mean, Hedge-sd, Intensity-mean,Rawred-mean, Rawblue-mean, Rawgreen-mean, Exred-mean,Exblue-mean, Exgreen-mean, Value-mean, Saturatoin-meanand Hue-mean.


11. page-blocks0 Height, Lenght, Area, Eccen, P black, P and, Mean tr,Blackpix, Blackand and Wb trans.


12. Vowel 0 TT, SpeakerNumber, Sex, F0, F1, F2, F3, F4, F5, F6, F7, F8,F9.




______________________________________________________________________________________

VI. EXPERIMENTAL RESULTS

TABLE VI: The AUC comparison of SMOTE, AdaBoost, RUSBoost, SMOTEBoost, EUSBoost, DataBoost, MSMOTEBoost,Over-Bagging, EasyEnsemble, BalanceCascade, Under-Bagging, SMOTEBagging on 27 imbalanced datasets.

Dataset SMOTE AdaBoost

RUSBoost

SMOTEBoost

EUSBoost

DataBoost

MSMOTEBoost

OverBagging

EasyEnsem-ble

BalanceCascade

UnderBagging

SMOTEBag-ging

glass1 0.992 0.809 0.77 0.783 0.783 0.7 0.762 0.77 0.77 0.847 0.754 0.744ecoli-0 vs 1

1 0.969 0.976 0.972 0.962 0.94 0.969 0.979 0.941 0.941 0.979 0.979

wisconsin 0.955 0.961 0.956 0.965 0.944 0.960 0.939 0.98 0.989 0.99 0.961 0.967pima 0.731 0.689 0.739 0.729 0.7 0.711 0.734 0.712 0.718 0.881 0.76 0.749iris0 1 0.99 0.99 0.99 0.99 0.99 0.99 0.98 0.99 0.99 0.99 0.96glass0 0.875 0.823 0.852 0.823 0.845 0.835 0.828 0.837 0.817 0.817 0.841 0.816yeast1 0.769 0.652 0.704 0.712 0.688 0.699 0.709 0.707 0.689 0.677 0.723 0.694haberman 0.741 0.581 0.629 0.614 0.554 0.561 0.628 0.598 0.639 0.636 0.664 0.666vehicle2 0.968 0.966 0.972 0.981 0.971 0.966 0.947 0.960 0.952 0.952 0.961 0.967vehicle1 0.753 0.7 0.743 0.752 0.79 0.966 0.728 0.708 0.712 0.712 0.766 0.746vehicle3 0.755 0.681 0.764 0.744 0.722 0.691 0.738 0.720 0.728 0.728 0.791 0.791glass-012-3 vs 4-56

0.953 0.877 0.879 0.918 0.914 0.897 0.918 0.884 0.901 0.901 0.888 0.909

vehicle0 0.959 0.931 0.958 0.976 0.922 0.944 0.939 0.949 0.929 0.929 0.952 0.947ecoli1 0.961 0.822 0.912 0.847 0.898 0.806 0.892 0.867 0.882 0.882 0.899 0.904new-thyroid1

0.986 0.931 0.932 0.988 0.96 0.957 0.988 0.954 0.955 0.955 0.966 0.977

new-thyroid2

0.972 0.954 0.918 0.971 0.935 0.957 0.948 0.934 0.946 0.954 0.954 0.969

ecoli2 0.964 0.851 0.865 0.909 0.928 0.808 0.909 0.871 0.867 0.867 0.898 0.914segment0 0.997 0.99 0.991 0.993 0.991 0.991 0.991 0.990 0.985 0.987 0.985 0.982glass6 0.986 0.849 0.909 0.855 0.906 0.841 0.922 0.883 0.893 0.893 0.890 0.911yeast3 0.918 0.840 0.916 0.887 0.885 0.897 0.914 0.897 0.907 0.907 0.932 0.934ecoli3 0.95 0.716 0.871 0.858 0.881 0.812 0.856 0.743 0.801 0.841 0.880 0.870page-blocks0

0.964 0.930 0.947 0.939 0.902 0.874 0.989 0.936 0.953 0.956 0.956 0.955

yeast-2 vs 4

0.91 0.889 0.965 0.86 0.905 0.866 0.848 0.861 0.886 0.886 0.954 0.882

glass01-6 vs 2

0.782 0.588 0.679 0.632 0.785 0.649 0.601 0.582 0.611 0.625 0.678 0.672

vowel0 0.971 0.97 0.957 0.969 0.9854 0.97 0.943 0.951 0.947 0.947 0.946 0.971yeast-0567-9 vs 4

0.817 0.645 0.845 0.781 0.757 0.737 0.765 0.715 0.751 0.751 0.788 0.802

ecoli-013-7 vs 26

0.731 0.648 0.83 0.831 0.804 0.548 0.844 0.74 0.817 0.817 0.802 0.828



______________________________________________________________________________________

Fig. 2: The comparison among classifiers (AUROC value) for imbalanced data classification.

Fig. 3: The comparison among classifiers (G-mean value) for imbalanced data classification.



______________________________________________________________________________________

TABLE VII: The G-mean comparison of SMOTE, AdaBoost, RUSBoost, SMOTEBoost, EUSBoost, DataBoost, MSMOTE-Boost, Over-Bagging, EasyEnsemble, BalanceCascade, Under-Bagging, SMOTEBagging on 27 imbalanced datasets.


RUSBoost

SMOTEBoost

EUSBoost

DataBoost

MSMOTEBoost

OverBag-ging

EasyEnsem-ble

BalanceCascade

UnderBagging

SMOTEBag-ging

glass1 0.992 0.8006 0.77 0.784 0.788 0.794 0.787 0.743 0.743 0.743 0.751 0.740ecoli-0 vs 1

0.999 0.976 0.976 0.7764 0.961 0.94 0.969 0.979 0.940 0.940 0.76 0.76


0.977 0.868 0.876 0.916 0.912 0.901 0.9160.878

0.899 0.899 0.885 0.906


0.986 0.928 0.930 0.988 0.959 0.954 0.988 0.951 0.953 0.953 0.965 0.976

new-thyroid2

0.972 0.952 0.917 0.971 0.934 0.954 0.947 0.931 0.945 0.954 0.954 0.968

ecoli2 0.97 0.84 0.861 0.907 0.926 808 0.908 0.866 0.822 0.865 0.897 0.912segment0 0.997 0.989 0.991 0.993 0.991 0.991 0.991 0.993 0.985 0.987 0.985 0.982glass6 0.986 0.836 0.908 0.841 0.824 0.845 0.991 0.993 0.985 0.987 0.984 0.981yeast3 0.918 0.83 0.915 0.884 0.944 0.869 0.913 0.893 0.907 0.907 0.932 0.934ecoli3 0.95 0.928 0.946 0.938 0.812 0.812 0.951 0.935 0.953 0.955 0.938 0.955page-blocks0

0.964 0.657 0.869 0.851 0.878 0.792 0.853 0.698 0.832 0.799 0.874 0.868

yeast-2 vs 4

0.9 0.906 0.912 0.8528 0.9034 0.8592 0.8374 0.853 0.8824 0.8824 0.953 0.9214

glass-0-1-6 vs 2

0.785 0.4198 0.6668 0.5004 0.7762 0.51 0.4556 0.4184 0.574 0.5962 0.6654 0.5858

vowel0 0.97 0.968 0.9554 0.9692 0.9454 0.968 0.9404 0.9482 0.9454 0.9458 0.944 0.9704yeast-0-5-6-7-9 vs 4

0.82 0.5538 0.842 0.7626 0.754 0.6954 0.7418 0.6688 0.7452 0.7452 0.7856 0.7958

ecoli-0-1-3-7 vs 2-6

0.731 0.34 0.809 0.726 0.6942 0.14 0.736 0.5338 0.7934 0.7934 0.7748 0.7192



______________________________________________________________________________________

TABLE VIII: The F-measure comparison of SMOTE, AdaBoost, RUSBoost, SMOTEBoost, EUSBoost, DataBoost,MSMOTEBoost, Over-Bagging, EasyEnsemble, BalanceCascade, Under-Bagging, SMOTEBagging on 27 imbalanceddatasets.


RUSBoost

SMOTEBoost

EUSBoost

DataBoost

MSMOTEBoost

OverBag-ging

EasyEnsem-ble

BalanceCascade

UnderBagging

SMOTEBag-ging

glass1 0.901 0.753 0.706 0.722 0.723 0.729 0.733 0.687 0.681 0.681 0.691 0.677ecoli-0 vs 1

0.98 0.954 0.967 0.722 0.944 0.94 0.955 0.973 0.909 0.909 0.691 0.717


0.91 0.835 0.805 0.872 0.847 0.867 0.872 0.840 0.829 0.829 0.808 0.849


0.988 0.884 0.847 0.946 0.874 0.951 0.946 0.937 0.862 0.862 0.899 0.946

new-thyroid2

0.977 0.940 0.810 0.922 0.833 0.951 0.915 0.898 0.863 0.899 0.896 0.911

ecoli2 0.95 0.765 0.699 0.814 0.778 0.722 0.814 0.773 0.697 0.655 0.755 0.778segment0 0.905 0.986 0.966 0.984 0.99 0.984 0.981 0.984 0.940 0.951 0.939 0.969glass6 0.955 0.774 0.749 0.772 0.782 0.755 0.859 0.790 0.689 0.689 0.693 0.809yeast3 0.85 0.712 0.693 0.749 0.752 0.748 0.760 0.762 0.686 0.686 0.743 0.778ecoli3 0.82 0.874 0.797 0.845 0.856 0.759 0.863 0.860 0.804 0.8008 0.845 0.839page-blocks0

0.912 0.548 0.567 0.632 0.603 0.611 0.631 0.521 0.519 0.481 0.537 0.601

yeast-2 vs 4

0.81 0.815 0.713 0.706 0.671 0.754 0.679 0.709 0.662 0.662 0.716 0.720

glass-0-1-6 vs 2

0.682 0.333 0.273 0.404 0.383 0.488 0.317 0.317 0.206 0.219 0.270 0.404

vowel0 0.771 0.951 0.847 0.906 0.810 0.951 0.882 0.908 0.774 0.777 0.766 0.918yeast-0-5-6-7-9 vs 4

0.717 0.370 0.536 0.542 0.405 0.505 0.515 0.477 0.368 0.368 0.421 0.468

ecoli-0-1-3-7 vs 2-6

0.631 0.75 0.141 0.444 0.388 0.283 0.741 0.869 0.128 0.128 0.194 0.45



______________________________________________________________________________________

Fig. 4: The comparison among classifiers (F-measure) for imbalanced data classification.

Fig. 5: The comparison among classifiers (Execution-time) for imbalanced data classification.



______________________________________________________________________________________

TABLE IX: The Processing speed i.e. the Execution Time (second) of SMOTE, AdaBoost, RUSBoost, SMOTEBoost,EUSBoost, DataBoost, MSMOTEBoost, Over-Bagging, EasyEnsemble, BalanceCascade, Under-Bagging, SMOTEBaggingon 27 imbalanced datasets.

DatasetSMOTE Ada

BoostRUSBoost

SMOTEBoost

EUSBoost

DataBoost

MSMOTEBoost

OverBag-ging

EasyEnsem-ble

BalanceCascade

UnderBagging

SMOTEBag-ging

glass1 0.02 0.80 1.12 2.47 36.57 16.04 1.91 1.46 0.60 0.61 1.50 1.58ecoli-0 vs 1

0.14 0.59 0.36 1.14 12.76 14.0 0.88 0.39 0.90 0.43 0.65 0.76

wisconsin 0.07 1.42 1.64 3.49 5:40.51 3.37 4.77 2.89 0.94 1.16 1.79 2.31pima 0.09 1.75 2.40 4.84 6:42.64 7.18 5.41 3.47 1.42 1.41 2.16 3.32iris0 0.12 0.37 0.25 0.25 1.88 0.25 0.25 0.24 0.24 0.27 0.15 0.24glass0 0.01 0.78 0.81 1.83 40.7

5:30.761.84 1.447 0.65 0.61 0.94 1.28

yeast1 0.23 4.14 3.83 10.2 19:55.82 29.26 14.9 8.09 3.70 4.21 3.33 6.29haberman 0.02 0.52 0.63 1.39 29.3 1.48 1.17 1.55 0.89 0.79 0.52 0.96vehicle2 0.19 2.25 2.49 9.83

8:33.5010.59 13.07 6.80 1.31 1.35 2.97 6.10

vehicle1 0.16 2.55 3.33 10.06 8:15.48 9.87 11.87 7.36 1.35 1.38 3.08 5.87vehicle3 0.19 2.46 3.11 10.36 0.762 27.2 12.23 8.48 1.44 1.63 3.31 2.89glass-012-3 vs 4-56

0.01 0.49 0.50 2.31 17.85 2.44 2.08 1.04 0.36 0.37 0.61 1.23

vehicle0 0.14 2.08 2.52 8.6215:22.81

21.95 12.44 5.61 1.19 1.18 2.49 5.46

ecoli1 0.02 0.504 0.74 2.042:34.58

2.23 1.62 0.75 0.48 0.71 1.47

new-thyroid1

0.01 0.34 0.20 1.48 4.98 0.83 1.39 0.57 0.25 0.25 0.24 1.17

new-thyroid2

0.01 0.30 0.18 1.64 4.08 0.82 0.98 0.50 0.22 0.21 0.19 0.66

ecoli2 0.01 0.53 0.43 2.34 23.6 2.43 1.71 0.43 0.36 0.57 1.614segment0 0.58 6.53 3.94 25.6 40.2 3:16.28 22.5 2.79 2.73 4.70 23.6glass6 0.01 0.51 0.34 2.47 9.34 2.86 2.06 1.09 0.29 0.30 0.34 1.55yeast3 0.11 2.33 1.54 11.6 7:12.56

5:42.6716.4 7.78 1.06 1.01 1.43 7.17

ecoli3 0.02 0.65 0.52 2.48 22.4 14.7 2.54 1.82 0.39 0.37 5.22 1.69page-blocks0

1.53 27.9 5.76 1:20.76 71.5 2:57.311:35.86

12.6 13.0 5.036 40.3

yeast-2 vs 4

0.0346 1:23:88 1:1.78 5.9974 1:1.0876 24.3274 4.639 2.6286 0.4326 0.5346 1:8.02 3.0108

glass-0-1-6 vs 2

0.0088 0.5302 0.339 3.14 13.1314 8.3452 2.0932 1.4462 0.2984 0.3358 0.5076 1.748

vowel0 0.0938 1.2344 1.2376 10.5064 2:50.21 6.1576 6.892 1.6404 1.4578 0.7336 1.2776 7.9642yeast-0-5-6-7-9 vs 4

0.0262 1.591 0.7512 4.469 56.5524 34.8614 5.0538 3.4094 0.4764 0.6704 0.7926 3.1722

ecoli-0-1-3-7 vs 2-6

0.01 0.4786 0.1468 2.2808 6.2428 4.3724 2.3444 1.6222 0.185 0.2336 0.1646 1.7924



______________________________________________________________________________________

Fig. 6: AUCROC line charts of algorithms and datasets (Randomly given 4 out of 27 : PIMA, ecoli2, ecoli3, glass01).



______________________________________________________________________________________

VII. RESULTS

We have tested the performance of 12 machine learningalgorithms for imbalanced data classification using the AreaUnder Curve (AUC) with 5-fold cross-validation. The ex-perimental results are tabulated in Table VI (in terms ofROC) and in Table IX (in terms of processing speed). Alsoin the table VII and VIII we present the experiment resultof other two popular performance measure of imbalancedata set such as G-mean and F-measure. Fig. 2, 3, 4 and 5shows the comparison of the 12 classifiers using differenceperformance evaluator such as AUC, G-mean and F-measureand execution speed of the classifiers on 27 imbalanceddatasets respectively. In all cases, we can see based onperformance measures and processing speed of the classifiersin spite of being a single classifier SMOTE performs bestcompare with other ensemble classifiers.

VIII. CONCLUSION

In the last decade, several machine learning methods havebeen proposed for dealing with class imbalanced datasetsto improve the performance of classifiers for classifyingminority class instances. In this paper, we have reviewedsome machine learning algorithms for imbalanced data clas-sification. We have tested the performances of 12 imbalanceddata classification methods: SMOTE, AdaBoost, RUSBoost,EUSBoost, SMOTEBoost, MSMOTEBoost, DataBoost, EasyEnsemble, BalanceCascade, OverBagging, UnderBagging,SMOTEBagging on 27 datasets from the KEEL Repository[9] with a high imbalanced ratio. The experimental resultssum up that SMOTE has performed better in most datasetsin comparison with ensemble classifiers. On the other hand,ensemble classifiers are more complicated to implement andunderstood. In general, a combination of both over-samplingand under-sampling techniques with ensemble classifier suchas bagging and boosting achieve the highest accuracy forclassifying both majority and minority class instances. Infuture work, we will apply these imbalanced data classi-fication methods in real-life high-dimensional imbalancedbig data and apply sampling techniques with RandomForestalgorithm. In Table IV we represent the common strengthsor weaknesses based on the experimental results of ourmentioned algorithm with some differences.

REFERENCES

[1] N. Tomasev and D. Mladenic, “Class imbalance and the curse ofminority hubs,” Knowledge-Based Systems, vol. 53, pp. 157–172,2013.

[2] H. He and E. A. Garcia, “Learning from imbalanced data,” IEEETransactions on Knowledge and Data Engineering, vol. 21, no. 9,pp. 1263–1284, 2009.

[3] J. T. Lalis, “A new multiclass classification method for objects withgeometric attributes using simple linear regression,” IAENG Interna-tional Journal of Computer Science, vol. 43, no. 2, pp. 198–203, 2016.

[4] N. V. Chawla, K. W. Bowyer, L. O. Hall, and W. P. Kegelmeyer,“Smote: Synthetic minority over-sampling technique,” Journal of Ar-tificial Intelligence Research, vol. 16, no. 10, pp. 321–357, October2002.

[5] M. Galar, A. Fernandez, E. Barrenechea, H. Bustince, and F. Herrera,“A review on ensembles for the class imbalance problem: Bagging-, boosting-, and hybrid-based approaches,” IEEE Transactions onSystems, Man, and Cybernetics, Part C (Applications and Reviews),vol. 42, no. 4, pp. 463–484, 2012.

[6] Z. Sun, Q. Song, X. Zhu, H. Sun, B. Xu, and Y. Zhou, “A novel en-semble method for classifying imbalanced data,” Pattern Recognition,vol. 48, no. 5, pp. 1623–1637, 2015.

[7] Y. Freund and R. E. Schapire, “A decision-theoretic generalization ofon-line learning and an application to boosting,” Journal of Computerand System Sciences, vol. 55, no. 1, pp. 119–139, 1997.

[8] R. Sarno, R. D. Dewandono, T. Ahmad, M. F. Naufal, and F. Sinaga,“Hybrid association rule learning and process mining for fraud de-tection,” IAENG International Journal of Computer Science, vol. 42,no. 2, pp. 59–72, 2015.

[9] J. Alcala-Fdez, A. Fernandez, J. Luengo, J. Derrac, S. Garcıa,L. Sanchez, and F. Herrera, “Keel data-mining software tool: Dataset repository, integration of algorithms and experimental analysisframework,” Journal of Multiple-Valued Logic and Soft Computing,vol. 17, no. 2-3, pp. 255–287, 2011.

[10] K. Veropoulos, C. Campbell, and N. Cristianini, “Controlling thesensitivity of support vector machines,” 16th International Joint Con-ferences on Artificial Intelligence (IJCAI-99). Stockholm (Sweden,1999), vol. 55, p. 60, 1999.

[11] R. P. Espındola and N. F. F. Ebecken, “On extending f-measureand g-mean metrics to multi-class problems,” WIT Transactions onInformation and Communication Technologies, vol. 35, p. 10, 2005.

[12] L. A. Jeni, J. F. Cohn, and F. D. L. Torre, “Facing imbalanced datarecommendations for the use of performance metrics,” Int Conf AffectComput Intell Interact Workshops, pp. 245–251, 2013.

[13] N. G. Hoang, A. Bouzerdoum, and S. L. Phung, “Learning patternclassification tasks with imbalanced data sets,” Pattern recognition,pp. 193–208, 2009.

[14] R. Barandela, R. Valdovinos, and J. Sanchez, “New applications ofensembles of classifiers,” Pattern Analysis and Applications, vol. 6,no. 3, pp. 245–256, 2003.

[15] H. Han, W.-Y. Wang, and B.-H. Mao, “Borderline-smote: a newover-sampling method in imbalanced data sets learning,” InternationalConference on Intelligent Computing, vol. 3644, pp. 878–887, 2005.

[16] C. Seiffert, T. M. Khoshgoftaar, J. V. Hulse, and A. Napolitano,“Rusboost: Improving classification performance when training datais skewed,” 19th International Conference on Pattern Recognition, pp.1–4, 2008.

[17] H. Guo and H. L. Viktor, “Learning from imbalanced data sets withboosting and data generation: the databoost-im approach,” SIGKDDExplorations, vol. 6, no. 1, pp. 30–39, 2004.

[18] H. He, Y. Bai, E. A. Garcia, and S. Li, “Adasyn: Adaptive syntheticsampling approach for imbalanced learning,” Proceedings of the Inter-national Joint Conference on Neural Networks, pp. 1322–1328, 2008.

[19] N. V. Chawla, N. Japkowicz, and A. Kotcz, “Special issue on learningfrom imbalanced data sets,” ACM SIGKDD Explorations Newsletter,vol. 6, no. 1, pp. 1–6, June 2004.

[20] G. M. Weiss, “Mining with rarity: A unifying framework,” SIGKDDExplorations, vol. 6, no. 1, pp. 7–19, 2004.

[21] K. Savetratanakaree, K. Sookhanaphibarn, S. Intakosum, and R. Tha-wonmas, “Borderline over-sampling in feature space for learningalgorithms in imbalanced data environments,” IAENG InternationalJournal of Computer Science, vol. 43, no. 3, pp. 363–373, 2016.

[22] M. Y. Arafat, S. Hoque, and D. M. Farid, “Cluster-based under-sampling with random forest for multi-class imbalanced classification,”11’th International Conference on Software, Knowledge, InformationManagement and Applications (SKIMA), pp. 1–6, 2017.

[23] K. M. Ting, “An instance weighting method to induce cost-sensitivetrees,” IEEE Transactions on Knowledge and Data Engineering,vol. 14, no. 3, pp. 659–665, 2002.

[24] S. Hoque, M. Y. Arafat, and D. M. Farid, “Machine learning for miningimbalanced data,” International Conference on Emerging Technologyin Data Mining and Information Security (IEMIS 2018), no. 1, p. 10,2018.

[25] J. R. Quinlan, “Boosting, bagging, and c4.5,” Proc. 13th Nat’l Conf.Artificial Intelligence, pp. 725–730, 1996.

[26] A. Graves, M. Liwicki, S. Fernandez, R. Bertolami, H. Bunke, andJ. Schmidhuber, “A novel connectionist system for unconstrainedhandwriting recognition,” IEEE transactions on pattern analysis andmachine intelligence, vol. 31, no. 5, pp. 855–868, 2009.

[27] N. Cesa-Bianchi, Y. Freund, D. P. Helmbold, D. Haussler, R. E.Schapire, and M. K. Warmuth, “How to use expert advice,” Journalof the ACM (JACM), vol. 44, no. 3, pp. 427–485, 1997.

[28] E. Bauer and R. Kohavi, “An empirical comparison of voting classifica-tion algorithms: Bagging, boosting, and variants,” Machine Learning,vol. 36, no. 1-2, pp. 105–139, 1999.

[29] T. M. Khoshgoftaar, C. Seiffert, J. V. Hulse, A. Napolitano, and A. Fol-leco, “Learning with limited minority class data,” Sixth InternationalConference on Machine Learning and Applications (ICMLA 2007),pp. 348–353, 2007.

[30] G. E. A. P. A. Batista, R. C. Prati, and M. C. Monard, “A studyof the behavior of several methods for balancing machine learningtraining data,” ACM SIGKDD Explorations Newsletter - Special issueon learning from imbalanced datasets, vol. 6, no. 1, pp. 20–29, 2004.



______________________________________________________________________________________

[31] S. Garcıa and F. Herrera, “Evolutionary undersampling for classifica-tion with imbalanced datasets: proposals and taxonomy,” EvolutionaryComputation, vol. 17, no. 3, pp. 275–306, 2009.

[32] M. Galar, A. Fernandez, E. Barrenechea, and F. Herrera, “Eusboost:Enhancing ensembles for highly imbalanced data-sets by evolutionaryundersampling,” Pattern Recognition, vol. 46, no. 12, pp. 3460–3471,2013.

[33] D. J. Drown, T. M. Khoshgoftaar, and N. Seliya, “Evolutionarysampling and software quality modeling of high-assurance systems,”IEEE Transactions on Systems,Man, and Cybernetics, Part A: Systemsand Humans, vol. 39, no. 5, pp. 1097–1107, 2009.

[34] K. S. WOODS, C. C. DOSS, K. W. BOWYER, J. L. SOLKA, C. E.PRIEBE, and J. W. PHILIP KEGELMEYER, “Comparative evaluationof pattern recognition techniques for detection of microcalcificationsin mammography,” International Journal of Pattern Recognition andArtificial Intelligence, vol. 7, no. 6, pp. 1417–1436, 1993.

[35] M. Buckland and F. Gey, “The relationship between recall and preci-sion, journal of the american society for information science,” Journalof the American Society for Information Science, vol. 45, no. 1, pp.12–19, 1994.

[36] S. Wang and X. Yao, “Diversity analysis on imbalanced data sets byusing ensemble models,” IEEE Symposium Series on ComputationalIntelligence and Data Mining, vol. 19, no. 8, pp. 324–331, 2009.

[37] X.-Y. Liu, J. Wu, and Z.-H. Zhou, “Exploratory under sampling forclass imbalance learning,” IEEE Transactions on Systems, Man, andCybernetics, Part B (Cybernetics), vol. 39, no. 2, pp. 539–550, 2009.

[38] C. P. Chen and C.-Y. Zhang, “Data-intensive applications, challenges,techniques and technologies: A survey on big data,” InformationSciences, vol. 275, pp. 314–347, 2014.

[39] J. H. Friedman, “Stochastic gradient boosting,” Comput. Stat. DataAnal., vol. 38, no. 4, pp. 367–378, February 2002.

[40] G. I. Webb, “Multiboosting: A technique for combining boosting andwagging,” Machine Learning, vol. 40, no. 2, pp. 159–196, 2000.

[41] G. I. Webb and Z. Zheng, “Multistrategy ensemble learning: Reducingerror by combining ensemble learning techniques,” IEEE Transactionson Knowledge and Data Engineering, vol. 16, no. 8, pp. 980–991,2004.

[42] Y. Yu, Z. H. Zhou, and K. M. Ting, “Cocktail ensemble for regression,”Seventh IEEE International Conference on Data Mining (ICDM 2007),pp. 721–726, 2007.

[43] J. H. Friedman, “Stochastic gradient boosting,” Comput. Stat. DataAnal, vol. 38, no. 4, pp. 367–378, February 2002.

[44] D. H. Wolpert, “Stacked generalization,” Neural Networks, vol. 5,no. 2, pp. 241–259, 1992.

[45] C. Seiffert, T. M. Khoshgoftaar, J. V. Hulse, and A. Napolitano,“Rusboost: A hybrid approach to alleviating class imbalance,” IEEETransactions on Systems, Man, and Cybernetics - Part A: Systems andHumans, vol. 40, no. 1, pp. 185–197, 2010.

[46] S. Hu, Y. Liang, L. Ma, and Y. He, “Msmote: Improving classificationperformance when training data is imbalanced,” 2nd InternationalWorkshop on Computer Science and Engineering(WCSE 2009), vol. 2,pp. 13–17, 2009.



______________________________________________________________________________________

iaeng international journal of computer science, 46:2, ijcs 46 2 … · 2019-06-01 · f measure =...

Documents