dmdw record

7/31/2019 DMDW Record

1/43

Data Mining Lab Record:

1) INTRODUCTION OF WEKA

Weka (Waikato Environment for Knowledge Analysis) is a popular suite of machine learning software written in Java, developed at the University of Waikato, New Zealand. Weka is free software available under the GeneralPublic License.

The Weka workbench contains a collection of visualization tools andalgorithms for data analysis and predictive modeling, together with graphicaluser interfaces for easy access to this functionality.

This original version was primarily designed as a tool for analyzing datafrom agricultural domains, but the more recent fully Java-based version(Weka 3), for which development started in 1997, is now used in manydifferent application areas, in particular for educational purposes andresearch.

2) ADVANTAGES OF WEKA OVER DBMS

The obvious advantage of a package like Weka is that a whole range of data preparation, feature selection and data mining algorithms are integrated. Thismeans that only one data format is needed, and trying out and comparingdifferent approaches becomes really easy.The package also comes with aGUI, which should make it easier to use.

portability, since it is fully implemented in the Java programming languageand thus runs on almost any modern computing platform.

a comprehensive collection of data preprocessing and modeling techniques.

ease of use due to its graphical user interfaces.


2/43

Weka supports several standard data mining tasks, more specifically, data preprocessing, clustering, classification, regression, visualization, andfeature selection.

All of Weka's techniques are predicated on the assumption that the data isavailable as a single flat file or relation, where each data point is described

by a fixed number of attributes (normally, numeric or nominal attributes, butsome other attribute types are also supported).

Weka provides access to SQL databases using Java Database Connectivityand can process the result returned by a database query.

It is not capable of multi-relational data mining, but there is separatesoftware for converting a collection of linked database tables into a singletable that is suitable for processing using Weka. Another important area issequence modeling.

Attribute Relationship File Format (ARFF) is the text format file used byWeka to store data in a database.

The ARFF file contains two sections: the header and the data section. Thefirst line of the header tells us the relation name.

Then there is the list of the attributes (@attribute...). Each attribute isassociated with

a unique name and a type.


3/43

The latter describes the kind of data contained in the variable and whatvalues it can have. The variables types are: numeric, nominal, string anddate.

The class attribute is by default the last one of the list. In the header sectionthere can also be some comment lines, identified with a '%' at the beginning,which can describe the database content or give the reader information aboutthe author. After that there is the data itself (@data), each line stores theattribute of a single entry separated by a comma.

Weka's main user interface is the Explorer, but essentially the samefunctionality can be accessed through the component-based KnowledgeFlow interface and from the command line. There is also the Experimenter,which allows the systematic comparison of the predictive performance of Weka's machine learning algorithms on a collection of datasets.

1 Launching WEKA

The WEKA GUI Chooser window is used to launch WEKAs graphical

environments.At the bottom of the window are four buttons:

1. Simple CLI. Provides a simple command-line interface that allows directexecution of WEKA commands for operating systems that do not providetheir own command line Interface.

2. Explorer. An environment for exploring data with WEKA.

3. Experimenter. An environment for performing experiments andconducting.

4. Knowledge Flow. This environment supports essentially the samefunctions as the

Explorer but with a drag-and-drop interface. One advantage is that itsupports incremental learning.


4/43

2. The WEKA Explorer

Section Tabs

At the very top of the window, just below the title bar, is a row of tabs. When theExplorer is first started only the first tab is active; the others are greyed out. This is

because it is necessary to open (and potentially pre-process) a data set beforestarting to explore the data.

The tabs are as follows:1. Preprocess. Choose and modify the data being acted on.

2. Classify. Train and test learning schemes that classify or performregression.

3. Cluster. Learn clusters for the data.

4. Associate. Learn association rules for the data.

5. Select attributes . Select the most relevant attributes in the data.

6. Visualize . View an interactive 2D plot of the data.

Once the tabs are active, clicking on them flicks between differentscreens, on which the respective actions can be performed. The bottom areaof the window

(including the status box, the log button, and the Weka bird) stays visibleregardless of which section you are in.

Status Box

The status box appears at the very bottom of the window. It displaysmessages that keep you informed about whats going on. For example, if theExplorer is busy loading a file, the status box will say that.

TIP right-clicking the mouse anywhere inside the status box brings up a

little menu. The menu gives two options:1. Available memory. Display in the log box the amount of memory avail-able to WEKA.

2. Run garbage collector . Force the Java garbage collector to search for memory that is no longer needed and free it up, allowing more memoryfor new tasks. Note that the garbage collector is constantly running as a

background task anyway.


5/43

Log Button

Clicking on this button brings up a separate window containing ascrollable text field. Each line of text is stamped with the time it was entered intothe log. As you perform actions in WEKA, the log keeps a record of what hashappened.

WEKA Status IconTo the right of the status box is the WEKA status icon. When no processesare running, the bird sits down and takes a nap. The number beside the symbol gives the number of concurrent processes running. When the systemis idle it is zero, but it increases as the number of processes increases. Whenany process is started, the bird gets up and starts moving around. If itsstanding but stops moving for a long time, its sick: something has gonewrong! In that case you should restart the WEKA explorer.

3. Preprocessing

Opening files:The first three buttons at the top of the preprocess section enable you to loaddata into WEKA:

1. Open file.... Brings up a dialog box allowing you to browse for the datafile on the local filesystem.

2. Open URL.... Asks for a Uniform Resource Locator address for wherethe data is stored.

3. Open DB.... Reads data from a database. (Note that to make this work you might have to edit the file in weka/experiment/DatabaseUtils.props.)

Using the Open file... button you can read files in a variety of formats: WekasARFF format, CSV format, C4.5 format, or serialized Instances format. ARFF filestypically have a .arff extension, CSV files a .csv extension, C4.5 files a data and

names extension, and serialized Instances objects a .bsi extension.

The Current RelationOnce some data has been loaded, the Preprocess panel shows a variety of information. The Current relation box (the current relation is thecurrentlyloaded data, which can be interpreted as a single relational table indatabase terminology) has three entries:


6/43

1. Relation. The name of the relation, as given in the file it was loaded from.Filters (described below) modify the name of a relation.

2. Instances. The number of instances (data points/records) in the data.

3. Attributes . The number of attributes (features) in the data.

Working With Attributes

Below the Current relation box is a box titled Attributes. There are three buttons, and beneath them is a list of the attributes in the current relation.The list has three columns:

1. No.. A number that identifies the attribute in the order they are specifiedin the data file.

2. Selection tick boxes . These allow you select which attributes are presentin the relation.

3. Name . The name of the attribute, as it was declared in the data file.

When you click on different rows in the list of attributes, the fields change inthe box to the right titled Selected attribute. This box displays the char-acteristics of the currently highlighted attribute in the list:

1. Name. The name of the attribute, the same as that given in the attribute list.

2. Type . The type of attribute, most commonly Nominal or Numeric.

3. Missing. The number (and percentage) of instances in the data for whichthis attribute is missing (unspecified).

4. Distinct . The number of different values that the data contains for thisAttribute.

5. Unique. The number (and percentage) of instances in the data having avalue for this attribute that no other instances have.

Below these statistics is a list showing more information about the values stored inthis attribute, which differ depending on its type. If the attribute is nominal,the listconsists of each possible value for the attribute along with the number of instancesthat have that value. If the attribute is numeric, the list gives four statistics describing


7/43

the distribution of values in the datathe minimum,maximum, mean and standarddeviation. And below these statistics there is a colored histogram, color-codedaccording to the attribute chosen as the Class using the box above the histogram.(This box will bring up a drop-down list of available selections when clicked.) Notethat only nominal Class attributes will result in a color-coding. Finally, after pressingthe Visualize All button, histograms for all the attributes in the data are shown in aseparate witting. Returning to the attribute list, to begin with all the tick boxes areunticked.

They can be toggled on/off by clicking on them individually. The three buttons above can also be used to change the selection:

1. All. All boxes are ticked.

2. None. All boxes are cleared (unticked).

3. Invert. Boxes that are ticked become unticked and vice versa.

Once the desired attributes have been selected, they can be removed byclicking the Remove button below the list of attributes. Note that this can beundone by clicking the Undo button, which is located next to the Save

button in the top-right corner of the Preprocess panel.

Test Options

The result of applying the chosen classifier will be tested according to theoptionsthat are set by clicking in the Test options box. There are four test modes:

1. Use training set. The classifier is evaluated on how well it predicts theclass of the instances it was trained on.

2. Supplied test set . The classifier is evaluated on how well it predicts theclass of a set of instances loaded from a file. Clicking the Set... button

brings up a dialog allowing you to choose the file to test on.

3. Cross-validation. The classifier is evaluated by cross-validation, usingthe number of folds that are entered in the Folds text field.

4. Percentage split. The classifier is evaluated on how well it predicts acertain percentage of the data which is held out for testing. The amountof data held out depends on the value entered in the % field.


8/43

Note: No matter which evaluation method is used, the model that is output isalways the one build from all the training data. Further testing options can be set

by clickingon the More options... button:

1. Output model. The classification model on the full training set is outputso that it can be viewed, visualized, etc. This option is selected by default.

2. Output per-class stats. The precision/recall and true/false statisticsfor each class are output. This option is also selected by default.

3. Output entropy evaluation measures. Entropy evaluation measuresare included in the output. This option is not selected by default.

4. Output confusion matrix. The confusion matrix of the classifiers pre-

dictions is included in the output. This option is selected by default.

5. Store predictions for visualization. The classifiers predictions areremembered so that they can be visualized. This option is selected bydefault.

6. Output predictions. The predictions on the evaluation data are output. Note that in the case of a cross-validation the instance numbers do notcorrespond to the location in the data!

7. Cost-sensitive evaluation. The errors is evaluated with respect to acost matrix. The Set... button allows you to specify the cost matrix used.

1. Run information. A list of information giving the learning schemeoptions, relation name, instances, attributes and test mode that wereinvolved in the process.

2. Classifier model (full training set). A textual representation of theclassification model that was produced on the full training data.

3. The results of the chosen test mode are broken down thus:

4. Summary. A list of statistics summarizing how accurately the classifier was able to predict the true class of the instances under the chosen testmode.


9/43

5. Detailed Accuracy By Class. A more detailed per-class break down of the classifiers prediction accuracy.

6. Confusion Matrix. Shows how many instances have been assigned toeach class. Elements show the number of test examples whose actualclass is the row and whose predicted class is the column.

2) CREDIT RISK ASSESSMENT

The business of banks is making loans.Assessing the credit worthiness of anapplicantis of crucial importance.We have to develop a system to help a loanofficer decide whether the credit of a customer is good or bad.A banks

business rules regarding loans must consider two opposing factors.On theone hand,a bank wants to make as many loans as possible.Interest on theseloans is the banks profit source.On the other hand,a bank cannot afford tomake too many bad loans.To manybad could leads to the collapse of the

bank.The banks loan policy must involve acompromise;not too strict,andnot too lenient.

Credit risk is an investor's risk of loss arising from a borrower who does notmake payments as promised. Such an event is called a default. Other terms for credit risk are default risk and counterparty risk.

Credit risk is most simply defined as the potential that a bank borrower or counterparty will fail to meet its obligations in accordance with agreed terms.

The goal of credit risk management is to maximise a bank's risk-adjusted rate of return by maintaining credit risk exposure within acceptable parameters.

Banks need to manage the credit risk inherent in the entire portfolio as well asthe risk in individual credits or transactions.

Banks should also consider the relationships between credit risk and other risks.


10/43

The effective management of credit risk is a critical component of acomprehensive approach to risk management and essential to the long-termsuccess of any banking organisation.

A good credit assessment means you should be able to qualify, within the limitsof your income, for most loans.

Description of the German credit dataset in ARFF (Attribute Relation FileFormat) Format:

Structure of ARFF Format:%comment lines@relation relation name@attribute attribute name@DataSet of data items separated by commas.

% 1. Title: German Credit data%% 2. Source Information

%% Professor Dr. Hans Hofmann% Institut f"ur Statistik und "Okonometrie% Universit"at Hamburg%% 3. Number of Instances: 1000%% One datasets are provided. the original dataset, in the form provided% by Prof. Hofmann, contains categorical/symbolic attributes and% is in the file "german.data".%%% 6. Number of Attributes german: 20 (7 numerical, 13 categorical)% %%% 7. Attribute description for german%


11/43

% Attribute 1: (qualitative)% Status of existing checking account% A11 : ... < 0 DM% A12 : 0 = 200 DM /% salary assignments for at least 1 year % A14 : no checking account

% Attribute 2: (numerical)% Duration in month%% Attribute 3: (qualitative)% Credit history% A30 : no credits taken/% all credits paid back duly% A31 : all credits at this bank paid back duly

% A32 : existing credits paid back duly till now% A33 : delay in paying off in the past% A34 : critical account/% other credits existing (not at this bank)%% Attribute 4: (qualitative)% Purpose% A40 : car (new)% A41 : car (used)% A42 : furniture/equipment% A43 : radio/television% A44 : domestic appliances% A45 : repairs% A46 : education% A47 : (vacation - does not exist?)% A48 : retraining% A49 : business% A410 : others%

% Attribute 5: (numerical)% Credit amount%% Attibute 6: (qualitative)% Savings account/bonds% A61 : ... < 100 DM% A62 : 100


12/43

% A64 : .. >= 1000 DM% A65 : unknown/ no savings account%% Attribute 7: (qualitative)% Present employment since% A71 : unemployed% A72 : ... < 1 year % A73 : 1


13/43

% Attribute 14: (qualitative)% Other installment plans% A141 : bank % A142 : stores% A143 : none%% Attribute 15: (qualitative)% Housing% A151 : rent% A152 : own% A153 : for free%% Attribute 16: (numerical)% Number of existing credits at this bank %% Attribute 17: (qualitative)

% Job% A171 : unemployed/ unskilled - non-resident

% A172 : unskilled - resident% A173 : skilled employee / official% A174 : management/ self-employed/% highly qualified employee/ officer %% Attribute 18: (numerical)% Number of people being liable to provide maintenance for %% Attribute 19: (qualitative)% Telephone% A191 : none% A192 : yes, registered under the customers name%% Attribute 20: (qualitative)% foreign worker

% A201 : yes% A202 : no%%%% 8. Cost Matrix%% This dataset requires use of a cost matrix (see below)


14/43

%%% 1 2% ----------------------------% 1 0 1% -----------------------% 2 5 0%% (1 = Good, 2 = Bad)%% the rows represent the actual classification and the columns% the predicted classification.%% It is worse to class a customer as good when they are bad (5),% than it is to class a customer as bad when they are good (1).%

%%%%% Relabeled values in attribute checking_status% From: A11 To: '


15/43

% From: A45 To: repairs% From: A46 To: education% From: A47 To: vacation% From: A48 To: retraining% From: A49 To: business% From: A410 To: other %%% Relabeled values in attribute savings_status% From: A61 To: '


16/43

% From: A123 To: car % From: A124 To: 'no known property'%%% Relabeled values in attribute other_payment_plans% From: A141 To: bank % From: A142 To: stores% From: A143 To: none%%% Relabeled values in attribute housing% From: A151 To: rent% From: A152 To: own% From: A153 To: 'for free'%%

% Relabeled values in attribute job% From: A171 To: 'unemp/unskilled non res'% From: A172 To: 'unskilled resident'% From: A173 To: skilled% From: A174 To: 'high qualif/self emp/mgmt'%%% Relabeled values in attribute own_telephone% From: A191 To: none% From: A192 To: yes%%% Relabeled values in attribute foreign_worker % From: A201 To: yes% From: A202 To: no%%% Relabeled values in attribute class% From: 1 To: good

% From: 2 To: bad%

Subtasks ( Turn in your answers to the following tasks )

1. List all the categorical (or nominal) attributes and the real-valuedattributes seperately.


17/43

From the German Credit Assessment Case Study given to us,the followingattributes are found to be applicable for Credit-Risk Assessment:

Total Valid Attributes:-

1. checking_status2. duration3. credit history4. purpose5. credit amount6. savings_status7. employment duration8. installment rate9. personal status10. debitors

11. residence_since12. property14. installment plans15. housing16. existing credits17. job18. num_dependents19. telephone20. foreign worker

Categorical or Nomianal attributes( which takes True/false,Yes/no etcvalues ):-

1. checking_status2. credit history3. purpose4. savings_status5. employment6. personal status

7. debtors8. property9. installment plans10. housing11. job12. telephone13. foreign worker


18/43

Real valued attributes:-1. duration2. credit amount3. credit amount4. residence5. age6. existing credits7. num_dependents

2. What attributes do you think might be crucial in making the creditassessement ? Come up with some simple rules in plain English using yourselected attributes.

Sol:- According to me(Person to person varies) the following attributes may be crucial in making the credit risk assessment.1. Credit_history

2. Employment3. Property_magnitude4. job5. duration6. crdit_amount7. installment8. existing credit Based on the above attributes, we can make a decision whether to give creditor not.

3. One type of model that you can create is a Decision Tree - train a DecisionTree using the complete dataset as the training data. Report the modelobtained after training.

A decision tree is a flow chart like tree structure where each internal node(non-leaf)denotes a test on the attribute, each branch represents an outcome of thetest ,and each leaf node(terminal node)holds a class label.Decision trees can be easily converted into classification rules.e.g. ID3,C4.5 and CART.

J48 pruned tree1. Using WEKA Tool, we can generate a decision tree by selecting theclassify tab .

2. In classify tab select choose option where a list of different decisiontrees are available. From that list select J48 .


19/43

3. Now under test option ,select training data test option.

4. The resulting window in WEKA is as follows:

5. To generate the decision tree, right click on the result list and selectvisualize tree option by which the decision tree will be generated.


20/43

6. The obtained decision tree for credit risk assessment is very large to fit onthe

screen.


21/43

7. The decision tree above is unclear due to a large number of attributes.

4. Suppose you use your above model trained on the complete dataset, andclassify credit good/bad for each of the examples in the dataset. What % of examples can you classify correctly? (This is also called testing on thetraining set) Why do you think you cannot get 100 % training accuracy?

In the above model we trained complete dataset and we classified creditgood/bad for each of the examples in the dataset.

For example:IF

purpose=vacation THENcredit=badELSE

purpose=business THEN


22/43

Credit=good

In this way we classified each of the examples in the dataset.We classified 85.5% of examples correctly and the remaining 14.5% of examples areincorrectly classified. We cant get 100% training accuracy

because out of the 20 attributes, we have some unnecessaryattributesconsists of bad data are also been analyzed and trained.

Due to this the accuracy is affected and hence we cant get 100% trainingaccuracy.

5. Is testing on the training set as you did above a good idea? Why orWhy not?

According to the rules, for the maximum accuracy, we have to take 2/3 of the dataset as training set and the remaining 1/3 as test set. But here in theabove model we have taken complete dataset as training set which resultsonly 85.5% accuracy.This is done for the analyzing and training of the unnecessaryattributes(consists of bad data) which does not make a crucial role in creditrisk assessment. And by this complexity is increasing and finally it leads tothe minimum accuracy.

UseTraining Set Result for the table GermanCreditData:

Correctly Classified Instances 855 85.5 %

Incorrectly Classified Instances 145 14.5 %

Kappa statistic 0.6251

Mean absolute error 0.2312

Root mean squared error 0.34

Relative absolute error 55.0377 %

Root relative squared error 74.2015 %

Total Number of Instances 1000

6. One approach for solving the problem encountered in the previous questionis using cross-validation? Describe what cross-validation is briefly. Train aDecision Tree again using cross-validation and report your results. Doesyour accuracy increase/decrease? Why?


23/43

Cross validation:-In k-fold cross-validation, the initial data are randomly portioned into kmutuallyexclusive subsets or folds D1, D2, D3, . . . . . ., Dk. Each of approximatelyequal size.Training and testing is performed k times. In iteration I, partition Di isreserved as the test set and the remaining partitions are collectively used totrain the model.That is in the first iteration subsets D2, D3, . . . . . ., Dk collectively serve asthe training set in order to obtain as first model. Which is tested on Di. Thesecond trained on the subsets D1, D3, . . . . . ., Dk and test on the D2 and soon.

1. Select classify tab and J48 decision tree and in the test option select crossvalidation radio button and the number of folds as 10.

2. Number of folds indicates number of partition with the set of attributes.


24/43

3. Kappa statistics nearing 1 indicates that there is 100% accuracy and henceall the errors will be zeroed out, but in reality there is no such training setthat gives 100% accuracy.

Cross Validation Result at folds: 10 for the table GermanCreditData:

Correctly Classified Instances 705 70.5 %

Incorrectly Classified Instances 295 29.5 %

Kappa statistic 0.2467


Root mean squared error 0.4796

Relative absolute error 82.5233 %

Root relative squared error 104.6565 %

Total Number of Instances 1000

Here there are 1000 instances with 100 instances per partition.


25/43

Incorrectly Classified Instances 1 100 %

Kappa statistic 0


7. Check to see if the data shows a bias against "foreign workers" (attribute20), or "personal-status"(attribute 9). One way to do this (Perhaps rathersimple minded) is to remove these attributes from the dataset and see if thedecision tree created in those cases is significantly different from the fulldataset case which you have already done. To remove an attribute you canuse the reprocess tab in Weka's GUI Explorer. Did removing theseattributes have any significant effect? Discuss.

This increases in accuracy because the two attributes foreign workers and

personal statusare not much important in training and analyzing.By removing this, the time has been reduced to some extent and then itresults inincrease in the accuracy.The decision tree which is created is very large compared to the decision treewhichwe have trained now. This is the main difference between these twodecision trees.

After forign worker is removed, the accuracy is increased to 85.9%


26/43

If we remove 9 th attribute, the accuracy is further increased to 86.6% which showsthat these two attributes are not significant to perform training.


27/43

8. Another question might be, do you really need to input so many attributesto get good results? Maybe only a few would do. For example, you could try

just having attributes 2, 3, 5, 7, 10, 17 (and 21, the class attribute(naturally)). Try out some combinations. (You had removed two attributesin problem 7 Remember to reload the arff data file to get all the attributesinitially before you start selecting the ones you want.

Select attribute 2,3,5,7,10,17,21 and click on invert to remove the remaining

attributes.


28/43

Here accuracy is decreased.

9. Sometimes, the cost of rejecting an applicant who actually has a good

credit(case 1) might be higher than accepting an applicant who hasbad credit (case 2).Instead of counting the misclassifications equally inboth cases, give a higher cost to the first case (say cost 5) and lower costto the second case. You can do this by using a cost matrix in Weka.Train your Decision Tree again and report the Decision Tree and cross-validation results. Are they significantly different from results obtainedin problem 6 (using equal cost)?

In the Problem 6, we used equal cost and we trained the decision tree. But here,we consider two cases with different cost. Let us take cost 5 in case 1 and cost 2in case 2.

When we give such costs in both cases and after training the decision tree,wecan observe that almost equal to that of the decision tree obtained in problem6.Case1 (cost 5) Case2 (cost 5)


29/43

Total Cost 3820 1705Average cost 3.82 1.705

We dont find this cost factor in problem 6. As there we use equal cost. Thisis themajor difference between the results of problem 6 and problem 9. The cost matrices we used here:Case 1: 5 1

1 5 Case 2: 2 1

1 2

1.Select classify tab.2. Select More Option from Test Option.


30/43

3.Tick on cost sensitive Evaluation and go to set .


31/43

4.Set classes as 2.5.Click on Resize and then well get cost matrix.6.Then change the 2 nd entry in 1 st row and 2 nd entry in 1 st column to 5.07.Then confusion matrix will be generated and you can find out thedifference

between good and bad attribute.8.Check accuracy whether its changing or not.

10. Do you think it is a good idea to prefer simple decision treesinstead of having long complex decision trees? How does the

complexity of a Decision Tree relate to the bias of the model?

When we consider long complex decision trees, we will have manyunnecessary attributes in the tree which results in increase of the biasof the model. Because of this, the accuracy of the model can also

effected.This problem can be reduced by considering simple decision tree. Theattributes will be less and it decreases the bias of the model. Due tothis the result will be more accurate.

So it is a good idea to prefer simple decision trees instead of longcomplex trees.

1. Open any existing ARFF file e.g labour.arff.

2. In preprocess tab, select ALL to select all the attributes.3. Go to classify tab and then use traning set with J48algorithm.


32/43


33/43

4. To generate the decision tree, right click on the result list and selectvisualize tree option, by which the decision tree will be generated.

5. Right click on J48 algorithm to get Generic Object Editor window

6. In this,make the unpruned option as true .7. Then press OK and then start . we find the tree will become morecomplex if not pruned.


34/43


35/43

Visualize tree


36/43

8. The tree has become more complex.

11.You can make your Decision Trees simpler by pruning the nodes. One

approach is to use Reduced Error Pruning - Explain this idea briefly.Try reduced error pruning for training your Decision Trees using cross-validation (you can do this in Weka) and report the Decision Tree youobtain ? Also, report your accuracy using the pruned model. Does youraccuracy increase ?

Reduced-error pruning:-

The idea of using a separate pruning set for pruningwhich is applicable to decisiontrees as well as rule setsis called reduced-error pruning. The variant described

previously prunes a rule immediately after it has been grown and is calledincremental reduced-error pruning.

Another possibility is to build a full, unpruned rule set first, pruning it afterwards bydiscarding individual tests.

However, this method is much slower. Of course, there are many different ways toassess the worth of a rule based on the pruning set. A simple measure is to consider how well the rule would do at discriminating the predicted class from other classes if

it were the only rule in the theory, operating under the closed world assumption.

If it gets p instances right out of the t instances that it covers, and there are Pinstances of this class out of a total T of instances altogether, then it gets positiveinstances right. The instances that it does not cover include N - n negative ones,where n = t p is the number of negative instances that the rule covers and N = T - Pis the total number of negative instances.

Thus the rule has an overall success ratio of [p +(N - n)] T , and this quantity,evaluated on the test set, has been used to evaluate the success of a rule when usingreduced-error pruning.

1. Right click on J48 algorithm to get Generic Object Editor window

2. In this,make reduced error pruning option as true and also the unprunedoption as true .


37/43

3. Then press OK and then start.


38/43

4. We find that the accuracy has been increased by selecting the reduced error pruning option.

12. (Extra Credit): How can you convert a Decision Trees into "if-then-else

rules". Make up your own small Decision Tree consisting of 2-3 levelsand convert it into a set of rules. There also exist different classifiersthat output the model in the form of rules - one such classifier in Wekais rules. PART, train this model and report the set of rules obtained.Sometimes just one attribute can be good enough in making thedecision, yes, just one ! Can you predict what attribute that might be inthis dataset? OneR classifier uses a single attribute to make decisions (itchooses the attribute based on minimum error). Report the ruleobtained by training a one R classifier. Rank the performance of j48,

PART and oneR.

In weka, rules.PART is one of the classifier which converts thedecision trees into IF-THEN-ELSE rules.

Converting Decision trees into IF-THEN-ELSE rules usingrules.PART classifier:-


39/43

PART decision listoutlook = overcast: yes (4.0)windy = TRUE: no (4.0/1.0)outlook = sunny: no (3.0/1.0): yes (3.0)

Number of Rules : 4Yes, sometimes just one attribute can be good enough in making thedecision.In this dataset (Weather), Single attribute for making the decision isoutlookoutlook:sunny -> noovercast -> yesrainy -> yes(10/14 instances correct)With respect to the time , the oneR classifier has higher ranking and

J48 is in 2nd

place and PART gets 3rd place.J48 PART oneR TIME (sec) 0.12 0.14 0.04RANK II III IBut if you consider the accuracy, The J48 classifier has higher ranking, PART gets second place and oneR gets lst placeJ48 PART oneR ACCURACY (%) 70.5 70.2% 66.8%

1.Open existing file as weather.nomial.arff

2.Select All .

3.Go to classify .

4.Start.


40/43


41/43

Here the accuracy is 100%


42/43

The tree is something like if-then-else rule

If outlook=overcast then play=yes

If outlook=sunny and humidity=high

Then play = no

Else play = yes

If outlook=rainy and windy=true

Then play = noElse play = yes

To click out the rules


43/43

1. Go to choose then click on Rule then select PART.

2. Click on Save and start.

3. Similarly for oneR algorithm.

dmdw record

Documents