Question: SVM caret training set confusion matrix
1
gravatar for Matina
3.1 years ago by
Matina160
United Kingdom/University of Edinburgh
Matina160 wrote:

Hi guys,

I am using the caret package for binary classification on my RNA-seq data (59 samples x 15 features). I am trying to figure out the confusion matrix of the cross validation but I cant really seem to be able to find it.

Is this command correct in order to get the confusion matrix?

confusionMatrix(predict(final_svmFit_radial, new_data), class)

I am training the model using the following commands

fitControl <- trainControl(method = "repeatedcv",
                       number = 10,
                       repeats = 10,
                       classProbs = TRUE,
                       savePred=TRUE,
                       returnResamp = "all",
                       summaryFunction = twoClassSummary)



set.seed(123)
final_svmFit_radial = train(class~.,  new_data,
                         method = "svmRadial",
                         trControl = fitControl,
                         prox=TRUE,
                         allowParallel=TRUE,
                         preProc = c("center", "scale"),
                         tuneLength = 9,
                         metric = "ROC")

This are the resampling results

> final_svmFit_radial
Support Vector Machines with Radial Basis Function Kernel 

59 samples
15 predictors
 2 classes: 'Cancer', 'Normal' 

Pre-processing: centered (15), scaled (15) 
Resampling: Cross-Validated (10 fold, repeated 10 times) 
Summary of sample sizes: 53, 53, 53, 53, 54, 53, ... 
Resampling results across tuning parameters:

  C      ROC  Sens    Spec  ROC SD  Sens SD  Spec SD
   0.25  1    0.9975  1     0       0.025    0      
   0.50  1    1.0000  1     0       0.000    0      
   1.00  1    1.0000  1     0       0.000    0      
   2.00  1    1.0000  1     0       0.000    0      
   4.00  1    1.0000  1     0       0.000    0      
   8.00  1    1.0000  1     0       0.000    0      
  16.00  1    1.0000  1     0       0.000    0      
  32.00  1    1.0000  1     0       0.000    0      
  64.00  1    1.0000  1     0       0.000    0      

Tuning parameter 'sigma' was held constant at a value of 0.05198751
ROC was used to select the optimal model using  the largest value.
The final values used for the model were sigma = 0.05198751 and C = 0.25.

and this is the final model

> final_svmFit_radial$finalModel
Support Vector Machine object of class "ksvm" 

SV type: C-svc  (classification) 
 parameter : cost C = 0.25 

Gaussian Radial Basis kernel function. 
 Hyperparameter : sigma =  0.0519875137214014 

Number of Support Vectors : 45 

Objective Function Value : -6.7424 
Training error : 0.050847 
Probability model included.

Thank you in advance! Matina

caret classification svm • 2.5k views
ADD COMMENTlink modified 3.1 years ago • written 3.1 years ago by Matina160

Actually I think the correct way to find the confusion matrix of the cross validation is like this, please correct me if im wrong

confusionMatrix(final_svmFit_radial$pred$pred, final_svmFit_radial$pred$obs)



 Confusion Matrix and Statistics

          Reference
Prediction Cancer Normal
    Cancer   3330      0
    Normal      0   1980

               Accuracy : 1          
                 95% CI : (0.9993, 1)
    No Information Rate : 0.6271     
    P-Value [Acc > NIR] : < 2.2e-16  

                  Kappa : 1          
 Mcnemar's Test P-Value : NA         

            Sensitivity : 1.0000     
            Specificity : 1.0000     
         Pos Pred Value : 1.0000     
         Neg Pred Value : 1.0000     
             Prevalence : 0.6271     
         Detection Rate : 0.6271     
   Detection Prevalence : 0.6271     
      Balanced Accuracy : 1.0000     

       'Positive' Class : Cancer
ADD REPLYlink modified 3.1 years ago • written 3.1 years ago by Matina160
Please log in to add an answer.

Help
Access

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.
Powered by Biostar version 2.3.0
Traffic: 1169 users visited in the last hour