SVM caret training set confusion matrix

Question

[Matina]

Hi guys,

I am using the caret package for binary classification on my RNA-seq data (59 samples x 15 features). I am trying to figure out the confusion matrix of the cross validation but I cant really seem to be able to find it.

Is this command correct in order to get the confusion matrix?

confusionMatrix(predict(final_svmFit_radial, new_data), class)

I am training the model using the following commands

fitControl <- trainControl(method = "repeatedcv",
                       number = 10,
                       repeats = 10,
                       classProbs = TRUE,
                       savePred=TRUE,
                       returnResamp = "all",
                       summaryFunction = twoClassSummary)



set.seed(123)
final_svmFit_radial = train(class~.,  new_data,
                         method = "svmRadial",
                         trControl = fitControl,
                         prox=TRUE,
                         allowParallel=TRUE,
                         preProc = c("center", "scale"),
                         tuneLength = 9,
                         metric = "ROC")

This are the resampling results

> final_svmFit_radial
Support Vector Machines with Radial Basis Function Kernel 

59 samples
15 predictors
 2 classes: 'Cancer', 'Normal' 

Pre-processing: centered (15), scaled (15) 
Resampling: Cross-Validated (10 fold, repeated 10 times) 
Summary of sample sizes: 53, 53, 53, 53, 54, 53, ... 
Resampling results across tuning parameters:

  C      ROC  Sens    Spec  ROC SD  Sens SD  Spec SD
   0.25  1    0.9975  1     0       0.025    0      
   0.50  1    1.0000  1     0       0.000    0      
   1.00  1    1.0000  1     0       0.000    0      
   2.00  1    1.0000  1     0       0.000    0      
   4.00  1    1.0000  1     0       0.000    0      
   8.00  1    1.0000  1     0       0.000    0      
  16.00  1    1.0000  1     0       0.000    0      
  32.00  1    1.0000  1     0       0.000    0      
  64.00  1    1.0000  1     0       0.000    0      

Tuning parameter 'sigma' was held constant at a value of 0.05198751
ROC was used to select the optimal model using  the largest value.
The final values used for the model were sigma = 0.05198751 and C = 0.25.

and this is the final model

> final_svmFit_radial$finalModel
Support Vector Machine object of class "ksvm" 

SV type: C-svc  (classification) 
 parameter : cost C = 0.25 

Gaussian Radial Basis kernel function. 
 Hyperparameter : sigma =  0.0519875137214014 

Number of Support Vectors : 45 

Objective Function Value : -6.7424 
Training error : 0.050847 
Probability model included.

Thank you in advance! Matina

[Matina]

Actually I think the correct way to find the confusion matrix of the cross validation is like this, please correct me if im wrong

confusionMatrix(final_svmFit_radial$pred$pred, final_svmFit_radial$pred$obs)



 Confusion Matrix and Statistics

          Reference
Prediction Cancer Normal
    Cancer   3330      0
    Normal      0   1980

               Accuracy : 1          
                 95% CI : (0.9993, 1)
    No Information Rate : 0.6271     
    P-Value [Acc > NIR] : < 2.2e-16  

                  Kappa : 1          
 Mcnemar's Test P-Value : NA         

            Sensitivity : 1.0000     
            Specificity : 1.0000     
         Pos Pred Value : 1.0000     
         Neg Pred Value : 1.0000     
             Prevalence : 0.6271     
         Detection Rate : 0.6271     
   Detection Prevalence : 0.6271     
      Balanced Accuracy : 1.0000     

       'Positive' Class : Cancer