I ran three model-building procedures with different parameters on the same sample and obtained the selection of my optimized hyperparameter for each outer fold (each of the analyses had 100 outer folds in my nested-cross validation). This gives me a contingency table, with the models as rows and the absolute frequencies for each of the possible hyperparameter categories as columns. For the sake of simplicity, let's call the rows day 1, day 2, and day 3 and let the columns be different ice cream sorts and let the 100 folds be the same sample of people buying ice cream on three different days:
As you can 'see', on in order of the days, the distribution becomes less 'spread out'.
I would like to measure this difference in variance, but I am not sure, which test to used. I came upon Chi2, McNemar and Cochrane's Q. Chi2 is not for repeated measures, McNemar only works for 2x2 tables and McNemar & Cochrane's Q are apparently designed for binary responses. Does anybody know which test might be appropriate for my contingency table?