Question: Compare the variances of multiple categorical distributions in a repeated measure design
0
gravatar for JohannesWiesner
9 weeks ago by
JohannesWiesner0 wrote:

I ran three model-building procedures with different parameters on the same sample and obtained the selection of my optimized hyperparameter for each outer fold (each of the analyses had 100 outer folds in my nested-cross validation). This gives me a contingency table, with the models as rows and the absolute frequencies for each of the possible hyperparameter categories as columns. For the sake of simplicity, let's call the rows day 1, day 2, and day 3 and let the columns be different ice cream sorts and let the 100 folds be the same sample of people buying ice cream on three different days:

enter image description here

As you can 'see', on in order of the days, the distribution becomes less 'spread out'.

I would like to measure this difference in variance, but I am not sure, which test to used. I came upon Chi2, McNemar and Cochrane's Q. Chi2 is not for repeated measures, McNemar only works for 2x2 tables and McNemar & Cochrane's Q are apparently designed for binary responses. Does anybody know which test might be appropriate for my contingency table?

machine learning • 78 views
ADD COMMENTlink modified 9 weeks ago by ATpoint35k • written 9 weeks ago by JohannesWiesner0

I could not embed the image in my post, that's why I just pasted the link.

ADD REPLYlink written 9 weeks ago by JohannesWiesner0

Right-click the image on the image hoster website to get the ful link (https://i.imgur.com/Hjt6RBO.png), then paste this into the image field. I made the change for you.

enter image description here

Even though we have some members here with statistics expertise I guess you could get more responses when posting this over at Cross-Validated: https://stats.stackexchange.com/

ADD REPLYlink modified 9 weeks ago • written 9 weeks ago by ATpoint35k

Awesome, thank you! :)

ADD REPLYlink written 9 weeks ago by JohannesWiesner0

Good idea! I tried it already on https://datascience.stackexchange.com/, but this forum seems to be quite inactive. Maybe https://stats.stackexchange.com/ is a better forum for my question.

ADD REPLYlink written 9 weeks ago by JohannesWiesner0
Please log in to add an answer.

Help
Access

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.
Powered by Biostar version 2.3.0
Traffic: 1082 users visited in the last hour