Neural networks feature importance
1
0
Entering edit mode
6.2 years ago
druggable ▴ 60

Hi everyone,

I am using a neural network with two output neurons for my omics data to predict the binary classification of genes. It is important in the study to know the feature importance. I know that the Garson's algorithm and connection weights algorithm are used in neural networks with one hidden layer, but in all the examples I've seen there is only one output neuron. My neural network has two output neurons. Can I still use these algorithms to evaluate feature importance in my neural network?

Thanks, teabonng

neural network • 1.7k views
ADD COMMENT
0
Entering edit mode

I could be wrong but I sense a misunderstanding here. A neural network for binary classification typically learns an output distribution, not a function. So their should only be one output, probability of a positive output given the inputs. 1 minus this value would then be the probability of a negative output. It is done this way since gradient descent with back propagation needs smooth outputs.

ADD REPLY
0
Entering edit mode

Multiple outcomes are useful if the classification is not binary. Perhaps there are two dimensions being evaluated simultaneously. Multiple classification can use as many outputs as classes in a "one-hot" encoding. If the training algorithm needs a single value for goodness of fit, the multiple output neurons should be summarized.

ADD REPLY
0
Entering edit mode

I agree. But in this case he said 'binary classification'.

ADD REPLY
1
Entering edit mode
6.2 years ago

You should be able to use your preferred algorithm to evaluate your network if you customize it to suit the structure. Either the algorithm needs adjustment or the network structure.

Typically one output neuron is sufficient to classify something to binary, since it will read 0 or -1 for FALSE, and 1 for TRUE. Two neurons give you the opportunity to be TRUE for two things, and 2^2=4 possible binary outcomes.

ADD COMMENT
0
Entering edit mode

Hi karl.stamm,

Can you provide an example on how to adapt Garson's algorithm for multiple neurons? Can I calculate the importances with respect to each neuron instead?

Thanks

ADD REPLY
0
Entering edit mode

Sorry I am not familiar with anything called Garson. So it's not clear what you mean by importances. I use least-squares optimization (minimize sum of squared differences) with customized endpoints, sometimes using what TensorFlow calls a softmax activation function.

ADD REPLY

Login before adding your answer.

Traffic: 1199 users visited in the last hour
Help About
FAQ
Access RSS
API
Stats

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.

Powered by the version 2.3.6