Here's a nice paper on Pubmed Central discussing issues surrounding inverse normal transforms. In general, the idea is to match the normalized rank of the trait to a quantile in a normal distribution. Have a look at equation 1 in the paper I linked to.
I think the paper mentioned by Devon Ryan worths to read.
To do INT(inverse-normal transformation), you can use the R code below:
See the supplement of this paper - Yang et al. Nature 2012.