**140**wrote:

Hello,

I explain what I did (with R) :

1 - I have a matrix of abundances of OTUs (more than 817.000 columns) . I need to compute the proportionality between these OTUs. For the moment, I can split a matrix in submatrices in order to compute the proportionality between each of these submatrices , and then, get the final matrix.

```
data=matrix(runif(10000), ncol=1000) #random matrix
data=clr(data)
ncol<-ncol(data)
rest<-ncol%%100 #100 columns by submatrix
blocks<-ncol%/%100 #10 submatrices
ngroup <- rep(1:blocks, each = 100)
#if (rest>0) ngroup<-c(ngroup,rep(blocks+1,rest))
split <- split(1:ncol, ngroup)
#I get all the combinations between my submatrices
combs <- expand.grid(1:length(split), 1:length(split))
combs <- t(apply(combs, 1, sort))
combs <- unique(combs)
combs <- combs[combs[,1] != combs[,2],]
res <- foreach(i = seq_len(nrow(combs))) %dopar% {
G1 <- split[[combs[i,1]]]
G2 <- split[[combs[i,2]]]
dat.i <- cbind(data[,G1], data[,G2])
rho <- cor_rho(dat.i) #cor_rho is my function to compute the proportionality
}
**And then, I get the final matrix :**
resMAT <- matrix(0, ncol(data), ncol(data))
for(i in 1:nrow(combs)){
batch1 <- split[[combs[i,1]]]
batch2 <- split[[combs[i,2]]]
patch.i <- c(batch1, batch2)
resMAT[patch.i, patch.i] <- res[[i]]
}
```

2 - I work with Slurm on a cluster with several nodes. I know that with a node (256G and 32CPUs, in one day, I can compute the proportionality between 60.000 columns). So, I need to use 817.000/60.000 ~ 14 submatrices which gives me (14*13)/2 = 91 combinations (=91 nodes).

3 - I don't know how I could use SLURM to create a SLURM code in order to distribute each combination calculation on a node.

Any advice ?

Bests, Vincent

**23k**• written 18 months ago by pablo •

**140**

Is there a reason you haven't asked your cluster admin for the appropriate syntax for your cluster?

97kActually, he gave me some pieces of advice but I'm still struggling with the SLURM syntax..

140Couldn't you just submit the job 14 times (with different input files being your 14 different submatrices)? This would require 14 different SLURM submission scripts, which would probably take less time to write than a Python or Perl script to automatically generate the 14 scripts.

1.4kI would like to stay with my R code, and incorporate it in a SLURM code, if it possible?

140`resMAT[patch.i, patch.i] <- res[[i]]`

- you're only going to update entries on the diagonal of resMAT5.5kBut it means my final matrix is false?

140ARG sorry, my mistake

5.5k