Problems with R (protect() -> stackoverflow)

Question

[Esra]

Hello people! I try to train models with Rstudio with the following code:

set.seed(17)
trctrl <- trainControl(method = "none")
rfFit <- train(subtype~., data = training, method = "ranger", trControl=trctrl, importance="permutation"
tuneGrid = data.frame(mtry=100, min.node.size = 1, `splitrule="gini")

And I am running always in the ame problem (even with other models I try to train):

ERROR: protect() protection stack overflow.

Of course I googled the problem and found some people that said the solution would be to increase the memory size (rstudio.exe --max-ppsize=5000000) what I did with the command "R --max-ppsize=5000000" in my terminal, since I am using linux. But it still didnt worked even when I ran the whole code thing again in my R terminal without Rstudio. Now two questions:

1. Does someone has an idea how to solve the problem described above
2. How can I start a new R-Studio Notebook with a new ppsize, since I just managed to do so in my terminal, where I was not sure if the command I provieded above is acutally the right one to solve this "protection" problem.