Error in check for remote errors (val): 5 nodes produced an error: object not found
Very related questions have already been answered on Stack Overflow. Basically, you have three solutions:
- use
clusterExport()
to export the variables you need to the clusters (the most common method) - pass all variables as arguments of your function
CrossValJM()
so that they are automatically exported to the clusters (the solution I prefer, the most programmatically correct one) - use R package {future} which should detect automatically variables to export (the lazy solution, but seems to work well also)
See for example this.
Related videos on Youtube
Oesj
Updated on June 04, 2022Comments
-
Oesj almost 2 years
Im trying to do a 10-fold cross validation and estimate the model performance of a joint model by using parallel processing (parLapply). Im trying to find out why I receive the error message: "Error in checkForRemoteErrors(val): five nodes produced an error: object 'Week' not found"
The code looks as follows:
# Validation using 10-fold CV library("parallel") set.seed(123) V <- 10 n <- nrow(dfC) splits <- split(seq_len(n), sample(rep(seq_len(V), length.out = n))) CrossValJM <- function (i) { library("JM") library("nlme") trainingData <- dfL[!dfL$ID %in% i, ] trainingData_ID <- trainingData[!duplicated(trainingData$ID), ] testingData <- dfL[dfL$ID %in% i, ] lmeFit <- lme(DA ~ ns(Week, 2), data = trainingData, random = ~ ns(Week, 2) | ID) coxFit <- coxph(Surv(TT_event, Event) ~ Gender * Age, data = trainingData_ID, x = TRUE) jointFit <- jointModel(lmeFit, coxFit, timeVar = "Week") pe <- prederrJM(jointFit, newdata = testingData, Tstart = 10, Thoriz = 20) auc <- aucJM(jointFit, newdata = testingData, Tstart = 10, Thoriz = 20) list(pe = pe, auc = auc) } cl <- makeCluster(5) res <- parLapply(cl, splits, CrossValJM) stopCluster(cl)
The function itself gets accepted but when running the Cluster commands I run into this error that mentions that it cannot recognize objects given within the function.. should they be defined within the function itself?? Or am I not using the parLapply function correctly?
P.S.: data looks as follows (dfL is a dataframe of length ~ 1000 and dfC ~ 200):
dfL <- data.frame(ID = c(1, 1, 1, 2, 2, 3), DA = c(0.4, 1.8, 1.2, 3.2, 3.6, 2.8), Week = c(0, 4, 16, 4, 20, 8), Event = c(1, 1, 1, 0, 0, 1), TT_Event = c(16, 20, 8), Gender = c(0, 0, 0, 1, 1, 0), Age = c(24, 24, 24, 56, 56, 76)) dfC <- data.frame(ID = c(1, 2, 3, 4, 5, 6), DA = c(1.2, 3.6, 2.8, 2.4, 1.9, 3.4), Week = c(16, 20, 8, 36, 24, 32), Event = c(1, 0, 1, 1, 1, 0), TT_Event = c(16, 20, 8, 36, 24, 32), Gender = c(0, 1, 0, 0, 1, 1), Age = c(24, 56, 76, 38, 44, 50))
Thnx :)
-
Oesj over 6 yearsThnx F.Privé.. but please read the other post above... already tried something like this