Error with parallel processing with multidog

[stephenrkeller]

Hi David,

I'm trying to get multidog running on my Unix server (Ubuntu 18.04.4 LTS; 32 cores; 64Gb Mem). When I go to run the example data ("uitdewilligen) using code identical to the multidog vignette, and specifying nc = 2, I get the following error:

    |                                   *.#,%    
   |||                                 *******/  
 |||||||    (**..#**.                  */   **/  
|||||||||    */****************************/*%   
   |||    &****..,*.************************/    
   |||     (....,,,*,...****%********/(******    
   |||                ,,****%////,,,,./.****/    
   |||                  /**//         .*///....  
   |||                  .*/*/%#         .,/   ., 
   |||               , **/   #%         .*    .. 
   |||                               ,,,*        

Working on it...Error in { : task 1 failed - "argument is missing, with no default"
Calls: multidog -> %dorng% -> do.call -> %dopar% -> <Anonymous>
Execution halted

Versions:

-R version 3.4.4 (running R either interactive at the commandline or in Rscript; both produce the same error)
-updog 2.1.0
-future 1.21.0

I assume this relates to how the parallel processing is being implemented on my system. When I run a test with flexdog, everything is fine and I get expected outputs.

FYI, there isn't a lot of load on the system right now, so I don't think it's a memory issue. Any help would be greatly appreciated.

Thanks!
Steve

[dcgerard]

Hey @stephenrkeller, thanks for trying out the dev version of updog!

Hmm, could you try running the vignette code with the following settings from the future package?

future::plan(future::sequential)
mout <- multidog(refmat = refmat, 
                 sizemat = sizemat, 
                 ploidy = ploidy, 
                 model = "norm",
                 nc = NA)

This will let me see if the issue is because of the parallelization, or because I introduced a bug in the changes I made a couple weeks ago.

Some other possibilities:

1. You could also try a different strategy for parallelization. Like using multicore from the second example in the vignette.

   future::plan(future::multicore, workers = 2)
   mout <- multidog(refmat = refmat, 
                    sizemat = sizemat, 
                    ploidy = ploidy, 
                    model = "norm",
                    nc = NA)
   
   ## Shut down parallel workers
   future::plan(future::sequential)

2. Are the doRNG, doFuture, foreach, and iterators packages all up-to-date?

[stephenrkeller]

Thanks for the quick reply, David!

I tried your suggestions. Here's the output from the first suggestions changing the settings on 'future':

Working on it...Loading required package: foreach
Loading required package: rngtools
Loading required package: methods
Error in { : task 1 failed - "argument is missing, with no default"
Calls: multidog -> %dorng% -> do.call -> %dopar% -> <Anonymous>
Execution halted

And for the second example following the vignette using 'multicore':

Working on it...Loading required package: foreach
Loading required package: rngtools
Loading required package: methods
Loading required package: foreach
Loading required package: rngtools
Loading required package: methods
Error in { : task 1 failed - "argument is missing, with no default"
Calls: multidog -> %dorng% -> do.call -> %dopar% -> <Anonymous>
Execution halted

All those packages are up to date.

Thanks for the help here!

[dcgerard]

Sorry about this. I guess my only other idea is to try out the version on CRAN --- it's possible that the issue is with the future and iterators packages that I started using in the GitHub version, but I'm not sure. I'll ask around for ideas. But without access to the computing environment, I'm not sure if I'll be able to solve this one.

[stephenrkeller]

OK, thanks taking a look at it David. I'll revert back to using the stable release version for now.