Skip to content

easy way to limit sampling to a maximum time duration #474

Description

@jpritikin

Is your feature request related to a problem? Please describe.
I have a model that often runs in a reasonable amount of time, but occasionally, when faced with unpleasant data, will continue to sample for 4+ hours. Rather than wait, I'd prefer just to record the failure and move on to the next dataset.

Describe the solution you'd like
I'd like to be able to bound the maximum time duration for sampling.

Describe alternatives you've considered
Maybe there is some R package that wrap cmdstanr's sampling API and offer this functionality? Oh, hm, I found withTimeout in the R.utils package.

Activity

  1. rok-cesnovar commented on Mar 23, 2021

    @rok-cesnovar
    Member

    Thanks for the feature request. Seems like a reasonable request to me.

    What do you envision should happen after the sampling is canceled? Should the fit and CSV file be discarded or should it be possible to read in the samples?

  2. jpritikin commented on Mar 23, 2021

    @jpritikin
    Author

    Should the fit and CSV file be discarded or should it be possible to read in the samples?

    I'm am against discarding anything automatically.

    What do you envision should happen after the sampling is canceled?

    For my application, I just need to know what happened so I can record the error.

  3. mike-lawrence commented on Mar 23, 2021

    @mike-lawrence
    Collaborator

    Related: #426

  4. jpritikin commented on Mar 23, 2021

    @jpritikin
    Author

    Until cmdstanr gets some improvement, here's a solution:

    library(R.utils)
    
    fit <- try(withTimeout(m1$sample(dl, chains = 4, parallel_chains = 4,
                     iter_warmup = totalIter/2, iter_sampling = totalIter/2,
    #                 metric="diag_e", inv_metric = im.prior,
                     init=function(chain_id) list(B=0,eVar=.95),
                     output_dir = localDir), timeout=60*60, cpu=Inf), silent = TRUE)
    
    if (is(fit, "try-error")) {
      recFail(paste("Timeout during sampling"))
    }
    
  5. added this to the future milestone on Jul 6, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    featureNew feature or request

    Type

    No type

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions