GitHub Actions-installed miniconda on ubuntu defies assumptions, and a regex that should probably be fixed

[krivard]

My lab has a project which uses GitHub Actions to build a blog from Rmarkdown files. Some of the source files include Python segments, which get built with reticulate. The last successful build was May 23, then we had a long quiet period before needing to build the blog again a couple of weeks ago. That build failed with:

Error in normalizePath(conda, winslash = "/", mustWork = TRUE) : 
  path[1]="# cmd: /usr/share/miniconda/condabin/mamba update --name www-main --file /home/runner/work/www-main/www-main/conda": No such file or directory
Calls: local ... python_munge_path -> get_python_conda_info -> normalizePath
Execution halted

I've assembled a reproducible example in a dedicated GitHub repo(and here's the workflow YAML as a static file in case that repo rots), but I have not been able to replicate this particular environment quirk in a system I can actually shell into, so unfortunately debugging is likely to be slow going.

I have managed to partially trace the problem to reticulate:::python_info_condaenv_find, which uses a more general regex to fetch lines from the history file than the one it uses to parse that line:

reticulate/R/python-tools.R

Lines 176 to 182 in a1d7f7f

	pattern <- "^[[:space:]]*#[[:space:]]*cmd:[[:space:]]*"
	lines <- grep(pattern, history, value = TRUE)
	if (length(lines) == 0)
	return(NULL)
	# get path to conda script used
	script <- sub("^#\\s+cmd: (.+)\\s+create\\s+.*", "\\1", lines[[1]])

For some reason, the system resulting from running conda-incubator/setup-miniconda@v2 on ubuntu-latest results in a history file where the first cmd line is a mamba update, not a create. The sub function at python_info_condaenv_find:182 thus substitutes nothing and passes the history line to normalizePath wholesale, causing the above misbehavior.

It also looks like there is no create line in my conda history file(screenshot below), so fixing the regex will only go so far. I've tried a bunch of other ways to tell reticulate how to configure itself for this environment but haven't found one that works yet. Any tips?

[krivard]

Things that didn't work:

• Setting reticulate.conda_binary
• use_condaenv
• use_miniconda
• use_python

For future travelers: I got this working in the meantime by switching to mambaforge, which seems to generate the expected content in the history file:

      - uses: conda-incubator/setup-miniconda@v2
        with:
          python-version: 3.9
          activate-environment: www-main
          miniforge-variant: Mambaforge
          miniforge-version: latest
          use-mamba: true
          use-only-tar-bz2: true # (for caching support)
      - name: Update environment
        run: mamba env update -n www-main -f environment.yml 

It would still be nice if reticulate made more consistent assumptions though -- if I use the provided hint functions, it shouldn't still try to locate conda through python_info_condaenv_find.

[kevinushey]

It would still be nice if reticulate made more consistent assumptions though -- if I use the provided hint functions, it shouldn't still try to locate conda through python_info_condaenv_find.

Thanks for the diagnosis! I agree; we should definitely be reading + respecting these. Presumedly, we could check reticulate.conda_binary first here:

reticulate/R/python-tools.R

Lines 148 to 166 in a1d7f7f

	# first, check if we have a condabin
	exe <- if (is_windows()) "conda.exe" else "conda"
	conda <- file.path(path, "condabin", exe)
	if (file.exists(conda))
	return(conda)
	if (is_windows()) {
	# in Anaconda base env, conda.exe lives under Scripts
	conda <- file.path(path, "Scripts", exe)
	if (file.exists(conda))
	return(conda)
	# in ArcGIS env, conda.exe lives in a parent directory
	conda <- file.path(path, "../..", "Scripts", exe)
	conda <- normalizePath(conda, winslash = "/", mustWork = FALSE)
	if (file.exists(conda))
	return(conda)
	}

[bschilder]

@kevinushey @t-kalinowski I've been running into this same issue with the rworkflows action. Just wondering, has a fixed been pushed to CRAN yet?

• https://github.com/neurogenomics/rworkflows
• Add step to enable conda envs neurogenomics/rworkflows#78

In my case, I noticed that reticulate wasn't recognizing that conda was installed when running the conda_create function: