Recording of build instructions

[joergfunger]

I was using the provenance plugin to create a Workflow Run RO-Crate to store the provenance information of a nextflow workflow. If tried the for me relevant sparql queries on the results, but there are some issues and I'm not sure whether this is due to me using outdated queries, or if there are some missing features in the plugin (or I'm using the plugin with a wrong configuration), or there are still some compatibility issues.

cq4: What is the environment/container file used in a specific workflow execution step?
The query essentially searches for MediaObjects or buildInstructions, both of which are not present in the ROCrate. (I've added a conda directive to each process, so I expected this to be documented somewhere, see also cq.10), I also tried to search in the provenance plugin where this information is stored but was not successful.

cq5: How long does this workflow component take to run?
The query searches for an action that is executed by a tool (main.nf), though there the tool is a http://schema.org/SoftwareApplication, where in my ROCrate this is a https://bioschemas.org/ComputationalWorkflow. Which version is correct? (is one maybe outdated)?

cq8: "What are the inputs and outputs of the overall workflow?"
For the outputs, the query tries to find the CreateActions where the instument is a ComputationalWorkflow (main.nf). In our case, this createAction has no results, the results are only stored on the level of the individual processes, but those do not have an instrument, as such the output of the query is empty.

cq10: What is the script used to wrap up a software component?
The example query is searching for softwareRequirements, which is not present in the ROCrate. In our case we are interested in the actual script (with the complete list of arguments) that was executed in a workflow run and the conda environment/docker image this script was executed with.

The current setup is work in progress, on a more complex project, a nextflow workflow (under development) is here, the provenance file is attached. If it helps, we could setup a github action where this provenance file is created such that all information is available.

ro-crate-metadata.json

[bentsherman]

The first iteration of RO-crate support in 1.4.0 focused on compliance with the RO-crate validator, and we tried to support entities that made sense without getting too complicated.

There may be more entities that could be added. For example, I don't think we currently save anything about conda packages or containers. We do extract some tool information from the nf-core meta.yml if it is present.

I don't have much time to look into this, but @simleo might be able to make some recommendations on what we could add to nf-prov based on your use cases.

[simleo]

The first iteration of RO-crate support in 1.4.0 focused on compliance with the RO-crate validator, and we tried to support entities that made sense without getting too complicated.

Exactly: support for tool wrapper dependencies and tool runtime environment is optional in Provenance Run Crate. They would be nice to have in some future version of the plugin, but the most important thing is having the required bits (by the way, WRROC ouptut is broken in the current development version). Action startTime and endTime are also not strictly required, but they are reported for the whole workflow (cq6); a nice addition to future versions would be to also report them for each step execution. The inputs and outputs of the workflow are reported by nf-prov 1.4.0. I ran the cq8 query on the attached metadata and got this:

INPUTS
======
file:///home/junger/github/NFDI4IngModelValidationPlatform/benchmarks/linear-elastic-plate-with-hole/create_mesh.py
file:///home/junger/github/NFDI4IngModelValidationPlatform/benchmarks/linear-elastic-plate-with-hole/parameters_geo2fun2.json
file:///home/junger/github/NFDI4IngModelValidationPlatform/benchmarks/linear-elastic-plate-with-hole/parameters_0125.json
file:///home/junger/github/NFDI4IngModelValidationPlatform/benchmarks/linear-elastic-plate-with-hole/parameters_00625.json
file:///home/junger/github/NFDI4IngModelValidationPlatform/benchmarks/linear-elastic-plate-with-hole/parameters_05.json
file:///home/junger/github/NFDI4IngModelValidationPlatform/benchmarks/linear-elastic-plate-with-hole/parameters_1.json
file:///home/junger/github/NFDI4IngModelValidationPlatform/benchmarks/linear-elastic-plate-with-hole/parameters_025.json
file:///home/junger/github/NFDI4IngModelValidationPlatform/benchmarks/linear-elastic-plate-with-hole/parameters_geo1fun2.json
file:///home/junger/github/NFDI4IngModelValidationPlatform/benchmarks/linear-elastic-plate-with-hole/parameters_geo2fun2fine.json
file:///home/junger/github/NFDI4IngModelValidationPlatform/benchmarks/linear-elastic-plate-with-hole/parameters_003125.json
file:///home/junger/github/NFDI4IngModelValidationPlatform/benchmarks/linear-elastic-plate-with-hole/fenics/run_fenics_simulation.py
OUTPUTS
=======

so there was no output (action result) in this specific run (I'm not sure whether this is consistent with how the workflow is defined or due to a bug in the plugin).

[joergfunger]

Thanks for the updates, I guess some of the issues are related to the problems mentioned in #49 (cq8). What is interesting to us is the documentation of the compute environment (in our case this is usually conda) that might include hundreds of packages (or we just document the ones with their version that are specified in the environment, all others would be just dependencies). But the challenge is how to uniquely identify and reference those packages. As far as I understand the approach in swhid, there is no direct link between the swhid and conda (the optimal solution would be something like conda -list -swhid) where the information about the versions is directly extracted. Any other idea how the swhid could be extracted automatically?
And can we make a distinction between actual parameters and inputs? In our workflow, we have defined the python scripts that are executed as inputs, because we would like those to be documented, but the scripts that are executed in a process are IMO rather a part of the SoftwareApplication than a real input, so essentially the SoftwareApplication is comprised of the conda environment, the script and the application (python) that runs the script.

[simleo]

Note that Provenance Run Crate has two sections that are relevant for this, "Tool wrapper dependencies" and "Tool runtime environment". The former lifts the details of each dependency to the RO-Crate metadata level, so it's more useful for running queries; the latter brings in the original environment file as-is, so it's more useful for reusability / reproducibility. Both can be used at the same time, it depends on what information is available from Nextflow.

@jmfernandez might have some advice since he's got experience with both WRROC and Nextflow.

[jmfernandez]

Hi, currently, WfExS-backend is using more the approach of "tool wrapper dependencies" for the workflow and its used containers to describe both the workflow engine and the container technology usable with each container.

As WfExS-backend is not able yet to automatically describe each one of the run steps, it is not following the "tool runtime environment" approach.

As each Nextflow process has its own working directory, script, etc... , and associated docker, singularity, conda, etc... environments, each Nextflow process could be properly described using "tool runtime environment" approach. Maybe not all the details, but most of them.