Skip to content

TES executor sends invalid/mismatched output fields in requests #11

@JaeAeich

Description

@JaeAeich

Description

While running workflows with TES, I noticed several mismatches between the requests sent by Nextflow to the TES server and the expected data format.

Examples of issues

  1. Incorrect file type handling
    In some cases, data was actually a file, but Nextflow sent it as a directory:

    {
      "name": null,
      "description": null,
      "url": "s3://poiesis/ribap/command/workdir/a7/409f72674fb381d844e1e17d085b31/95",
      "path": "/work/95",
      "path_prefix": null,
      "type": "FILE"
    }
  2. Regex mismatch
    The regex used for patterns did not align correctly, causing unexpected behavior.

  3. Nullable fields
    TES does not define request fields as nullable, but Nextflow includes null values.
    → Expected: optional fields should be omitted instead of sent as null.

Workaround

To get my use case working, I implemented workarounds in [Poiesis](https://github.com/jaeaeich/poiesis). While this helps locally, it would be much better if Nextflow handled these cases directly.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions