Repository navigation
Conversation
| self.default_workdir = Path(default_workdir).resolve() if default_workdir is not None else default_workdir | ||
| self.default_env = default_env or {} | ||
| self.name = name | ||
| self.stream = Path(stream).resolve() if stream is not None else None |
There was a problem hiding this comment.
Should the paths be resolved on the client? It could be relative to the worker directory, maybe?
There was a problem hiding this comment.
Good point. Then we should probably not resolve also default_workdir
There was a problem hiding this comment.
Hmm, maybe. There is a difference, because the workdir is applied before the worker starts, while the stream is then evaluated relative to the workdir, I assume? But yeah, even for the workdir, the relative path between the client and the server might be different. I think that normalizing the workdir based on the client is kinda intuitive though.
There was a problem hiding this comment.
As I see in the code, the stream path is resolved against woker cwd (that is probably accidental rather than designed).
In the case of share file system, you want stream path pointing to some globally accessed dir so you do not have to manually copy files for postprocessing. Contrary workdir should be on a local filesystem or even ramdisk.
There was a problem hiding this comment.
After some time thinking about it.
I am leaning to resolving stream path as a kind of global path different from worker dir.
Maybe we should do the resolve even after --stream (it is not done today).
There was a problem hiding this comment.
We resolve various paths kinda ad-hoc. Maybe we could forbid relative paths, and use % placeholders for specifying explicit relative path roots :)
There was a problem hiding this comment.
This is good idea as the current state is slightly chaotic. I will look at it if there is some potential problem with this change.
There was a problem hiding this comment.
What now? I have forbid relative paths (if they do no start with an abspath producing % subsitution) from Python, TOML and CLI if --array is used. For better basic compatibility, plain (no array) CLI submit still accepts relative paths and they are resolved as usual.
spirali
changed the title
This branch has not been deployed
2 participants
Fixes #668