Age | Commit message (Collapse) | Author |
|
|
|
|
|
|
|
This commit strips out all references to the Slurm API, instead making
subprocess calls to sbatch and scontrol.
Attempting to use the Slurm API seems to have been a mis-step. First,
it seems that nowhere has the Slurm headers pre-installed. Literally
none of the facilities where there are known deployments of CrystFEL
have them. And in a significant fraction of cases, getting them
installed is difficult, slow or impossible.
In addition, the API doesn't seem to work in all cases, so we already
shell out to 'scancel' to abort jobs - see d76fc3495.
There are some tricky implications for submitting Slurm jobs from a
container via the API. The Slurm REST API offers a solution, but is
very new and not widely available. Calls to the Slurm executables are
much easier to 'tunnel' out of a container.
This isn't a great solution. It's a net increase of only about 40 lines
of source code, but it incurs some unpleasant string handling and will
probably be less reliable overall. It completely relies on Slurm's not
being internationalised. If Slurm's messages start getting translated,
we will be in trouble.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
This adds a new routine, data_template_slabby_file_to_panel_coords, to
be used (only!) in places where external forces require assumptions of
slabbiness: pixel maps and MsgPack/HDF5 peak lists (including
CXI-style).
This also fixes the prototype of data_template_file_to_panel_coords to
make the panel number strictly an input parameter. This was an
oversight in the implementation of DataTemplate, and caused problems
when reading non-slabby streams.
Fixes: https://gitlab.desy.de/thomas.white/crystfel/-/issues/66
|
|
|
|
CrystFEL hasn't written any version other than 2.3 since before 2015.
Supporting the old versions is getting difficult because of slabbiness
assumptions, so it's time to get rid of it.
|
|
|
|
|
|
This takes a non-negligible amount of time (40 ms per frame for 16M), so
should be skipped if not needed.
|
|
|
|
|
|
|
|
|
|
|
|
This doesn't do anything. The very next lines override all the
settings.
|
|
Fixes: https://gitlab.desy.de/thomas.white/crystfel/-/issues/64
|
|
If there are lots of error messages (e.g. a missing mask file), looking
at the last 4k of the log file is not always enough. Now it will go
back to 16k, in steps of 4k. It will stop early if it finds the start
of the file, and it will complain if it still doesn't find what it's
looking for.
|
|
Fixes: https://gitlab.desy.de/thomas.white/crystfel/-/issues/61
|
|
This checks that indexamajig correctly recognises when no files could be
processed.
|
|
|
|
No other code path under image_expand_frames() actually opens the file,
because HDF5s are currently the only format supporting multiple frames.
The aim of the existence check is to give a better error message
(compared to the HDF5 backtrace splurge), but it doesn't need to be done
so early. In fact, checking so early had the side-effect of breaking
indexamajig --wait-for-file.
Fixes: https://gitlab.desy.de/thomas.white/crystfel/-/issues/65
|
|
|
|
This tests that indexamajig --wait-for-file works
|
|
|
|
|
|
|
|
Fixes: https://gitlab.desy.de/thomas.white/crystfel/-/issues/59
|
|
Fixes: https://gitlab.desy.de/thomas.white/crystfel/-/issues/58
|
|
|
|
correct a simple typo
|
|
|
|
|
|
This makes it explicit that the Meson and Ninja steps should be run from
the CrystFEL folder, not (e.g.) the Meson folder.
|
|
|
|
This is no longer needed after 48c02e71e5bf. Simply set the correct
PATH before starting the GUI, and it will be propagated to the SLURM
jobs.
Astute readers will notice that the feature has been silently broken
since last March (65a1afdce2da5cd), anyway. Whoops.
|
|
Fixes: https://gitlab.desy.de/thomas.white/crystfel/-/issues/56
|
|
Unfortunately, several environments rely on "magic" environment
variables to make things work. These need to be propagated.
The path to the GUI will no longer be added to the PATH. This was
intended to help when the indexing executables (mosflm, dirax etc) were
found alongside the CrystFEL executables. However, it's highly likely
that the path to the CrystFEL executables will be in PATH anyway.
This way, running the SLURM jobs in the same environment as the GUI
itself, also seems more compliant with the "principle of least
surprise".
|
|
|
|
|
|
When running online, there's no input file handle to close.
|
|
This avoids the process getting killed while waiting e.g. between runs
in an online context.
|
|
|
|
|