aboutsummaryrefslogtreecommitdiff
path: root/src/gui_backend_slurm.c
AgeCommit message (Collapse)Author
2022-03-18GUI: Use sbatch/scancel/scontrol instead of Slurm APIThomas White
This commit strips out all references to the Slurm API, instead making subprocess calls to sbatch and scontrol. Attempting to use the Slurm API seems to have been a mis-step. First, it seems that nowhere has the Slurm headers pre-installed. Literally none of the facilities where there are known deployments of CrystFEL have them. And in a significant fraction of cases, getting them installed is difficult, slow or impossible. In addition, the API doesn't seem to work in all cases, so we already shell out to 'scancel' to abort jobs - see d76fc3495. There are some tricky implications for submitting Slurm jobs from a container via the API. The Slurm REST API offers a solution, but is very new and not widely available. Calls to the Slurm executables are much easier to 'tunnel' out of a container. This isn't a great solution. It's a net increase of only about 40 lines of source code, but it incurs some unpleasant string handling and will probably be less reliable overall. It completely relies on Slurm's not being internationalised. If Slurm's messages start getting translated, we will be in trouble.
2022-01-12SLURM: Fix a silly mistake counting the environment entriesThomas White
2022-01-12SLURM: Get rid of path_addThomas White
This is no longer needed after 48c02e71e5bf. Simply set the correct PATH before starting the GUI, and it will be propagated to the SLURM jobs. Astute readers will notice that the feature has been silently broken since last March (65a1afdce2da5cd), anyway. Whoops.
2022-01-12SLURM: Copy environment from parent processThomas White
Unfortunately, several environments rely on "magic" environment variables to make things work. These need to be propagated. The path to the GUI will no longer be added to the PATH. This was intended to help when the indexing executables (mosflm, dirax etc) were found alongside the CrystFEL executables. However, it's highly likely that the path to the CrystFEL executables will be in PATH anyway. This way, running the SLURM jobs in the same environment as the GUI itself, also seems more compliant with the "principle of least surprise".
2021-10-20GUI: Track the indexing result from which each merge result cameThomas White
2021-09-17SLURM: Add field for time limitThomas White
Closes: https://gitlab.desy.de/thomas.white/crystfel/-/issues/49
2021-09-07GUI SLURM backend: Bail out if environment can't be allocatedThomas White
2021-07-23GUI: Write partialator log files ("pr-logs") in working folderThomas White
2021-07-23SLURM: Fix printf typeThomas White
2021-07-22Clean up shadowed variablesThomas White
2021-07-21Add missing cleanup on error pathsThomas White
2021-07-15GUI: Run all jobs with same CWD as GUI itselfThomas White
This requires changing all the paths to relative ones. Previously, the jobs ran with their CWDs set to the corresponding "result" folder. The trouble is, this breaks everything that uses a relative location: image filenames, mask/satmap files referenced by the geometry, etc. Note that this change also means that the temporary folders ("indexamajig.XXX") are now produced in the top-level working directory. Eventually, a GUI option should be added to allow these folders to be put somewhere better (i.e. somewhere fast). Closes: https://gitlab.desy.de/thomas.white/crystfel/-/issues/44
2021-07-12GUI: Add --wavelength-estimate and --camera-length-estimateThomas White
Only when using PinkIndexer. Closes: https://gitlab.desy.de/thomas.white/crystfel/-/issues/25
2021-07-06SLURM: Show the real error message if job can't be submittedThomas White
2021-06-17GUI: Add a tooltipThomas White
2021-05-28GUI: Store relative paths of resultsThomas White
This allows project folders to be relocated more easily. At the same time, I fixed the weird ownership semantics of add_indexing_result and add_merge_result. Fixes: https://gitlab.desy.de/thomas.white/crystfel/-/issues/31
2021-04-01GUI: Clean up old tasks when infobar is removedThomas White
2021-03-05Mass update of copyright dates in source code commentsThomas White
2021-03-04SLURM: Better handling of job cancellation/crashingThomas White
2021-03-04SLURM: Cancel job by calling scancelThomas White
The API for cancelling a job seems to be Complicated. The source code for scancel is nearly a thousand lines long! The API also seems to have just changed. So, let's just run scancel and be done with it.
2021-03-04SLURM: Add one to serial offsetsThomas White
They should start from 1, not 0.
2021-03-04SLURM: Set serial offset for indexamajig jobsThomas White
2021-03-04GUI: When starting indexing, prefix relative filenames with ../Thomas White
2021-03-04SLURM: Clean up some exit pathsThomas White
2021-03-04start_slurm_job: Save the real stderr filenameThomas White
2021-03-04SLURM: Implement indexing progress monitoring for job arraysThomas White
2021-03-03GUI: Submit indexing jobs as job arrays, not individual jobsThomas White
2021-03-03SLURM: Remove unused stuffThomas White
2021-03-03SLURM: Add access to account and feature constraintsThomas White
2021-03-03SLURM: Factorise common options even moreThomas White
2021-03-03SLURM: Factorise widget creation for common backend optionsThomas White
2021-02-22SLURM: Set 'running' properlyThomas White
2021-02-22SLURM: Use a faster counting method when the number of sub-jobs is largeThomas White
2021-02-22Read job progress from written log files, even for local BEThomas White
This simplifies the backends somewhat, and makes them look more similar - e.g. there is now only one routine to find out how far along a merging job is. It has the added bonus of adding a log file for local jobs, which we would've had to add soon anyway.
2021-02-18SLURM: Show submitted job ID for non-indexing tasksThomas White
2021-02-18SLURM: Create an ambigator script, not a merge oneThomas White
Whoops.
2021-02-18SLURM: Submit merging and ambigator jobs via scriptThomas White
2021-02-18Eliminate duplicate make_job_folder/make_workdirThomas White
2021-02-17GUI: Plumbing, ready to run ambigator jobThomas White
2020-12-11result -> indexing_resultThomas White
Because shortly we will have merging_result as well
2020-12-04SLURM: Cancel merging jobs properlyThomas White
2020-12-03SLURM BE: Submit merge jobThomas White
2020-11-30Pass result to backend for mergingThomas White
2020-11-30Move get_crystfel_exe and get_crystfel_path_str to crystfel_gui.cThomas White
2020-10-29Merge backend plumbingThomas White
2020-09-24SLURM: Avoid crash when batch system parameters are not setThomas White
2020-09-08Just use project structure in backendsThomas White
We have to pass the crystfelproject anyway, to add the new results to the list. It seems messy to pass everything else separately, when at this moment the values must be exactly what is in the project structure anyway.
2020-09-08Local/SLURM BE: Add full path to stream in resultsThomas White
2020-09-04Add results from new indexing jobThomas White
2020-09-02Fix job tracking logicThomas White
generated by cgit v1.2.3 (git 2.25.1) at 2024-11-17 11:53:53 +0000