From 65eeb589a63baf51703c992a0d9dc8c3c080024e Mon Sep 17 00:00:00 2001 From: Thomas White Date: Tue, 16 Apr 2024 17:11:35 +0200 Subject: This is CrystFEL 0.11.0 --- CONTRIBUTING.md | 4 +- ChangeLog | 4 +- meson.build | 4 +- relnotes-0.10.2.md | 161 ------------------------------------------------- relnotes-0.11.0.md | 172 +++++++++++++++++++++++++++++++++++++++++++++++++++++ 5 files changed, 178 insertions(+), 167 deletions(-) delete mode 100644 relnotes-0.10.2.md create mode 100644 relnotes-0.11.0.md diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 50cab3b5..349a9e50 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -13,8 +13,8 @@ The easiest way to ensure the future of CrystFEL is to use it and cite it in you You can also cite CrystFEL directly using one of the following DOIs: -* All versions: [10.5281/zenodo.8183384](https://zenodo.org/record/8183384). -* Version 0.10.2: [10.5281/zenodo.8183385](https://zenodo.org/record/8183385). +* All versions: [10.5281/zenodo.8183384](https://doi.org/10.5281/zenodo.8183384). +* This version (0.11.0): [10.5281/zenodo.10984700](https://10.5281/zenodo.10984700). Getting started diff --git a/ChangeLog b/ChangeLog index 7ea1073b..5d24622f 100644 --- a/ChangeLog +++ b/ChangeLog @@ -1,5 +1,5 @@ -Changes in this development version ------------------------------------ +CrystFEL version 0.11.0, 17 April 2024 +-------------------------------------- - New detector geometry refinement system, using Millepede-II - New Homebrew formula and better build/test support for Mac OS diff --git a/meson.build b/meson.build index 1d6a0130..4717bb2a 100644 --- a/meson.build +++ b/meson.build @@ -1,11 +1,11 @@ # Meson file for CrystFEL project('crystfel', 'c', - version: '0.10.2', + version: '0.11.0', license: 'GPL3+', meson_version: '>=0.55.0', default_options: ['buildtype=debugoptimized']) -libcrystfel_api_version = 16 +libcrystfel_api_version = 17 crystfel_rpath = '$ORIGIN/../lib64/:$ORIGIN/../lib:$ORIGIN/../../asapo-libraries/lib:$ORIGIN/../../asapo-libraries/lib64' add_project_arguments('-DHAVE_CONFIG_H', language: 'c') diff --git a/relnotes-0.10.2.md b/relnotes-0.10.2.md deleted file mode 100644 index 00bbad03..00000000 --- a/relnotes-0.10.2.md +++ /dev/null @@ -1,161 +0,0 @@ -Release notes for CrystFEL version 0.10.2 -========================================= - -Copyright © 2012-2023 Deutsches Elektronen-Synchrotron DESY, - a research centre of the Helmholtz Association. - -See AUTHORS as well as the individual source code files for full contributor details. - -CrystFEL is free software: you can redistribute it and/or modify it under the -terms of the GNU General Public License as published by the Free Software -Foundation, either version 3 of the License, or (at your option) any later -version. - -CrystFEL is distributed in the hope that it will be useful, but WITHOUT ANY -WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A -PARTICULAR PURPOSE. See the GNU General Public License for more details. - -You should have received a copy of the GNU General Public License along with -CrystFEL. If not, see . - - -Overview --------- - -This release is primarily a bug-fixing update to CrystFEL, and there have been -improvements around the entire suite. The most important changes are detailed -below. See the ChangeLog or the Git history for a comprehensive list of all -changes. - - -Graphical user interface (GUI) ------------------------------- - -This version adds a colour scale widget to the GUI, which can be used to -manually adjust the colours used to display the image. The colour scale is -found directly to the right of the image. Click and drag to move -the histogram of image values relative to the colour scale, and use the scroll -wheel to compress and expand the range. - -Resolution rings can now be added to the image. Enable them in the "View" menu. -There are also menu items for running `detector-shift` and `peakogram`, found -under the "Tools" menu. - -The figure of merit calculation tool has been improved, and now displays the -results as a graph. The graph values can be exported as a CSV file for further -processing. - -This release also adds specially-designed icons to the GUI, which fixes -problems with "broken" icons on some systems. - - -Performance ------------ - -Speed has been improved in many areas of CrystFEL. The profiling option -(`--profile`) of `indexamajig` has been re-implemented and can help you to find -bottlenecks in processing on your data. In addition, some new options were -added to `indexamajig`: - -New option `--peakfinder8-fast` tells peakfinder8 to calculate some values in -advance, approximately halving the time taken by the peak search. - -Another new option `--asdf-fast` slightly alters some thresholds within asdf, -resulting in a large speedup. Since this changes the behaviour from before, we -decided to make it optional. - -Finally, `--cell-parameters-only` completely circumvents the spot prediction and -integration calculations, which can make a very large speedup as long as you're -not interested in spot positions. Note that this isn't the same as the -existing option ``--integration=none``, where the spot positions are predicted -but the intensity integration is skipped. For very large unit cells, which are -often produced by unconstrained indexing (e.g. for online monitoring), the spot -prediction can take signficant time. - -For advice about making CrystFEL faster, see this page: - - - -Documentation improvements --------------------------- - -Two long-form documentation articles have been added, on top of the other two -mentioned in other sections of this page. - -A completely new tutorial has been added, describing how to process data using -the graphical user interface: - - -The following new article has been added, to help with the common question of -choosing the correct point group for merging data: - - - -Streaming interfaces --------------------- - -ZeroMQ and ASAP::O interfaces have been added, to allow data to be streamed -directly to CrystFEL without any intermediate file storage. For some -information on how to get started, read the following document: - - - -Refuses to overwrite stream ---------------------------- - -Starting from this version, `indexamajig` and other CrystFEL tools will refuse to -create a new stream if a file already exists with the same name. This is meant -to avoid the possibility of losing valuable results by (e.g.) re-running a -processing script. This becomes particularly important in a streaming data -situation, where the original data might no longer be available. If you really -want to overwrite the results, you'll need to move or delete the old file -yourself. - - -Installation ------------- - -CrystFEL has become easier to install because of several developments. First, -the Slurm API headers are no longer needed for the GUI to submit Slurm jobs. -In this version, it uses the Slurm commands directly -(`sbatch/scontrol/scancel`). - -For environments based totally on CBFs or other types of files, it's now -possible to compile CrystFEL without any reference to the HDF5 library, -avoiding a lot of possible complications. - -There is a new script (`scripts/install-indexers`) which helps with installing -Mosflm, DirAx and XDS. After downloading CrystFEL, run the script with -`--help` to get started. - -Finally, Docker images are now available. These can be used with various -container tools (not just Docker). For example, using Apptainer (the new name -for Singularity): -``` -$ apptainer pull docker://gitlab.desy.de:5555/thomas.white/crystfel/crystfel:latest -$ apptainer run -B /path/to/data crystfel_latest.sif -``` - -More details can be found on the installation page: - - - -API changes ------------ - -Added routines: -* `data_template_slabby_file_to_panel_coords` -* `data_template_get_2d_detgeom_if_possible` -* `data_template_get_clen_if_possible` -* `image_create_dp_bad_sat` -* `profile_{init,start,end,print_and_reset}` - -Removed routines: -* `image_set_zero_mask` - -Changed routine prototypes: -* `default_method_options` - addition of `asdf_opts` -* `peakfinder8` - addition of `fast_mode` and `private_data` - -Changed enumerations: -* `DataSourceType` - addition of `DATA_SOURCE_TYPE_SEEDEE` diff --git a/relnotes-0.11.0.md b/relnotes-0.11.0.md new file mode 100644 index 00000000..2691695b --- /dev/null +++ b/relnotes-0.11.0.md @@ -0,0 +1,172 @@ +Release notes for CrystFEL version 0.11.0 +========================================= + +Copyright © 2012-2024 Deutsches Elektronen-Synchrotron DESY, + a research centre of the Helmholtz Association. + +See AUTHORS as well as the individual source code files for full contributor details. + +CrystFEL is free software: you can redistribute it and/or modify it under the +terms of the GNU General Public License as published by the Free Software +Foundation, either version 3 of the License, or (at your option) any later +version. + +CrystFEL is distributed in the hope that it will be useful, but WITHOUT ANY +WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A +PARTICULAR PURPOSE. See the GNU General Public License for more details. + +You should have received a copy of the GNU General Public License along with +CrystFEL. If not, see . + + +Overview +-------- + +This CrystFEL release adds a totally new system for detector geometry +refinement, as well as better support for compiling on Mac OS. The old CMake +build system has been removed in favour of Meson. The simulation tools +`pattern_sim` and `partial_sim`, which were not well maintained nor widely used, +have been removed. + +These changes are detailed below. In addition, there were many smaller fixes +and improvements. See the ChangeLog or the Git history for a comprehensive +list of all changes. + + +New system for detector geometry refinement +------------------------------------------- + +The new detector geometry system is based on the Millepede-II program, the same +software used for high energy physics experiments including the CMS. It +allows us to do a full refinement including all of the inter-dependencies +between crystal orientations, cell parameters and the detector geometry. +Previously, the refined updates to the detector positions would be biased +because they were based on biased crystal parameters, which were in turn biased +because of the incorrect detector geometry. To break this circular dependency, +we have to refine the individual parameters for each crystal at the same time +as we refine the geometry. Normally, doing this would result in an enormous +and computationally prohibitive least squares calculation, even with +specialised sparse matrix solvers. However, the Millepede algorithm reduces +the calculation to one with a sized based only on the number of geometrical +parameters for the detector. + +To get started with the new geometry refinement, read the manual for the new +`align_detector` program by running `man align_detector` or visiting +. +An update of the geometry file will be necessary to add information about the +hierarchy of detector panels, but example files are available. The refinement +can also be performed using the CrystFEL GUI. + +A side-effect of this change is that the beam center is no longer refined +after indexing for each pattern individually. This should make the prediction +refinement more stable. + +Future CrystFEL versions will look at improving the stability and precision of +this method, in particular for three-dimensional and fine-grained refinement +tasks. + + +Homebrew formula and improved Mac OS support +-------------------------------------------- + +This release adds a Homebrew formula for CrystFEL. If you're running on a Mac, +you can now install CrystFEL with a single `brew install` command. See +`INSTALL.md` for more details. Eventually we hope to submit this formula to +the main Homebrew repository to make it even easier to install. + +In addition, we are now regularly testing CrystFEL on Mac OS as part of our +continuous integration pipeline. + + +Removal of simulation tools and CMake build system +-------------------------------------------------- + +Due mostly to a insufficient development resources, the simulation tools +`pattern_sim` and `partial_sim` have been removed from the CrystFEL suite +starting from this version. For more detailed discussion of the rationale for +this, see https://gitlab.desy.de/thomas.white/crystfel/-/issues/81 + +If you need quantitative image simulations, there are several better options +including [nanoBragg]. For +relative-scale geometry-only simulations of the kind done by `partial_sim`, +a better option is expected in a future CrystFEL version - watch this space! + +The CMake-based build system, which was barely maintained and lacked many +features compared to the Meson-based system, has now been removed altogether. + + +API changes +----------- + +Removed routines: +* `data_template_get_rigid_groups` +* `r_gradient` +* `x_gradient` +* `y_gradient` +* `image_hdf5_write` + +Added routines: +* `crystfel_mille_new` +* `crystfel_mille_free` +* `crystfel_mille_delete_last_record` +* `crystfel_mille_write_record` +* `mille_label` +* `mille_unlabel` +* `write_mille` +* `data_template_show_hierarchy` +* `data_template_translate_group_px` +* `data_template_translate_group_m` +* `data_template_rotate_group` +* `data_template_write_to_fh` +* `data_template_write_to_file` +* `data_template_group_info` +* `detgeom_find_panel` +* `detgeom_show_hierarchy` +* `detgeom_translate_detector_m` +* `detgeom_group_center` +* `make_panel_minvs` +* `is_cbf_file` +* `is_cbfgz_file` +* `is_hdf5_file` +* `image_data_arrays_new` +* `image_data_arrays_free` +* `image_create_dp_bad` +* `image_set_zero_data` +* `index_pattern_4` +* `r_dev` +* `fs_dev` +* `ss_dev` +* `r_gradient` +* `fs_ss_gradient` +* `read_reflections_3` +* `stream_write_data_template` +* `show_vector` +* `matrix_mult2` +* `matrix_mult3` +* `matrix_invert` +* `sq` +* `rotate2d` +* `is_dir` + +Added type definitions: +* `Mille` +* `struct dg_group_info` +* `struct detgeom_panel_group` +* `ImageDataArrays` +* `struct peak_params` +* `struct reflpeak` + +Removed type definitions: +* `struct rigid_group` +* `struct rg_collection` + +Changed structure definitions: +* `detgeom_panel` - addition of `group` +* `detgeom` - addition of `top_group` +* `image` - addition of `ida` +* `enum gparam` - changed values, moved from geometry.h to predict-refine.h + +Changed routine prototypes: +* `image_read` - addition of `ida` +* `image_read_data_block` - addition of `ida` +* `refine_prediction` - addition of `mille` and `max_mille_level` -- cgit v1.2.3