This is CrystFEL 0.9.00.9.0

6 files changed, 252 insertions, 230 deletions

diff --git a/CMakeLists.txt b/CMakeLists.txt
index 08e3905b..ec944a5a 100644
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -1,7 +1,7 @@
 cmake_minimum_required(VERSION 3.6)
-set(CRYSTFEL_SHORT_VERSION 0.8.0)
+set(CRYSTFEL_SHORT_VERSION 0.9.0)
 set(CRYSTFEL_VERSION ${CRYSTFEL_SHORT_VERSION})
-set(CRYSTFEL_API_VERSION 11)
+set(CRYSTFEL_API_VERSION 12)
 project(crystfel VERSION ${CRYSTFEL_VERSION} LANGUAGES C)
 
 cmake_policy(SET CMP0074 NEW)
diff --git a/ChangeLog b/ChangeLog
index 2b7f2dd1..f67ecec8 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,5 +1,5 @@
-Changes in this development version
------------------------------------
+CrystFEL version 0.9.0, 6th February 2020
+-----------------------------------------
 
 - Introduction of cell_tool
 - Addition of PinkIndexer interface
diff --git a/announcement-0.8.0 b/announcement-0.8.0
deleted file mode 100644
index 2666da94..00000000
--- a/announcement-0.8.0
+++ /dev/null
@@ -1,32 +0,0 @@
-Dear CrystFEL users and interested crystallographers,
-
-An early Christmas present for you all, as CrystFEL version 0.8.0 has been
-released.
-
-This new version adds a powerful new indexing algorithm, more stable B-factors
-when merging and rejection of rogue patterns via "delta CChalf".  Besides those,
-there are several other exciting features such as an option to wait for files
-to appear on disk (useful for certain automated processing environments).
-In addition, the pixelmap creation utility (make_pixelmap) has become a core
-CrystFEL program with this release.
-
-Note that the build system has changed from GNU autotools ("./configure") to
-CMake.  The README file has instructions for how to compile and install.
-
-See the release notes for more discussion of these improvements, and the
-ChangeLog for full details of the other improvements.
-
-Thanks for all of your past and future feedback and contributions, and
-your continued use of CrystFEL.  As ever, please contact me directly if
-you would prefer to be unsubscribed from this mailing list.
-
-CrystFEL website:
-https://www.desy.de/~twhite/crystfel
-
-Release notes for version 0.8.0:
-https://www.desy.de/~twhite/crystfel/relnotes-0.8.0
-
-CrystFEL tutorial:
-https://www.desy.de/~twhite/crystfel/tutorial
-
-Tom
diff --git a/announcement-0.9.0 b/announcement-0.9.0
new file mode 100644
index 00000000..7915cece
--- /dev/null
+++ b/announcement-0.9.0
@@ -0,0 +1,34 @@
+Dear CrystFEL users and interested crystallographers,
+
+CrystFEL version 0.9.0 has been released!
+
+This release adds a new indexing method ("PinkIndexer") and improvements to
+reflection prediction which will enable data processing for wide bandwidth and
+electron diffraction experiments.
+
+A new core program has been added, "cell_tool", which can perform several
+useful tasks on unit cells such as transformation, finding indexing ambiguities
+and calculating power ring positions.
+
+There have been many other improvements across the whole suite.  Notably,
+CBFlib is no longer required by CrystFEL, manual resolution limits are no
+longer needed by ambigator, and "--no-cell-combinations" is no longer needed by
+indexamajig.
+
+See the release notes for more discussion of these improvements, and the
+ChangeLog for full details of the other improvements.
+
+Thanks for all of your past and future feedback and contributions, and
+your continued use of CrystFEL.  As ever, please contact me directly if
+you would prefer to be unsubscribed from this mailing list.
+
+CrystFEL website:
+https://www.desy.de/~twhite/crystfel
+
+Release notes for version 0.9.0:
+https://www.desy.de/~twhite/crystfel/relnotes-0.9.0
+
+CrystFEL tutorial:
+https://www.desy.de/~twhite/crystfel/tutorial
+
+Tom
diff --git a/relnotes-0.8.0 b/relnotes-0.8.0
deleted file mode 100644
index ed05d678..00000000
--- a/relnotes-0.8.0
+++ /dev/null
@@ -1,194 +0,0 @@
-CrystFEL - Data processing for serial crystallography
------------------------------------------------------
-
-Release notes for version 0.8.0
-
-Copyright © 2012-2018 Deutsches Elektronen-Synchrotron DESY,
-                      a research centre of the Helmholtz Association.
-
-Authors:
-  Thomas White <taw@physics.org>
-  Richard Kirian <rkirian@asu.edu>
-  Kenneth Beyerlein <kenneth.beyerlein@desy.de>
-  Andrew Aquila <andrew.aquila@cfel.de>
-  Andrew Martin <andrew.martin@desy.de>
-  Lorenzo Galli <lorenzo.galli@desy.de>
-  Chun Hong Yoon <chun.hong.yoon@desy.de>
-  Karol Nass <karol.nass@desy.de>
-  Nadia Zatsepin <nadia.zatsepin@asu.edu>
-  Anton Barty <anton.barty@desy.de>
-  Cornelius Gati <cornelius.gati@desy.de>
-  Fedor Chervinskii <fedor.chervinskii@gmail.com>
-  Alexandra Tolstikova <alexandra.tolstikova@desy.de>
-  Wolfgang Brehm <wolfgang.brehm@gmail.com>
-  Valerio Mariani <valerio.mariani@desy.de>
-  Parker de Waal <Parker.deWaal@vai.org>
-  Takanori Nakane <nakane.t@gmail.com>
-  Keitaro Yamashita <k.yamashita@spring8.or.jp>
-  Oleksandr Yefanov <oleksandr.yefanov@cfel.de>
-  Steve Aplin <steve.aplin@desy.de>
-  Helen Ginn <helen@strubi.ox.ac.uk>
-  Thomas Grant <tgrant@hwi.buffalo.edu>
-  Mamoru Suzuki <mamoru.suzuki@protein.osaka-u.ac.jp>
-  Nicolas Riebesel <nicolas.riebesel@tuhh.de>
-  Yaroslav Gevorkov <yaroslav.gevorkov@desy.de>
-  Omri Mor <omor1@asu.edu>
-
-CrystFEL is free software: you can redistribute it and/or modify it under the
-terms of the GNU General Public License as published by the Free Software
-Foundation, either version 3 of the License, or (at your option) any later
-version.
-
-CrystFEL is distributed in the hope that it will be useful, but WITHOUT ANY
-WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A
-PARTICULAR PURPOSE.  See the GNU General Public License for more details.
-
-You should have received a copy of the GNU General Public License along with
-CrystFEL.  If not, see <http://www.gnu.org/licenses/>.
-
-
-Overview
---------
-
-The most important new features in this version of CrystFEL are:
-
-- Switch of build system to CMake from autotools
-
-- Support for peakfinder9 and XGANDALF
-
-- More stable B factors during merging
-
-- Rejection of patterns during merging according to deltaCCHalf
-
-- Introduction of make_pixelmap as a core CrystFEL program
-
-- indexamajig --wait-for-file
-
-- Support for gzipped CBF files
-
-These changes have sections below.  In addition, there were many bug fixes and
-other improvements.  See the "ChangeLog" or the changes page on the CrystFEL
-website for details.
-
-
-Switch of build system to CMake from autotools
-----------------------------------------------
-
-The build system has been changed from GNU autotools (./configure etc) to CMake.
-The README file contains updated instructions for how to compile and install.
-Briefly:
-  $ mkdir build
-  $ cd build
-  $ cmake ..
-  $ make
-  $ sudo make install
-To set the installation path, add the option -DCMAKE_INSTALL_PREFIX to the cmake
-command.  To manually set the paths to external libraries, there are options
-such as -DOpenCL_INCLUDE_DIR and -DCBF_LIBRARIES.
-
-Note that there will be no further development snapshots in .tar.gz format made
-available on the CrystFEL website.  These are now unnecessary, since a version
-from the Git repository is now structurally identical to a packaged release
-version.  If you want to try out the latest features (and possibly the latest
-bugs!), simply follow the instructions on the website "download" page to check
-out a Git version.
-
-
-Support for peakfinder9 and XGANDALF
-------------------------------------
-
-The new peak finding algorithm, peakfinder9, is based on local background
-estimates, similar to peakfinder8 which uses radial background estimates.
-It's implemented in the Fast Diffraction Image Processing library by Yaroslav
-Gevorkov.  You can download it from here:
-https://stash.desy.de/users/gevorkov/repos/fastdiffractionimageprocessing
-
-The new indexing algorithm, "eXtended GrAdient Descent Algorithm for Lattice
-Finding", XGANDALF for short, is also implemented as a separate library written
-by Yaroslav Gevorkov.  Download it here:
-https://stash.desy.de/users/gevorkov/repos/xgandalf/
-
-Once they are installed, the CrystFEL build system should detect these librares
-and enable you to use respectively "indexamajig --peaks=peakfinder9" and
-"indexamajig --indexing=xgandalf".  See the indexamajig manual page for details
-of the advanced options.
-
-
-More stable B factors during merging
-------------------------------------
-
-Previously, it was very easy for the relative Debye-Waller factors for all
-crystals to drift downwards during merging, resulting in reduced intensity at
-high resolution.  In this new version, the merging algorithm has been changed so
-that, while the scaling factors are being determined, logarithms of the
-intensities are merged rather than the intensities themselves.  This makes the
-calculation more stable and avoids the drifting B-factors.  Although the actual
-information content of the merged dataset is not affected very much, this
-usually produces a higher apparent resolution in the electron density maps.
-(Credit: Ken Beyerlein)
-
-
-Rejection of patterns during merging according to deltaCChalf
--------------------------------------------------------------
-
-The rogue dataset rejection algorithm from Assmann, Brehm and Diederichs,
-J. Appl. Cryst. 49 (2016) p1021, has been implemented in partialator.  It is
-enabled by default.  To turn it off, use the option "--no-deltacchalf".
-
-
-Introduction of make_pixelmap as a core CrystFEL program
---------------------------------------------------------
-
-make_pixelmap is now a core CrystFEL program, so you no longer need to install
-a separate program if you need to make pixel maps for Cheetah, OnDA and others.
-The program is largely unchanged from the previous standalone version, except
-that the functionality of make_badmap is now accessed using "make_pixelmap
---badmap".  You can use --good-pixel and --bad-pixel to set the values for good
-and bad pixels respectively.
-
-
-indexamajig --wait-for-file
----------------------------
-
-This option behaves similarly to XDS's "SECONDS" parameter.  When set to a value
-other than zero, this causes indexamajig to wait this number of seconds for each
-image file to appear.  This can be useful when using CrystFEL as part of an
-automated processing pipeline.
-
-
-Support for gzipped CBF files
------------------------------
-
-Finally, CBF files compressed using gzip (.cbf.gz) can now be read directly by
-CrystFEL, without any need to temporarily uncompress them.
-
-
-API changes
------------
-
-The following changes have been made to the libcrystfel API.
-
-New functions:
-	- index_pattern_3()
-	- search_peaks_peakfinder9()
-	- estimate_peak_resolution()
-	- free_contribs()
-	- {set,get}_contributions()
-	- ph_eV_to_k (macro)
-	- strip_extension()
-	- run_xgandalf, xgandalf_{probe,prepare,cleanup}
-
-Removed functions:
-	(none)
-
-Changed function prototypes:
-	- set_dim_structure_entry()
-	- setup_indexing()
-	- refine_radius()
-
-Changed structure/enum definitions:
-	- enum SpectrumType: new value SPECTRUM_FROMFILE
-	- enum imagefile_file: new value IMAGEFILE_CBFGZ
-	- struct image: new members "hit", "n_indexing_tries", "spectrum0" and
-	  peak_resolution
-	- enum IndexingMethod: new value INDEXING_XGANDALF
diff --git a/relnotes-0.9.0 b/relnotes-0.9.0
new file mode 100644
index 00000000..43c5f147
--- /dev/null
+++ b/relnotes-0.9.0
@@ -0,0 +1,214 @@
+CrystFEL - Data processing for serial crystallography
+-----------------------------------------------------
+
+Release notes for version 0.9.0
+
+Copyright © 2012-2020 Deutsches Elektronen-Synchrotron DESY,
+                      a research centre of the Helmholtz Association.
+
+Authors:
+  Thomas White <taw@physics.org>
+  Richard Kirian <rkirian@asu.edu>
+  Kenneth Beyerlein <kenneth.beyerlein@desy.de>
+  Andrew Aquila <andrew.aquila@cfel.de>
+  Andrew Martin <andrew.martin@desy.de>
+  Lorenzo Galli <lorenzo.galli@desy.de>
+  Chun Hong Yoon <chun.hong.yoon@desy.de>
+  Karol Nass <karol.nass@desy.de>
+  Nadia Zatsepin <nadia.zatsepin@asu.edu>
+  Anton Barty <anton.barty@desy.de>
+  Cornelius Gati <cornelius.gati@desy.de>
+  Fedor Chervinskii <fedor.chervinskii@gmail.com>
+  Alexandra Tolstikova <alexandra.tolstikova@desy.de>
+  Wolfgang Brehm <wolfgang.brehm@gmail.com>
+  Valerio Mariani <valerio.mariani@desy.de>
+  Parker de Waal <Parker.deWaal@vai.org>
+  Takanori Nakane <nakane.t@gmail.com>
+  Keitaro Yamashita <k.yamashita@spring8.or.jp>
+  Oleksandr Yefanov <oleksandr.yefanov@cfel.de>
+  Steve Aplin <steve.aplin@desy.de>
+  Helen Ginn <helen@strubi.ox.ac.uk>
+  Thomas Grant <tgrant@hwi.buffalo.edu>
+  Mamoru Suzuki <mamoru.suzuki@protein.osaka-u.ac.jp>
+  Nicolas Riebesel <nicolas.riebesel@tuhh.de>
+  Yaroslav Gevorkov <yaroslav.gevorkov@desy.de>
+  Omri Mor <omor1@asu.edu>
+
+CrystFEL is free software: you can redistribute it and/or modify it under the
+terms of the GNU General Public License as published by the Free Software
+Foundation, either version 3 of the License, or (at your option) any later
+version.
+
+CrystFEL is distributed in the hope that it will be useful, but WITHOUT ANY
+WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A
+PARTICULAR PURPOSE.  See the GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License along with
+CrystFEL.  If not, see <http://www.gnu.org/licenses/>.
+
+
+Overview
+--------
+
+The most important new features in this version of CrystFEL are:
+
+- Features for wide bandwidth and electron diffraction processing
+
+- Introduction of cell_tool
+
+- New unit cell comparison algorithm (eliminating --no-cell-combinations)
+
+- CBF files now handled natively (CBFlib not required)
+
+- Resolution limits no longer needed for ambigator
+
+See below for information about each of these features.
+
+In addition, there were many bug fixes and other improvements.  See the
+"ChangeLog" or the changes page on the CrystFEL website for details.
+
+
+Features for wide bandwidth and electron diffraction processing
+---------------------------------------------------------------
+
+A new indexing method has been added: PinkIndexer.  This is a new indexing
+algorithm which can index wide bandwidth data as well as flat Ewald sphere
+(i.e. electron diffraction) data.  For more information, see:
+Gevorkov et al., Acta Cryst. A72 (2020) doi:10.1107/S2053273319015559
+
+To accompany the new indexing algorithm, the reflection position calculation
+("prediction") has also been improved.  The results should be better for all
+types of data, particularly at low resolution.
+
+Further improvements for wide bandwidth and electron data are planned for future
+releases of CrystFEL.
+
+
+Introduction of cell_tool
+-------------------------
+
+A new CrystFEL core program has been added: cell_tool.  This program can perform
+a variety of operations on unit cell files, similar to how get_hkl can do things
+with reflection data files.  The tasks include: finding indexing ambiguities,
+turning a centered cell into a primitive one, calculating power ring positions,
+comparing unit cells, transforming unit cells, and working out all the possible
+cell choices for centered monoclinic cells.
+
+
+New unit cell comparison algorithm
+----------------------------------
+
+The way that indexamajig compares indexing results to your target cell has been
+improved.  Previously,  the option --no-cell-combinations was needed for certain
+unit cells.  This is no longer the case, and the new comparison should be more
+reliable in all cases.
+
+
+CBF files now handled natively
+------------------------------
+
+The CBF library (CBFlib) is no longer used in CrystFEL.  Most common types of
+CBF file can now be read directly by CrystFEL.  If you find a CBF file that
+cannot be read, please get in touch so we can fix it.
+
+
+Resolution limits no longer needed for ambigator
+------------------------------------------------
+
+Previously, a careful choice of resolution bounds (--highres and --lowres) was
+needed for good results with ambigator.  Now, the reflections will be compared
+and correlated in three resolution bins (>10, 10-2.5 and <2.5 Angstroms).  For
+most datasets, this allows ambiguities to be resolved easily without any manual
+adjustment.  You can still override this and use your own resolution limits,
+if you need to.
+
+
+API changes
+-----------
+
+The following changes have been made to the libcrystfel API.
+
+New functions:
+	- cell_print_full
+	- compare_cell_parameters
+	- compare_cell_parameters_and_orientation
+	- compare_permuted_cell_parameters_and_orientation
+	- compare_derivative_cell_parameters
+	- compare_reindexed_cell_parameters
+	- cell_get_G6
+	- cell_transform_gsl_direct
+	- cell_transform_rational
+	- cell_transform_rational_inverse
+	- cell_transform_intmat
+	- cell_transform_intmat_inverse
+	- crystal_get_image_const
+	- get_detector_geometry_from_string
+	- write_detector_geometry_3
+	- multi_event_geometry
+	- parse_polarisation
+	- indexing_methods
+	- intmat_size
+	- intmat_zero
+	- intmat_create_3x3
+	- transform_indices
+	- intmat_times_intmat
+	- indexing_peak_check
+	- pinkIndexer_prepare/cleanup/probe/run_pinkIndexer
+	- rtnl_zero
+	- rtnl
+	- rtnl_as_double
+	- rtnl_mul/div/add/sub/cmp/abs
+	- rtnl_list
+	- rtnl_mtx_new/copy/get/set/free/print/dev
+	- rtnl_mtx_from_intmat
+	- rtnl_mtx_is_identity/perm
+	- intmat_from_rtnl_mtx
+	- rtnl_mtx_identity
+	- rtnlmtx_times_rtnlmtx
+	- rtnlmtx_times_intmat
+	- intmat_times_rtnlmtx
+	- transform_fractional_coords_rtnl
+	- transform_fractional_coords_rtnl_inverse
+	- spectrum_new/free/load
+	- spectrum_set_gaussians
+	- spectrum_get_gaussian
+	- spectrum_get_num_gaussians
+	- spectrum_set_pdf
+	- spectrum_get_range
+	- spectrum_get_density_at_k
+	- spectrum_generate_tophat/gaussian/sase/twocolour
+	- free_stuff_from_stream
+	- stream_geometry_file
+	- parse_symmetry_operation
+	- parse_cell_transformation
+	- mean_variance
+
+Removed functions:
+	- cell_transform
+	- cell_transform_inverse
+	- tfn_identity/from_intmat/combine/print/inverse/vector/free
+	- transform_cell_gsl  [use cell_transform_ functions instead]
+	- match_cell
+	- match_cell_ab
+	- compare_cells  [use compare_*_cell_parmeters functions instead]
+	- intmat_intvec_mult
+	- intmat_intmat_mult  [use intmat_times_intmat instead]
+
+Changed function prototypes:
+	- uncenter_cell
+	- cell_get_parameters/cartesian/reciprocal  [const added]
+	- panel_number  [const added]
+	- polarisation_correction  [uses new 'polarisation' structure]
+	- setup_indexing
+	- intmat_copy  [const added]
+	- taketwo_prepare/index
+
+Changed structure/enum definitions:
+	- UnitCellTransformation  [removed, use IntegerMatrix/RationalMatrix instead]
+	- struct rigid_group  [removed unused panel position deltas]
+	- PartialityModel  [added PMODEL_OFFSET and PMODEL_GGPM]
+	- struct spectrum/struct sample  [removed, use Spectrum instead]
+	- struct image  [removed num_peaks and num_saturated_peaks]
+	- IndexingMethod  [added INDEXING_PINKINDEXER]
+	- IndexingFlags  [removed _CHECK_CELL_COMBINATIONS/AXES, added _CHECK_CELL]
+	- IntegrationMethod  [removed INTEGRATION_RESCUT]