diff options
| -rw-r--r-- | ChangeLog | 4 | ||||
| -rw-r--r-- | Makefile.am | 2 | ||||
| -rw-r--r-- | announcement-0.6.2 | 44 | ||||
| -rw-r--r-- | announcement-0.6.3 | 35 | ||||
| -rw-r--r-- | configure.ac | 2 | ||||
| -rw-r--r-- | doc/reference/libcrystfel/CrystFEL-docs.sgml | 2 | ||||
| -rw-r--r-- | libcrystfel/Makefile.am | 2 | ||||
| -rw-r--r-- | relnotes-0.6.2 | 154 | ||||
| -rw-r--r-- | relnotes-0.6.3 | 211 |
9 files changed, 252 insertions, 204 deletions
@@ -1,5 +1,5 @@ -Changes in this development version ------------------------------------ +CrystFEL version 0.6.3, 24th July 2017 +-------------------------------------- - indexamajig --peaks=peakfinder8 was added (Valerio Mariani, Oleksandr Yefanov) - CBF files are now supported as input. diff --git a/Makefile.am b/Makefile.am index fda28fbc..78456727 100644 --- a/Makefile.am +++ b/Makefile.am @@ -29,7 +29,7 @@ TESTS = tests/list_check $(MERGE_CHECKS) $(PARTIAL_CHECKS) \ tests/prediction_gradient_check EXTRA_DIST += $(MERGE_CHECKS) $(PARTIAL_CHECKS) -EXTRA_DIST += relnotes-0.6.2 announcement-0.6.2 +EXTRA_DIST += relnotes-0.6.3 announcement-0.6.3 if BUILD_HDFSEE bin_PROGRAMS += src/hdfsee diff --git a/announcement-0.6.2 b/announcement-0.6.2 deleted file mode 100644 index e92ce453..00000000 --- a/announcement-0.6.2 +++ /dev/null @@ -1,44 +0,0 @@ -Dear CrystFEL users and interested crystallographers, - -CrystFEL version 0.6.2 has been released. The most interesting changes -in this version are improvements to the stability of scaling in partialator, -the addition of "custom dataset splitting" for time-resolved/isomorphous -replacement experiments and more, "retry" indexing to increase the indexing -rate, and support for per-pixel saturation values needed to correctly handle -dual-gain CSPAD data. - -The improvements to scaling stability should improve the results with most -data sets, but particularly those where there have been problems with large -numbers of crystals getting rejected due to excessively large relative B -factors. - -"Custom dataset splitting" enables you to process multiple datasets in one -large scaling (or post-refinement) job, and separate them afterwards. This -eliminates any possible concerns over uniqueness of the solution with these new -data processing methods. - -"Retry" indexing is where the indexing method is repeated, if it fails the first -time, after deleting some of the weakest peaks. In many cases, this leads to a -successfully indexed pattern. This can sometimes double the indexing rate! -However, it makes it slower to process if the indexing rate is low, and you can -disable it by adding "-noretry" to all the indexing methods. - -See the release notes for more discussion of these improvements. - -In addition, there are many smaller improvements, new features and bug -fixes. See the ChangeLog for details. - -Thanks for all of your past and future feedback and contributions, and -your continued use of CrystFEL. As ever, please contact me directly if -you would prefer to be unsubscribed from this mailing list. - -CrystFEL website: -http://www.desy.de/~twhite/crystfel - -Release notes for version 0.6.2: -http://www.desy.de/~twhite/crystfel/relnotes-0.6.2 - -CrystFEL tutorial: -http://www.desy.de/~twhite/crystfel/tutorial - -Tom diff --git a/announcement-0.6.3 b/announcement-0.6.3 new file mode 100644 index 00000000..db8d0adb --- /dev/null +++ b/announcement-0.6.3 @@ -0,0 +1,35 @@ +Dear CrystFEL users and interested crystallographers, + +It's been a long time since the last CrystFEL release, but we have not been +resting. CrystFEL version 0.6.3 has been released. + +This version incorporates a large number of "backstage" improvements and +alterations. The most interesting changes for users are: + +- The incorporation of "peak finder 8" from Cheetah (also used in OnDA) + +- Support for images in CBF format + +- Incorporation of "TakeTwo" indexing algorithm + +- Hit finder functionality (skip frames with few peaks) + +- Better support for three-dimensional detector geometries + +See the release notes for more discussion of these improvements, and the +ChangeLog for full details of the other improvements. + +Thanks for all of your past and future feedback and contributions, and +your continued use of CrystFEL. As ever, please contact me directly if +you would prefer to be unsubscribed from this mailing list. + +CrystFEL website: +https://www.desy.de/~twhite/crystfel + +Release notes for version 0.6.3: +https://www.desy.de/~twhite/crystfel/relnotes-0.6.3 + +CrystFEL tutorial: +https://www.desy.de/~twhite/crystfel/tutorial + +Tom diff --git a/configure.ac b/configure.ac index 2a5f40cf..41071139 100644 --- a/configure.ac +++ b/configure.ac @@ -1,6 +1,6 @@ dnl Process this file with autoconf to produce a configure script. -AC_INIT([crystfel],[0.6.2],[taw@physics.org]) +AC_INIT([crystfel],[0.6.3],[taw@physics.org]) AC_CONFIG_AUX_DIR([build-aux]) AC_CONFIG_HEADERS([config.h]) AM_INIT_AUTOMAKE([subdir-objects]) diff --git a/doc/reference/libcrystfel/CrystFEL-docs.sgml b/doc/reference/libcrystfel/CrystFEL-docs.sgml index 407f23b4..048f4167 100644 --- a/doc/reference/libcrystfel/CrystFEL-docs.sgml +++ b/doc/reference/libcrystfel/CrystFEL-docs.sgml @@ -8,7 +8,7 @@ <bookinfo> <title>CrystFEL Reference Manual</title> <releaseinfo> - For libcrystfel from CrystFEL 0.6.2. + For libcrystfel from CrystFEL 0.6.3. </releaseinfo> <abstract> This is the internal documentation for CrystFEL. Unless you are looking at diff --git a/libcrystfel/Makefile.am b/libcrystfel/Makefile.am index fef2e11f..24cb7358 100644 --- a/libcrystfel/Makefile.am +++ b/libcrystfel/Makefile.am @@ -1,6 +1,6 @@ lib_LTLIBRARIES = libcrystfel.la libcrystfel_la_LIBADD = ../lib/libgnu.la @LIBCRYSTFEL_LIBS@ $(PTY_LIB) -libcrystfel_la_LDFLAGS = -version-info 8:0:0 +libcrystfel_la_LDFLAGS = -version-info 9:0:0 libcrystfel_la_SOURCES = src/reflist.c src/utils.c src/cell.c src/detector.c \ src/thread-pool.c src/image.c src/hdf5-file.c \ diff --git a/relnotes-0.6.2 b/relnotes-0.6.2 deleted file mode 100644 index daf48f3f..00000000 --- a/relnotes-0.6.2 +++ /dev/null @@ -1,154 +0,0 @@ -CrystFEL - Crystallography with a FEL -------------------------------------- - -Release notes for version 0.6.2 - -Copyright © 2012-2016 Deutsches Elektronen-Synchrotron DESY, - a research centre of the Helmholtz Association. - -Authors: - Thomas White <taw@physics.org> - Richard Kirian <rkirian@asu.edu> - Kenneth Beyerlein <kenneth.beyerlein@desy.de> - Andrew Aquila <andrew.aquila@cfel.de> - Andrew Martin <andrew.martin@desy.de> - Lorenzo Galli <lorenzo.galli@desy.de> - Chun Hong Yoon <chun.hong.yoon@desy.de> - Kenneth Beyerlein <kenneth.beyerlein@desy.de> - Karol Nass <karol.nass@desy.de> - Nadia Zatsepin <nadia.zatsepin@asu.edu> - Anton Barty <anton.barty@desy.de> - Cornelius Gati <cornelius.gati@desy.de> - Fedor Chervinskii <fedor.chervinskii@gmail.com> - Alexandra Tolstikova <alexandra.tolstikova@desy.de> - Wolfgang Brehm <wolfgang.brehm@gmail.com> - Valerio Mariani <valerio.mariani@desy.de> - Parker de Waal <Parker.deWaal@vai.org> - Takanori Nakane <nakane.t@gmail.com> - Keitaro Yamashita <k.yamashita@spring8.or.jp> - Oleksandr Yefanov <oleksandr.yefanov@cfel.de> - -CrystFEL is free software: you can redistribute it and/or modify it under the -terms of the GNU General Public License as published by the Free Software -Foundation, either version 3 of the License, or (at your option) any later -version. - -CrystFEL is distributed in the hope that it will be useful, but WITHOUT ANY -WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A -PARTICULAR PURPOSE. See the GNU General Public License for more details. - -You should have received a copy of the GNU General Public License along with -CrystFEL. If not, see <http://www.gnu.org/licenses/>. - - -Overview --------- - -The most important new features in this version of CrystFEL are: - -- Stability improvements to the scaling algorithm in partialator. - -- Addition of "custom dataset splitting" in partialator. - -- "retry" indexing. - -- Per-pixel saturation values - -These changes have sections below. In addition, there were many bug fixes and -smaller improvements. - - -Stability improvements in partialator -------------------------------------- - -The scaling algorithm in partialator has been altered such that only the strong -reflections contribute to the scaling factors at all. This reduces the -"drifting" of scale factors which happened before, and helps avoid the situation -where a very large number of crystals get rejected with large B factors. -Of course, all reflections contribute to the final merged data set as before. - -In CrystFEL 0.6.1, we needed to reduce the amount of noise going into the -scaling algorithm to make it work. This was done by including reflections only -up to the apparent (conservative) resolution limit of each crystal made by -indexamajig, and could be overridden by using the "--push-res" option of -partialator. The default --push-res value was zero, which meant to ignore any -reflections higher than the limit. With the stabilised algorithm, this is no -longer necessary, and hence the default --push-res value has been changed to -infinity, i.e. no cutoff at all. You can restore the old behaviour with ---push-res=0. - - -Custom dataset splitting in partialator ---------------------------------------- - -When performing a time-resolved or isomorphous replacement experiment (plus some -other types of experiment), it is preferable to ensure that the data for all -time points has been processed identically. Rather than processing each time -point independently with separate runs of partialator, it is better to process -them all together and do the splitting into time points just before the final -output. Consider, for example, the case of simple scaling (without a B -factor): when merging independently, the resulting datasets would probably end -up with different overall scaling factors. When comparing the results, you -would need to take this difference into account. In practice, most programs -can do that job easily, but what about if a B factor is included? And what if -partialities are included - how unique is the solution? - -Partialator's new "custom dataset splitting" option allows you to provide a -text file containing a list of filenames, event numbers and dataset -names, one event (detector frame) per line. All crystals will be refined -together, but they will be merged according to the dataset names you give. The -parameters (scaling factors, partialities etc) determined during the joint -refinement will be applied. For each dataset, a separate pair of split -half-datasets will also be written, allowing you to calculate figures of merit -such as Rsplit and CC1/2 for each one. - - -"retry" indexing ----------------- - -If an indexing method fails to index a pattern (or if the solution doesn't get -through the prediction refinement stage), this version of CrystFEL will delete -the weakest few peaks and try again. This can increase the indexing yield a lot -(sometimes a factor of two!), but decreases the speed. If you'd rather keep -things fast, add "-noretry" to each of your indexing methods. - - -Per-pixel saturation values ---------------------------- - -When using dual-gain mode on the CSPAD (or any other detector which offers a -similar feature), our current advice is to correct the pixel values in one of -the regions by multiplying them by the relative gain factor. This will give -image data which looks good, but the saturation values will be different in the -two regions. To handle this, CrystFEL 0.6.2 allows you to prepare a map, in -HDF5 format, giving the saturation value for each pixel individually. The -script "gaincal-to-saturation-map" can be used to create a suitable file from -the gain map (e.g. as generated and used by Cheetah). Specify the filename and -HDF5 path of the saturation map using saturation_map and saturation_map_file -in the geometry file - these work exactly like the bad pixel map. Reflections -containing any pixel above this value will be rejected, in addition to any -reflections which get rejected by the usual saturation criteria (so remove any -max_adu directives if you want the saturation map to be the only decider!). - - -API changes ------------ - -The following changes have been made to the libcrystfel API: - -New functions: - - write_cell() - - find_orig_panel_number() - - felix_{prepare,index,cleanup}() - - sort_peaks() - - remove_flagged_crystals() - - open_stream_for_write_3() - -Removed functions: - (none) - -Changed function prototypes: - - prepare_indexing() - -"data", "flags" and "twotheta" were removed from "struct image", and "sat" was -added. diff --git a/relnotes-0.6.3 b/relnotes-0.6.3 new file mode 100644 index 00000000..11a1f93b --- /dev/null +++ b/relnotes-0.6.3 @@ -0,0 +1,211 @@ +CrystFEL - Crystallography with a FEL +------------------------------------- + +Release notes for version 0.6.3 + +Copyright © 2012-2017 Deutsches Elektronen-Synchrotron DESY, + a research centre of the Helmholtz Association. + +Authors: + Thomas White <taw@physics.org> + Richard Kirian <rkirian@asu.edu> + Kenneth Beyerlein <kenneth.beyerlein@desy.de> + Andrew Aquila <andrew.aquila@cfel.de> + Andrew Martin <andrew.martin@desy.de> + Lorenzo Galli <lorenzo.galli@desy.de> + Chun Hong Yoon <chun.hong.yoon@desy.de> + Kenneth Beyerlein <kenneth.beyerlein@desy.de> + Karol Nass <karol.nass@desy.de> + Nadia Zatsepin <nadia.zatsepin@asu.edu> + Anton Barty <anton.barty@desy.de> + Cornelius Gati <cornelius.gati@desy.de> + Fedor Chervinskii <fedor.chervinskii@gmail.com> + Alexandra Tolstikova <alexandra.tolstikova@desy.de> + Wolfgang Brehm <wolfgang.brehm@gmail.com> + Valerio Mariani <valerio.mariani@desy.de> + Parker de Waal <Parker.deWaal@vai.org> + Takanori Nakane <nakane.t@gmail.com> + Keitaro Yamashita <k.yamashita@spring8.or.jp> + Oleksandr Yefanov <oleksandr.yefanov@cfel.de> + Steve Aplin <steve.aplin@desy.de> + Helen Ginn <helen@strubi.ox.ac.uk> + +CrystFEL is free software: you can redistribute it and/or modify it under the +terms of the GNU General Public License as published by the Free Software +Foundation, either version 3 of the License, or (at your option) any later +version. + +CrystFEL is distributed in the hope that it will be useful, but WITHOUT ANY +WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A +PARTICULAR PURPOSE. See the GNU General Public License for more details. + +You should have received a copy of the GNU General Public License along with +CrystFEL. If not, see <http://www.gnu.org/licenses/>. + + +Overview +-------- + +The most important new features in this version of CrystFEL are: + +- Support for CBF files as input + +- TakeTwo indexing algorithm + +- Improved peak detection and basic hitfinder functionality + +- Detector panels not perpendicular to X-ray beam, and "rail direction" + +These changes have sections below. In addition, there were many bug fixes and +other improvements. See the "ChangeLog" or the changes page on the CrystFEL +website for details. + + +Support for CBF files as input +------------------------------ + +Crystallographic Binary Files (CBF) can now be used as input for indexamajig +and hdfsee. To enable this functionality, you must have "CBFlib" installed. +Some distributions (eg Fedora) include CBFlib in their standard repositories - +simply install "cbflib-devel" or similar. Then compile and install CrystFEL as +usual. If CBFlib is installed somewhere non-standard, e.g. if you has to +install it manually, you will need to give the path with the "--with-cbflib-dir" +option to "./configure". + +After that, you can simply give the names of CBF files in the indexamajig input, +and everything should work as usual! There are some catches, for example you +cannot yet use a bad pixel mask with CBF files, but these will be fixed in a +future release. + + +TakeTwo indexing algorithm +-------------------------- + +This version of CrystFEL includes the TakeTwo indexing algorithm by Helen Ginn. +TakeTwo is a new indexing algorithm, designed specifically for diffraction +snapshots and operating completely differently to conventional FFT-based methods. + +To use it, simply use "taketwo" as the indexing method, or add it to the list of +indexing methods you have already: + $ indexamajig --indexing=taketwo + $ indexamajig --indexing=mosflm,taketwo,dirax + +If you use TakeTwo, please take careful note of the citation reminder which will +be shown to you, and cite the following paper in any resulting publications: + Ginn et al., Acta Cryst. (2016). D72, 956-965 + +TakeTwo is considered experimental in this version of CrystFEL. We welcome any +feedback, and will use the information to improve its performance for future +versions. + + +Improved peak detection and basic hitfinder functionality +--------------------------------------------------------- + +"Peak finder 8" from Cheetah, also used in OnDA, has been incorporated into +CrystFEL. It uses a radially varying background level to find peaks, which +works very well with most diffraction data. + +Simply use "--peaks=peakfinder8", and set --threshold, --min-snr, +--min-pix-count and --local-bg-radius. See the manual for other options. + +Indexamajig can now skip over patterns which have a small number of peaks. To +use this, use the option "--min-peaks=N", where N is the minimum number of peaks +before indexamajig will proceed to indexing (and hopefully integration) with the +pattern. + +By combining the improved peak detection with the hitfinder functionality and +CBF reading, you can process data from several different types of detector, +without converting the files to HDF5 format or using any external program for +hit finding! + +If the hit rate is very low, you might want to make CrystFEL stream smaller by +excluding non-hits from it using "--no-non-hits-in-stream". Without this +option, standard information about the non-hits (e.g. photon energy and peaks +found) will be recorded. + + +Detector panels not perpendicular to X-ray beam, and "rail direction" +---------------------------------------------------------------------- + +CrystFEL now supports three-dimensional detector geometry. The "fs" and "ss" +vectors (which are the real-space directions of the fast scan and slow scan +directions in the image data) can now contain a z component as well as x and y. + +For many detectors, the "rail" direction, along which the detector moves when +the "camera length" (sample-detector distance) is adjusted, is not perpendicular +to the detector plane. The geometry file now allows you to specify this "rail" +direction, containing x, y and z components. See the crystfel_geometry manual +page for more details. + + +API changes +----------- + +The following changes have been made to the libcrystfel API. The biggest +changes are the switch from the old "hdfile" API (which is still supported), +to the new "imagefile" API, while allows multiple file types to be used. +The indexing system was also updated, and there were changes to the detector +geometry system, which no longer uses the "data slab" representation internally. + +New functions: + - crystal_{get,set}_det_shift() + - panel_number() + - get_detector_geometry_2() + - fill_in_adu() + - adjust_centering_for_rail() + - get_peaks_2() + - get_peaks_cxi_2() + - imagefile_open() + - imagefile_read() + - imagefile_read_simple() + - imagefile_get_hdfile() + - imagefile_get_type() + - imagefile_copy_fields() + - imagefile_close() + - {new,free}_imagefile_field_list() + - add_imagefile_field() + - get_indm_from_string() + - setup_indexing() + - index_pattern_2() + - search_peaks_peakfinder8() + - reflist_add_command_and_version() + - reflist_{add,get}_notes() + - write_chunk_2() + - pointgroup_warning() + +New structure definitions: + - struct image + - struct imagefile_field_list + +Removed functions: + - get_q() + - find_panel() + - find_panel_number() + - fill_in_values() + - partial_event_substitution() + - build_indexer_list() + - prepare_indexing() + +Changed function prototypes: + - in_bad_region() + - simple_geometry() + - reverse_2d_mapping() + - hdfile_set_image() + - image_add_feature() + - image_feature_closest() + - image_reflection_closest() + - index_pattern() + - cleanup_indexing() + - write_chunk() + +Changed structure definitions: + - struct panel: removed {min,max}_{fs,ss}, added adu_per_photon, + added fsz,ssz, rail_{x,y,z} and clen_for_centering + - struct detector: removed max_{fs,ss} + - struct imagefeature: replaced "pn" with "p" + - struct beam_params: "copyme" is not an imagefile_field_list + (previously: copy_hdf5_field) + - struct image: "copyme" is not an imagefile_field_list + (previously: copy_hdf5_field). width and height remoted. + |