diff options
| -rw-r--r-- | ChangeLog | 31 | ||||
| -rw-r--r-- | Makefile.am | 2 | ||||
| -rw-r--r-- | announcement-0.6.1 | 43 | ||||
| -rw-r--r-- | announcement-0.6.2 | 44 | ||||
| -rw-r--r-- | configure.ac | 2 | ||||
| -rw-r--r-- | doc/reference/libcrystfel/CrystFEL-docs.sgml | 2 | ||||
| -rw-r--r-- | libcrystfel/Makefile.am | 2 | ||||
| -rw-r--r-- | relnotes-0.6.1 | 165 | ||||
| -rw-r--r-- | relnotes-0.6.2 | 154 |
9 files changed, 233 insertions, 212 deletions
@@ -1,3 +1,34 @@ +CrystFEL version 0.6.2, 21st March 2016 +--------------------------------------- + +- partialator: Stability improvements to scaling algorithm, default --push-res + value changed from zero to infinity (no resolution cutoff by default). +- partialator: Added --custom-split, --max-rel-B and --output-every-cycle +- Added "retry" indexing method to improve indexing rate. +- Per-pixel saturation values can now be used (needed for CSPAD dual-gain mode). +- Fixed a regression with indexing using MOSFLM introduced near 0.6.1. +- indexamajig: Fixed a bug which occasionally caused corrupted stream output + ("Failed while reading reflections"). +- indexamajig: gradient fitting of background now optional and OFF by default. +- indexamajig: Added --profile for speed monitoring. +- indexamajig: Added "felix" indexing method. +- scripts/ave-resolution: Show resolution histogram. +- scripts/turbo-index: Added a missing line +- hdfsee: Improve quality of display by rounding pixel coordinates. +- hdfsee: Can now safely save new geometry file over old one. +- Target unit cell is now written to the stream. +- pattern_sim: Fixed bugs in --gpu and --powder. +- compare_hkl: Avoid double counting of reflections when calculating anomalous + figures of merit. +- geoptimiser: Major refactoring +- ambigator: fixes for a few problem when using very large numbers of correlation + coefficients or patterns. +- Some crashes and hangs in asdf were fixed. +- The "random" partiality model was improved. +- Prototype for prepare_indexing() changed, otherwise only additions were made + to the API. + + CrystFEL version 0.6.1, 3rd August 2015 --------------------------------------- diff --git a/Makefile.am b/Makefile.am index b1a477c4..5a916cad 100644 --- a/Makefile.am +++ b/Makefile.am @@ -29,7 +29,7 @@ TESTS = tests/list_check $(MERGE_CHECKS) $(PARTIAL_CHECKS) \ tests/prediction_gradient_check EXTRA_DIST += $(MERGE_CHECKS) $(PARTIAL_CHECKS) -EXTRA_DIST += relnotes-0.6.1 announcement-0.6.1 +EXTRA_DIST += relnotes-0.6.2 announcement-0.6.2 if BUILD_HDFSEE bin_PROGRAMS += src/hdfsee diff --git a/announcement-0.6.1 b/announcement-0.6.1 deleted file mode 100644 index 85137402..00000000 --- a/announcement-0.6.1 +++ /dev/null @@ -1,43 +0,0 @@ -Dear CrystFEL users and interested crystallographers, - -CrystFEL version 0.6.1 has been released. The most interesting changes -in this version are the new "prediction refinement" stage in -"indexamajig" and improvements to the scaling and post-refinement -program "partialator". There are also a couple of new indexing methods. - -The prediction refinement stage improves the quality of spot prediction -and hence the initial parameter estimates of the spot partialities. It -is also able to update the central beam position. - -The improvements to partialator include a significantly more stable -scaling procedure. Partialator is no longer considered experimental as -of this version, so please try it and send feedback. With the -combination of this and the new prediction refinement, many users are -already finding significant improvements in data quality. See the -release notes (link below) and the updated tutorial on the CrystFEL -website for more details. - -The new indexing methods are "asdf", which is an algorithm built into -CrystFEL which therefore requires no external programs, and -"mosflm-cell" which uses the prior unit cell algorithm in recent (7.2.0 -or later) versions of mosflm. "mosflm-cell" is now the default -behaviour for mosflm - if it doesn't work for you, explicitly specify -"mosflm-nocell" to restore the old behaviour. - -In addition, there are many smaller improvements, new features and bug -fixes. - -Thanks for all of your past and future feedback and contributions, and -your continued use of CrystFEL. As ever, please contact me directly if -you would prefer to be unsubscribed from this mailing list. - -CrystFEL website: -http://www.desy.de/~twhite/crystfel - -Release notes for version 0.6.1: -http://www.desy.de/~twhite/crystfel/relnotes-0.6.1 - -CrystFEL tutorial: -http://www.desy.de/~twhite/crystfel/tutorial - -Tom diff --git a/announcement-0.6.2 b/announcement-0.6.2 new file mode 100644 index 00000000..e92ce453 --- /dev/null +++ b/announcement-0.6.2 @@ -0,0 +1,44 @@ +Dear CrystFEL users and interested crystallographers, + +CrystFEL version 0.6.2 has been released. The most interesting changes +in this version are improvements to the stability of scaling in partialator, +the addition of "custom dataset splitting" for time-resolved/isomorphous +replacement experiments and more, "retry" indexing to increase the indexing +rate, and support for per-pixel saturation values needed to correctly handle +dual-gain CSPAD data. + +The improvements to scaling stability should improve the results with most +data sets, but particularly those where there have been problems with large +numbers of crystals getting rejected due to excessively large relative B +factors. + +"Custom dataset splitting" enables you to process multiple datasets in one +large scaling (or post-refinement) job, and separate them afterwards. This +eliminates any possible concerns over uniqueness of the solution with these new +data processing methods. + +"Retry" indexing is where the indexing method is repeated, if it fails the first +time, after deleting some of the weakest peaks. In many cases, this leads to a +successfully indexed pattern. This can sometimes double the indexing rate! +However, it makes it slower to process if the indexing rate is low, and you can +disable it by adding "-noretry" to all the indexing methods. + +See the release notes for more discussion of these improvements. + +In addition, there are many smaller improvements, new features and bug +fixes. See the ChangeLog for details. + +Thanks for all of your past and future feedback and contributions, and +your continued use of CrystFEL. As ever, please contact me directly if +you would prefer to be unsubscribed from this mailing list. + +CrystFEL website: +http://www.desy.de/~twhite/crystfel + +Release notes for version 0.6.2: +http://www.desy.de/~twhite/crystfel/relnotes-0.6.2 + +CrystFEL tutorial: +http://www.desy.de/~twhite/crystfel/tutorial + +Tom diff --git a/configure.ac b/configure.ac index 9d4d7edf..8044a145 100644 --- a/configure.ac +++ b/configure.ac @@ -1,6 +1,6 @@ dnl Process this file with autoconf to produce a configure script. -AC_INIT([crystfel],[0.6.1],[taw@physics.org]) +AC_INIT([crystfel],[0.6.2],[taw@physics.org]) AC_CONFIG_AUX_DIR([build-aux]) AC_CONFIG_HEADERS([config.h]) AM_INIT_AUTOMAKE([subdir-objects]) diff --git a/doc/reference/libcrystfel/CrystFEL-docs.sgml b/doc/reference/libcrystfel/CrystFEL-docs.sgml index 9ef5d945..407f23b4 100644 --- a/doc/reference/libcrystfel/CrystFEL-docs.sgml +++ b/doc/reference/libcrystfel/CrystFEL-docs.sgml @@ -8,7 +8,7 @@ <bookinfo> <title>CrystFEL Reference Manual</title> <releaseinfo> - For libcrystfel from CrystFEL 0.6.1. + For libcrystfel from CrystFEL 0.6.2. </releaseinfo> <abstract> This is the internal documentation for CrystFEL. Unless you are looking at diff --git a/libcrystfel/Makefile.am b/libcrystfel/Makefile.am index f2760dac..d7142f6a 100644 --- a/libcrystfel/Makefile.am +++ b/libcrystfel/Makefile.am @@ -1,6 +1,6 @@ lib_LTLIBRARIES = libcrystfel.la libcrystfel_la_LIBADD = ../lib/libgnu.la @LIBCRYSTFEL_LIBS@ $(PTY_LIB) -libcrystfel_la_LDFLAGS = -version-info 7:0:0 +libcrystfel_la_LDFLAGS = -version-info 8:0:0 libcrystfel_la_SOURCES = src/reflist.c src/utils.c src/cell.c src/detector.c \ src/thread-pool.c src/image.c src/hdf5-file.c \ diff --git a/relnotes-0.6.1 b/relnotes-0.6.1 deleted file mode 100644 index 4c189cd6..00000000 --- a/relnotes-0.6.1 +++ /dev/null @@ -1,165 +0,0 @@ -CrystFEL - Crystallography with a FEL -------------------------------------- - -Release notes for version 0.6.1 - -Copyright © 2012-2015 Deutsches Elektronen-Synchrotron DESY, - a research centre of the Helmholtz Association. - -Authors: - Thomas White <taw@physics.org> - Richard Kirian <rkirian@asu.edu> - Kenneth Beyerlein <kenneth.beyerlein@desy.de> - Andrew Aquila <andrew.aquila@cfel.de> - Andrew Martin <andrew.martin@desy.de> - Lorenzo Galli <lorenzo.galli@desy.de> - Chun Hong Yoon <chun.hong.yoon@desy.de> - Kenneth Beyerlein <kenneth.beyerlein@desy.de> - Karol Nass <karol.nass@desy.de> - Nadia Zatsepin <nadia.zatsepin@asu.edu> - Anton Barty <anton.barty@desy.de> - Cornelius Gati <cornelius.gati@desy.de> - Fedor Chervinskii <fedor.chervinskii@gmail.com> - Alexandra Tolstikova <alexandra.tolstikova@desy.de> - Wolfgang Brehm <wolfgang.brehm@gmail.com> - Valerio Mariani <valerio.mariani@desy.de> - Parker de Waal <Parker.deWaal@vai.org> - Takanori Nakane <nakane.t@gmail.com> - Keitaro Yamashita <k.yamashita@spring8.or.jp> - Oleksandr Yefanov <oleksandr.yefanov@cfel.de> - -CrystFEL is free software: you can redistribute it and/or modify it under the -terms of the GNU General Public License as published by the Free Software -Foundation, either version 3 of the License, or (at your option) any later -version. - -CrystFEL is distributed in the hope that it will be useful, but WITHOUT ANY -WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A -PARTICULAR PURPOSE. See the GNU General Public License for more details. - -You should have received a copy of the GNU General Public License along with -CrystFEL. If not, see <http://www.gnu.org/licenses/>. - - -Overview --------- - -The most important new features in this version of CrystFEL are: - -- Introduction of "prediction refinement". - -- Scaling and post-refinement with "partialator". - -- New indexing methods: mosflm-cell and asdf. - -- process_hkl --even-only and --odd-only instead of "alternate-stream". - -These new features have individual sections below. In addition, there are many -other new developments. See the ChangeLog or the changes page on the website -for more details. There were, of course, the usual large number of smaller -refinements and bug fixes. - - -Introduction of "prediction refinement" ---------------------------------------- - -A new refinement step has been introduced between the indexing and prediction/ -integration stages of indexamajig. The refinement aims to maximise the -agreement between the predicted spot positions and the peak positions found by -the peak search step, at the same time as putting the peaks as close to the -Bragg condition as possible. This increases the accuracy of spot prediction, -which has positive effects on the downstream processing stages. - -The prediction refinement stage allows the central beam position to move -relative to the entire detector. The required offset for each crystal is -recorded in the stream and can be visualised using scripts/detector-shift. - -If, somehow, the prediction refinement causes problems for you, you can disable -it using the indexamajig option "--no-refine". - - -Scaling and post-refinement with "partialator" ----------------------------------------------- - -The scaling and post-refinement program partialator is no longer considered -experimental in this version. Many users are already finding significant -improvements in their data simply by applying scaling without partiality: - -$ partialator -i my.stream -o my.hkl -y mypg --model=unity --iterations=3 - -A further improvement is sometimes obtained by applying partialities and doing -full post-refinement: - -$ partialator -i my.stream -o my.hkl -y mypg --model=scsphere --iterations=3 - -Note that partialator applies a per-crystal resolution cutoff to stabilise the -scaling calculation. Therefore, you should experiment with different values -for "--push-res". - -Further improvements to partiality modelling, scaling and merging will be -included in the near future. - - -New indexing methods --------------------- - -This version of CrystFEL can make use of the known unit cell algorithm included -with very recent (7.2.0 or later) versions of mosflm. The known unit cell -algorithm is used by default if you provide lattice parameters to indexamajig. -If this isn't what you want, or if it doesn't work well for you, use -"mosflm-nocell" as your indexing method to restore the old behaviour. - -A new indexing method, asdf, has been added. This algorithm is built into -CrystFEL directly, so no temporary files or external programs are needed. - -The old indexing methods "grainspotter" and "reax" have been removed. - - -process_hkl --even-only and --odd-only --------------------------------------- - -When using process_hkl for merging, you no longer need to use "alternate-stream" -to split the stream into two parts in order to calculate figures of merit such -as Rsplit. Instead, simply use the "--even-only" and "--odd-only" options to -merge only even-numbered and odd-numbered crystals from the stream: - -$ process_hkl -i my.stream -o my.hkl1 -y mypg --even-only -$ process_hkl -i my.stream -o my.hkl2 -y mypg --odd-only -$ compare_hkl my.hkl1 my.hkl2 -y mypg --fom=rsplit [..etc..] - -Note that the numbering of crystals for this purpose is simply according to the -order they appear in the stream. For most purposes, this is effectively random. - -Note further that partialator writes the ".hkl1" and ".hkl2" split-merged -datasets for you automatically. - - -API changes ------------ - -The following changes have been made to the libcrystfel API: - -New functions: - - asdf_prepare(), run_asdf(), asdf_cleanup() - - cell_get_volume() - - crystal_{get,set}_Bfac() - - crystal_{get,set,add}_notes() - - detector_has_clen_references() - - {x,y,r}_gradient() - - refine_prediction() - - refine_radius() - - {get,set}_panel() - - {get,set}_flag() - - solve_svd() - -Removed functions: - - select_intersections() - - update_partialities_2() - - get_excitation_error() - - extract_f_from_stuff() - - grainspotter_{prepare,index,cleanup}() - - reax_{prepare,index,cleanup}() - -Changed function prototypes: - - integrate_all_4() - - set_detector_pos() diff --git a/relnotes-0.6.2 b/relnotes-0.6.2 new file mode 100644 index 00000000..daf48f3f --- /dev/null +++ b/relnotes-0.6.2 @@ -0,0 +1,154 @@ +CrystFEL - Crystallography with a FEL +------------------------------------- + +Release notes for version 0.6.2 + +Copyright © 2012-2016 Deutsches Elektronen-Synchrotron DESY, + a research centre of the Helmholtz Association. + +Authors: + Thomas White <taw@physics.org> + Richard Kirian <rkirian@asu.edu> + Kenneth Beyerlein <kenneth.beyerlein@desy.de> + Andrew Aquila <andrew.aquila@cfel.de> + Andrew Martin <andrew.martin@desy.de> + Lorenzo Galli <lorenzo.galli@desy.de> + Chun Hong Yoon <chun.hong.yoon@desy.de> + Kenneth Beyerlein <kenneth.beyerlein@desy.de> + Karol Nass <karol.nass@desy.de> + Nadia Zatsepin <nadia.zatsepin@asu.edu> + Anton Barty <anton.barty@desy.de> + Cornelius Gati <cornelius.gati@desy.de> + Fedor Chervinskii <fedor.chervinskii@gmail.com> + Alexandra Tolstikova <alexandra.tolstikova@desy.de> + Wolfgang Brehm <wolfgang.brehm@gmail.com> + Valerio Mariani <valerio.mariani@desy.de> + Parker de Waal <Parker.deWaal@vai.org> + Takanori Nakane <nakane.t@gmail.com> + Keitaro Yamashita <k.yamashita@spring8.or.jp> + Oleksandr Yefanov <oleksandr.yefanov@cfel.de> + +CrystFEL is free software: you can redistribute it and/or modify it under the +terms of the GNU General Public License as published by the Free Software +Foundation, either version 3 of the License, or (at your option) any later +version. + +CrystFEL is distributed in the hope that it will be useful, but WITHOUT ANY +WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A +PARTICULAR PURPOSE. See the GNU General Public License for more details. + +You should have received a copy of the GNU General Public License along with +CrystFEL. If not, see <http://www.gnu.org/licenses/>. + + +Overview +-------- + +The most important new features in this version of CrystFEL are: + +- Stability improvements to the scaling algorithm in partialator. + +- Addition of "custom dataset splitting" in partialator. + +- "retry" indexing. + +- Per-pixel saturation values + +These changes have sections below. In addition, there were many bug fixes and +smaller improvements. + + +Stability improvements in partialator +------------------------------------- + +The scaling algorithm in partialator has been altered such that only the strong +reflections contribute to the scaling factors at all. This reduces the +"drifting" of scale factors which happened before, and helps avoid the situation +where a very large number of crystals get rejected with large B factors. +Of course, all reflections contribute to the final merged data set as before. + +In CrystFEL 0.6.1, we needed to reduce the amount of noise going into the +scaling algorithm to make it work. This was done by including reflections only +up to the apparent (conservative) resolution limit of each crystal made by +indexamajig, and could be overridden by using the "--push-res" option of +partialator. The default --push-res value was zero, which meant to ignore any +reflections higher than the limit. With the stabilised algorithm, this is no +longer necessary, and hence the default --push-res value has been changed to +infinity, i.e. no cutoff at all. You can restore the old behaviour with +--push-res=0. + + +Custom dataset splitting in partialator +--------------------------------------- + +When performing a time-resolved or isomorphous replacement experiment (plus some +other types of experiment), it is preferable to ensure that the data for all +time points has been processed identically. Rather than processing each time +point independently with separate runs of partialator, it is better to process +them all together and do the splitting into time points just before the final +output. Consider, for example, the case of simple scaling (without a B +factor): when merging independently, the resulting datasets would probably end +up with different overall scaling factors. When comparing the results, you +would need to take this difference into account. In practice, most programs +can do that job easily, but what about if a B factor is included? And what if +partialities are included - how unique is the solution? + +Partialator's new "custom dataset splitting" option allows you to provide a +text file containing a list of filenames, event numbers and dataset +names, one event (detector frame) per line. All crystals will be refined +together, but they will be merged according to the dataset names you give. The +parameters (scaling factors, partialities etc) determined during the joint +refinement will be applied. For each dataset, a separate pair of split +half-datasets will also be written, allowing you to calculate figures of merit +such as Rsplit and CC1/2 for each one. + + +"retry" indexing +---------------- + +If an indexing method fails to index a pattern (or if the solution doesn't get +through the prediction refinement stage), this version of CrystFEL will delete +the weakest few peaks and try again. This can increase the indexing yield a lot +(sometimes a factor of two!), but decreases the speed. If you'd rather keep +things fast, add "-noretry" to each of your indexing methods. + + +Per-pixel saturation values +--------------------------- + +When using dual-gain mode on the CSPAD (or any other detector which offers a +similar feature), our current advice is to correct the pixel values in one of +the regions by multiplying them by the relative gain factor. This will give +image data which looks good, but the saturation values will be different in the +two regions. To handle this, CrystFEL 0.6.2 allows you to prepare a map, in +HDF5 format, giving the saturation value for each pixel individually. The +script "gaincal-to-saturation-map" can be used to create a suitable file from +the gain map (e.g. as generated and used by Cheetah). Specify the filename and +HDF5 path of the saturation map using saturation_map and saturation_map_file +in the geometry file - these work exactly like the bad pixel map. Reflections +containing any pixel above this value will be rejected, in addition to any +reflections which get rejected by the usual saturation criteria (so remove any +max_adu directives if you want the saturation map to be the only decider!). + + +API changes +----------- + +The following changes have been made to the libcrystfel API: + +New functions: + - write_cell() + - find_orig_panel_number() + - felix_{prepare,index,cleanup}() + - sort_peaks() + - remove_flagged_crystals() + - open_stream_for_write_3() + +Removed functions: + (none) + +Changed function prototypes: + - prepare_indexing() + +"data", "flags" and "twotheta" were removed from "struct image", and "sat" was +added. |