aboutsummaryrefslogtreecommitdiff
path: root/doc
diff options
context:
space:
mode:
Diffstat (limited to 'doc')
-rw-r--r--doc/articles/online.rst85
1 files changed, 85 insertions, 0 deletions
diff --git a/doc/articles/online.rst b/doc/articles/online.rst
new file mode 100644
index 00000000..84e037ca
--- /dev/null
+++ b/doc/articles/online.rst
@@ -0,0 +1,85 @@
+==========================================
+Real-time data processing with indexamajig
+==========================================
+
+Instead of reading from files, indexamajig can get its data over a ZeroMQ
+socket. Use ``--zmq-input`` instead of ``--input`` or ``-i``. An error will
+be generated if you use ``--zmq-input`` and ``--input`` or ``-i``
+simultaneously.
+
+Indexamajig assumes that the socket is a *pub/sub* socket. You will also need
+to specify which message prefixes to subscribe to. Use ``--zmq-subscribe`` for
+this::
+
+ indexamajig --zmq-input=tcp://127.0.0.1:5002 \
+ --zmq-subscribe=ondaframedata \
+ -o output.stream -g Eiger.geom ...
+
+You can use ``--zmq-subscribe`` multiple times to subscribe to multiple message
+prefixes.
+
+The option ``--no-image-data`` will be honoured, if given. This makes it
+possible to quickly check streaming data for "indexability".
+
+
+Data format
+===========
+
+In this version, CrystFEL makes the following assumptions about the data
+received via ZeroMQ:
+
+* The data is serialised using MessagePack
+* The data consists of either a single MsgPack 'map' object, or an array of
+ maps.
+ If there are multiple map objects in the array, only the first one will be
+ used. The others will be ignored.
+* The image data is given as a two-dimensional array (i.e. no 3D arrays with
+ 'panel number' as one of the dimensions).
+* The image data itself is given as a MsgPack 'map' object with "type"=>"<i4".
+
+Note that *all* of the above assumptions are 'open for negotiation' and will be
+relaxed in future CrystFEL versions, as new online data formats arise. Anyone
+interested in streaming CBF files over ZeroMQ?
+
+You can specify which map objects to look at in the geometry file. The
+following example will get the incident photon energy (in eV) and detector
+distance (in mm) by looking up the ``beam_energy`` and ``detector_distance``
+keys, respectively, in the MsgPack map object. It will then look up
+``detector_data`` to find the image data itself. See the next section for an
+explanation of ``peak_list``::
+
+ photon_energy = beam_energy eV
+ adu_per_photon = 1
+ clen = detector_distance mm
+ res = 5814.0
+ peak_list = peak_list
+
+ thepanel/data = detector_data
+ thepanel/min_fs = 0
+ thepanel/max_fs = 2067
+ thepanel/min_ss = 0
+ thepanel/max_ss = 2161
+ thepanel/corner_x = -1034
+ thepanel/corner_y = -1081
+ thepanel/fs = x
+ thepanel/ss = y
+
+
+Peak lists
+==========
+
+You can use all of the usual peak search methods for streaming data.
+
+In addition, you can use ``--peaks=msgpack`` to get the peak locations from
+the MsgPack data. In this case, the ``peak_list`` directive in the geometry
+file specifies the key for the peak information in the MsgPack map object.
+The peak information itself is expected to be a map object with three keys:
+``fs``, ``ss`` and ``intensity``. Each of these keys should correspond to an
+array containing (respectively) the fast scan and slow scan coordinates of each
+peak, and their intensities. Obviously, the three arrays must have equal sizes.
+
+Note that there is no way, in this structure, to communicate which detector
+panel contains a peak, in the case where different detector panels cover the
+same pixel ranges (in this case, the pixel data would from multiple data
+blocks). In practice, this means that the ``data`` directives for all panels
+need to be the same when using ``--peaks=msgpack``.