Karabo: The European XFEL
software framework
Burkhard Heisen
2nd Collaboration Meeting of the European XFEL
April 24, 2013
Karabo: The European XFEL software framework
What I always start with…
A typical use case:
drive hardware and
complex experiments
monitor variables &
trigger alarms
data readout
online processing
quality monitoring
allow some
control & show
hardware status
Accelerator Undulator Beam Transport
show online data
whilst running
Tight integration of applications
Easy, efficient and comfortable usage of those
Burkhard Heisen (WP76)
storage of experiment
& control data
data access,
authorization etc.
setup computation &
show scientific results
processing pipelines
distributed and GPU
specific algorithms
(e.g. reconstruction)
Karabo: The European XFEL software framework
Outline – Follow the requirements
Proposal phase
Identify needs, Accept/Reject/Postpone
several month before
Preparation phase
Prepare configurations, custom developments
several month – few weeks before
Setup phase
Verify setup, find signal, alignments
few days – few hours before
Data taking phase
Clean up phase
Data analysis phase
Publication phase
Burkhard Heisen (WP76)
Data storage, sample switching/delivery
Remove custom h/w, bring s/w back to safe
Data analysis workflows, data management
Data cleanup
Karabo: The European XFEL software framework
Phase independent
 Distinction between different users with associated
access restrictions
 Solution
 Karabo is user centered, i.e. any access to
control, (meta-)data, or computing infrastructure
requires login.
 Karabo interacts with a central database
managing all user data.
 Access is controlled using a role-based system.
 Roles are assigned based on login context. The
context involves username, current time and
current location.
Central DB
 The same user for example may have more rights
during his beamtime and at a specific computer
than at other places and during different times.
Computes context based
access role
Burkhard Heisen (WP76)
Karabo: The European XFEL software framework
Preparation phase
 Setup of experiment configuration, scan macros, etc.
 Solution
 XFEL system operators can prepare configurations and scan macros
according to information of the proposal.
 Configurations and macros are managed within Karabo’s central database.
 The control system is fully scriptable. The default language is Python.
 Control commands can be issued sequentially, or concurrently (e.g. move
several axes at the same time)
 Scripting can be done procedurally (python program) or interactively (IPython)
 Being Python any complicated constructs can be coded (i.e. n-D scans with
conditionals, etc.)
 Scripting can be done within the GUI
 Main API: get, set, monitor, execute, instantiate, kill
Burkhard Heisen (WP76)
Karabo: The European XFEL software framework
Preparation phase (special needs)
 Implementation of new control devices and/or integration of new
algorithms to analysis pipeline
 Karabo is split into a core framework and extension plugins (devices).
 XFEL developers (or even the beamline users) can create new devices
for Karabo that can be injected into the runtime and will enable new
control or analysis capabilities.
 Several code templates exist for different categories of devices.
 Devices can be written in either C++ or pure Python (transparent to the
Karabo runtime).
 Devices impose a standard for self-description (properties, commands,
state-machine). GUI widgets and behavior such as auto-completion and
interactive scripting is automatically generated using this information.
 Devices can be deployed at any computer and are automatically made
available and ready to use for any control-clients.
Burkhard Heisen (WP76)
Karabo: The European XFEL software framework
Setup phase
 Verification of h/w and s/w functionality, alignments, signal search, etc.
 Solution
 Sanity functionality within Karabo ensures correct technical setup (hosts
available, correct plugins loaded etc.)
 XFEL beamline scientist can use the GUI for inspecting parameters,
trendlines, histograms and images in conjunction with prepared macros
to verify correct behavior
 The GUI system uses the Device self-description to provide basic access
to all features.
 Custom panel creation needs no coding but is a process that is similar to
the way you create a PowerPoint slide.
 Panels can be loaded and saved per user. Sharing with other users is
also possible.
Burkhard Heisen (WP76)
Karabo: The European XFEL software framework
Multi-purpose GUI system
Custom composition area
Logging / Scripting console
Kerstin Weger (WP76)
Karabo: The European XFEL software framework
Property/Command composition
Display widget (Trend-Line)
Editable widget
drag & drop
Display widget
Kerstin Weger (WP76)
Karabo: The European XFEL software framework
Property/Command composition
Display widget
(Image View)
Display widget
Kerstin Weger (WP76)
drag & drop
Karabo: The European XFEL software framework
Custom panels – PowerPoint like
Open/Save panel view
Cut, copy, paste, remove item
Group items
Change between
“Design/Live” mode
Bring to front/back
Insert text, line, rectangle, …
Kerstin Weger (WP76)
Rotate, scale item
Karabo: The European XFEL software framework
Data taking phase
 Data collection, modification of variables, switching samples, online data
 Solutions
 Karabo is tightly integrated into the data management system. Within the
GUI, experiment runs can be started and stopped.
 Whilst running, Karabo monitors and saves all (slow-)control data which can
later be associated to the experiment raw data.
 Raw data is streamed to the PC Layer, formatted and is saved in the online
storage as HDF5 files.
 Karabo scientific workflow system can directly be fed with the raw data
allowing highly parallel, pipelined processing whilst data taking.
 The GUI allows monitoring experiment statistics and data-analysis runs.
 Results of online data analysis can be used for tuning the experiment and
vetoing bad data (those won’t even travel to the offline storage)
Burkhard Heisen (WP76)
Karabo: The European XFEL software framework
Device (workflow) composition
Workflow node (device)
drag & drop
Draw connection
Kerstin Weger (WP76)
Karabo: The European XFEL software framework
Data analysis phase
 Pre-analysis on-site up to preliminary results, full analysis off-site
 Solutions
 The same workflow system as already available during data taking can be
used in offline mode. Workflows are now fed by data as available from the
offline storage (or - if still available - from the online storage).
 Data analysis for the user starts with already calibrated data
 Data calibration is also implemented as a workflow (but maintained by XFEL
and accessing calibration parameters from Karabo’s central resources)
 The analysis system will provide common image processing algorithms
making also use of GPU technology
 Community algorithms can easily be integrated to Karabo and shared
amongst scientists. XFEL.EU plans to build up a central resource for state of
the art analysis tools.
Burkhard Heisen (WP76)
Karabo: The European XFEL software framework
Processing workflows
IO whilst computing
Pixel parallel processing
(one GPU thread per pixel)
Notification about new data possible to obtain
Burkhard Heisen (WP76)
Karabo: The European XFEL software framework
Yes, it’s there!
Burkhard Heisen (WP76)
Karabo: The European XFEL software framework
Conclusions and timelines
Challenges are important to learn about detailed requirements
 pnCCD (combines control, DAQ, calibration and analysis)
 Beckhoff test stands
 Slice test (gives ideas about data throughput)
 2-tile LPD, later ¼ Mpx LPD (imaging detector)
 CrystFEL integration (data analysis workflows)
First release of Karabo on 30th July
 Will cover what was mentioned in this talk
 Need more time to place the available technical solutions in
larger contexts and workflows
Burkhard Heisen (WP76)
Karabo: The European XFEL software framework
Thank you for your kind attention.
Burkhard Heisen (WP76)

similar documents