Dynamical system

Lecture series: Data analysis
Thomas Kreuz, ISC, CNR
[email protected]
Lectures: Each Tuesday at 16:00
(First lecture: May 21, last lecture: June 25)
(Very preliminary) Schedule
Lecture 1: Example (Epilepsy & spike train synchrony),
Data acquisition, Dynamical systems
Lecture 2: Linear measures, Introduction to non-linear
Lecture 3: Non-linear measures
Lecture 4: Measures of continuous synchronization (EEG)
Lecture 5: Application to non-linear model systems and to
epileptic seizure prediction, Surrogates
Lecture 6: Measures of (multi-neuron) spike train synchrony
Last lecture
• Example: Epileptic seizure prediction
• Data acquisition
• Introduction to dynamical systems
Epileptic seizure prediction
Epilepsy results from abnormal, hypersynchronous
neuronal activity in the brain
Accessible brain time series:
EEG (standard) and neuronal spike trains (recent)
Does a pre-ictal state exist (ictus = seizure)?
Do characterizing measures allow a reliable detection of
this state?
Specific example for prediction of extreme events
Data acquisition
System / Object
Dynamical system
• Described by time-dependent states  ∈ ℛ 
• Evolution of state
- continuous (flow)
- discrete (map)

= (, , )
+∆ = ( , ∆, )
 Control parameter
,  can be both be linear or non-linear
Today’s lecture
Non-linear model systems
Linear measures
Introduction to non-linear dynamics
Non-linear measures
- Introduction to phase space reconstruction
- Lyapunov exponent
[Acknowledgement: K. Lehnertz, University of Bonn, Germany]
model systems
Non-linear model systems
Discrete maps
Continuous Flows
• Logistic map
• Rössler system
• Hénon map
• Lorenz system
Logistic map
r - Control parameter
• Model of population dynamics
• Classical example of how complex, chaotic behaviour can
arise from very simple non-linear dynamical equations
[R. M. May. Simple mathematical models with very complicated dynamics. Nature, 261:459, 1976]
Hénon map
• Introduced by Michel Hénon as a simplified model of the
Poincaré section of the Lorenz model
• One of the most studied examples of dynamical systems
that exhibit chaotic behavior
[M. Hénon. A two-dimensional mapping with a strange attractor. Commun. Math. Phys., 50:69, 1976]
Rössler system

= −( + )

= ( + )

=  + ( − )

 = 0.15, b = 0.2; c = 10
• designed in 1976, for purely theoretical reasons
• later found to be useful in modeling equilibrium in
chemical reactions
[O. E. Rössler. An equation for continuous chaos. Phys. Lett. A, 57:397, 1976]
Lorenz system

= σ y−x

= − −  + 

=  − 

 = 28, σ = 10; b = 8/3
• Developed in 1963 as a simplified mathematical model for
atmospheric convection
• Arise in simplified models for lasers, dynamos, electric
circuits, and chemical reactions
[E. N. Lorenz. Deterministic non-periodic flow. J. Atmos. Sci., 20:130, 1963]
Linear measures
Dynamic of system (and thus of any time series measured
from the system) is linear if:
H describes the dynamics and ,  two state vectors
  +  = () + ()
() = ()
a, b,  scalar
  +  = () + ()
Static measures
- Moments of amplitude distribution (1st – 4th)
Dynamic measures
- Autocorrelation
- Fourier spectrum
- Wavelet spectrum
Static measures
• Based on analysis of distributions (e.g. amplitudes)
• Do not contain any information about dynamics
• Example: Moments of a distribution
- First moment: Mean
- Second moment: Variance
- Third moment: Skewness
- Fourth moment: Kurtosis
First moment: Mean
Average of distribution
Second moment: Variance
Width of distribution
(Variability, dispersion)
Standard deviation
Third moment: Skewness
Degree of asymmetry of distribution
(relative to normal distribution)
< 0 - asymmetric, more negative tails
= 0 - symmetric
> 0 - asymmetric, more positive tails
Fourth moment: Kurtosis
Degree of flatness / steepness of distribution
(relative to normal distribution)
< 0 - platykurtic (flat)
= 0 - mesokurtic (normal)
> 0 - leptokurtic (peaked)
Dynamic measures
Physical phenomenon
Time series
Time domain
Frequency domain
x (t)
Frequency amplitude
Complex number  Phase
−∞ <  < ∞
[ Cross correlation
Fourier spectrum
Covariance ]
One signal  with  = 1, … , 
(Normalized to zero mean and unit variance)
Time domain: Dependence on time lag 
 1 N  '
n  n
( )   N   
n 1
C XX (  )
C XX ( 0 )  C XX ( )
 0
 0
Autocorrelation: Examples



Discrete Fourier transform
Fourier series (sines and cosines):
Fourier coefficients:
Fourier series (complex exponentials):
Fourier coefficients:
Power spectrum

 ()= lim
Wiener-Khinchin theorem:
Parseval’s theorem:
Overall power:
 () =
 () − 
() =
() =
Tapering: Window functions
Fourier transform assumes periodicity  Edge effect
Solution: Tapering (zeros at the edges)
EEG frequency bands
Description of brain rhythms
• Delta:
0.5 –
4 Hz
• Theta:
8 Hz
• Alpha:
– 12 Hz
• Beta:
– 30 Hz
• Gamma:
> 30 Hz
[Buzsáki. Rhythms of the brain. Oxford University Press, 2006]
Example: White noise
Example: Rössler system
Example: Lorenz system
Example: Hénon map
Example: Inter-ictal EEG
Example: Ictal EEG
Time-frequency representation
Wavelet analysis
Basis functions with finite support
Example: complex Morlet wavelet
 – scaling;  – shift / translation
(Mother wavelet:  = 1,  = 0)
Implementation via filter banks (cascaded lowpass & highpass):
 – lowpass
ℎ – highpass
Wavelet analysis: Example
- Localized in both
frequency and time
- Mother wavelet can
be selected according
to the feature of interest
Further applications:
- Filtering
- Denoising
- Compression
[Latka et al. Wavelet mapping of sleep splindles in epilepsy, JPP, 2005]
Introduction to
non-linear dynamics
Linear systems
Weak causality
identical causes have the same effect
(strong idealization, not realistic in experimental situations)
Strong causality
similar causes have similar effects
(includes weak causality
applicable to experimental situations, small deviations in
initial conditions; external disturbances)
Non-linear systems
Violation of strong causality
Similar causes can have different effects
Sensitive dependence on initial conditions
(Deterministic chaos)
Linearity / Non-linearity
Linear systems
- have simple solutions
- Changes of parameters and initial
conditions lead to proportional effects
Non-linear systems
- can have complicated solutions
- Changes of parameters and initial conditions lead to nonproportional effects
Non-linear systems are the rule, linear system is special case!
Phase space example: Pendulum
Position x(t)
Time series:
Velocity v(t)
State space:
Phase space example: Pendulum
Ideal world:
Real world:
Phase space
Phase space: space in which all possible states of a system
are represented, with each possible system
state corresponding to one unique point in a
d dimensional cartesian space
(d - number of system variables)
Pendulum: d = 2 (position, velocity)
Trajectory: time-ordered set of states of a dynamical
system, movement in phase space
(continuous for flows, discrete for maps)
Vector fields in phase space
Dynamical system described by time-dependent states

= ()
: ℛ   ℛ 
ℛ  – d-dimensional phase space
 – Vector field (assignment of a vector to
each point in a subset of Euclidean space)
- Speed and direction of a moving fluid
- Strength and direction of a magnetic force
Here: Flow in phase space
Initial condition ( )  Trajectory (t)
Rate of change of an infinitesimal volume around a given point
of a vector field:
Source: outgoing flow
( with    > 0, expansion)
incoming flow ( with    < 0, contraction)
System classification via divergence
Liouville’s theorem:
Temporal evolution of an infinitesimal volume:
  = 0
conservative (Hamiltonian) systems
  < 0
dissipative systems
  > 0
instable systems
Dynamical systems in the real world
In the real world internal and external friction leads to
Impossibility of perpetuum mobile
(without continuous driving / energy input, the motion stops)
When disturbed, a system, after some initial transients,
settles on its typical behavior (stationary dynamics)
Attractor: Part of the phase space of the dynamical system
corresponding to the typical behavior.
Subset X of phase space which satisfies three conditions:
• X is forward invariant under f:
If x is an element of X, then so is f(t,x) for all t > 0.
• There exists a neighborhood of X, called the basin of
attraction B(X), which consists of all points b that "enter
X in the limit t → ∞".
• There is no proper subset of X having the first two
Attractor classification
Fixed point: point that is mapped to itself
Limit cycle: periodic orbit of the system that is isolated (i.e.,
has its own basin of attraction)
Limit torus: quasi-periodic motion defined by n
incommensurate frequencies (n-torus)
Strange attractor: Attractor with a fractal structure
Introduction to
phase space reconstruction
Phase space reconstruction
Dynamical equations known (e.g. Lorenz, Rössler):
System variables span d-dimensional phase space
Real world: Information incomplete
Typical situation:
- Measurement of just one or a few system variables
- Dimension (number of system variables, degrees of
freedom) unknown
- Noise
- Limited recording time
- Limited precision
Reconstruction of phase space possible?
Taken’s embedding theorem
Trajectory () of a dynamical system in  - dimensional
phase space ℛ .
One observable measured via some measurement function :
  = (  ); M: ℛ  ℛ
It is possible to reconstruct a topologically equivalent attractor
via time delay embedding:
  = [  ,   −  ,   − 2 , … ,   − ( − 1) ]
 - time lag, delay;  – embedding dimension
[F. Takens. Detecting strange attractors in turbulence. Springer, Berlin, 1980]
Taken’s embedding theorem
Main idea: Inaccessible degrees of freedoms are coupled into
the observable variable  via the system dynamics.
Mathematical assumptions:
- Observation function and its derivative must be differentiable
- Derivative must be of full rank (no symmetries in
- Whitney’s theorem: Embedding dimension  ≥ 2 + 1
Some generalizations:
Embedding theorem by Sauer, Yorke, Casdagli
[Whitney. Differentiable manifolds. Ann Math,1936; Sauer et al. Embeddology. J Stat Phys, 1991.]
Topological equivalence
Example: White noise
Example: Rössler system
Example: Lorenz system
Example: Hénon map
Example: Inter-ictal EEG
Example: Ictal EEG
Real time series
Phase space reconstruction / Embedding:
First step for many non-linear measures
Choice of parameters:
Window length T
- Not too long (Stationarity, control parameters constant)
- Not too short (sufficient density of phase space required)
Embedding parameters
- Time delay τ
- Embedding dimension 
Influence of time delay
Selection of time delay  (given optimal embedding dimension )
 too small:
- Correlation in time dominate
- No visible structure
- Attractor not unfolded
 too large:
- Overlay of attractor regions that are rather separated in the
original attractor
- Attractor overfolded
Attractor unfolded
Influence of time delay
 too small
 too large
Criterion: Selection of time delay
Aim: Independence of successive values
First zero crossing of autocorrelation function
(only linear correlations)
First minimum of mutual information function
(also takes into account non-linear correlations)
[Mutual information: how much does knowledge of x 
tell you about x( + )]
Criterion: Selection of embedding dimension
Aim: Unfolding of attractor (no projections)
• Attractor dimension  known: Whitney’s theorem:  ≥ 2 + 1
• Attractor dimension  unknown (typical for real time series):
 Method of false nearest neighbors
 <  : Trajectory crossings, phase space neighbors close
 ≥  : Increase of distance between phase space neighbors
Procedure: - For given m count neighbors with distance  < 
- Check if count decreases for larger 
(if yes some were false nearest neighbors)
- Repeat until number of nearest neighbors constant
[Kennel & Abarbanel, Phys Rev A 1992]
Non-linear measures
Non-linear deterministic systems
• No analytic solution of non-linear differential equations
• Superposition of solutions not necessarily a solution
• Behavior of system qualitatively rich
e.g. change of dynamics in dependence of control
parameter (bifurcations)
• Sensitive dependence on initial conditions
Deterministic chaos
Bifurcation diagram: Logistic map
Bifurcation: Dynamic change in dependence of control parameter
Fixed point
Period doubling Chaos
Deterministic chaos
• Chaos (every-day use):
- State of disorder and irregularity
• Deterministic chaos
- irregular (non-periodic) evolution of state variables
- unpredictable (or only short-time predictability)
- described by deterministic state equations
(in contrast to stochastic systems)
- shows instabilities and recurrences
Deterministic chaos
Rather unpredictable
Strong causality
No strong
Characterization of non-linear systems
Linear meaures:
Static measures (e.g. moments of amplitude distribution):
- Some hints on non-linearity
- No information about dynamics
Dynamic measures (autocorrelation and Fourier spectrum)
Fast decay, no memory
Typically broadband
Distinction from noise?
Characterizition of a dynamic in phase space
Stability (sensitivity
to initial conditions)
(Information / Entropy)
Determinism /
Linearity /
Analysis of long-term behavior ( → ∞) of a dynamic system
Unlimited growth (unrealistic)
Limited dynamics
- Fixed point / Some kind of equilibrium
- periodic or quasi-periodic motion
- chaotic motion (expansion and folding)
How stable is the dynamics?
- when the control parameter changes
- when disturbed (push to neighboring points in phase space)
Stability of equilibrium points
Dynamical system described by time-dependent states

= (())
 0 = 0
: ℛ   ℛ 
Suppose  has an equilibrium  .
• The equilibrium of the above system is Lyapunov stable, if,
for every  > 0, there exists a δ = δ() > 0 such that if
0 −  < δ, then () −  < , for every  ≥ 0.
• It is asymptotically stable, if it is Lyapunov stable and if
there exists δ > 0 such that if 0 −  < δ, then lim () −
Stability of equilibrium points
Lyapunov stability: Tube of diameter 
Solutions starting "close enough" to the equilibrium (within a
distance from it) remain "close enough" forever (within a
distance from it). Must be true for any  that one may want to
Asymptotic stability:
Solutions that start close enough not only remain close
enough but also eventually converge to the equilibrium.
Exponential stability:
Solutions not only converge, but in fact converge faster than
or at least as fast as a particular known rate.
Divergence and convergence
Chaotic trajectories are Lyapunov-instable:
Neighboring trajectories expand
Such that their distance increases
exponentially (Expansion)
Expansion of trajectories to the
attractor limits is followed by a
decrease of distance (Folding).
 Sensitive dependence on initial conditions
Quantification: Lyapunov-exponent
Calculated via perturbation theory:
δ infintesimal perturbation in the initial conditions

( + δ)
= ( + δ)

=   + δ() + (2) Taylor series
Local linearization:

= δ()
δ() = δ(0) 
() - Jacobi-Matrix
 - Lyapunov exponent
In m-dimensional phase space:
 ,  = 1, … , 
(expansion rates for different dimensions)
Relation to divergence:
Dissipative system:
Largest Lyapunov exponent (LLE)  (often ):
Regular dynamics
Chaotic dynamics
Stochastic dynamics
Stable fixed point
Example: Logistic map
Bifurcation diagram
Fixed point
Period doubling
Largest Lyapunov exponent 
Dependence of the
control parameter
Dynamic characterization of attractors (Stability properties)
Classification of attractors via the signs of the Lyapunovspectrum
Average loss of information regarding the initial conditions
Average prediction time:
( – localization precision of initial condition,
j+ – index of last positive Lyapunov exponent)
Largest Lyapunov-exponent: Estimation
Reference trajectory:
Neighboring trajectory:
Initial distance:
Distance after T time steps:
1 (0)
 0 =  0 − 1 (0)
  =   − 1 ()
Λ(1) =
- Expansion factor:
Largest Lyapunov exponent (LLE):
() =

- New neighboring trajectory 2 () to   , 3 (2) to  2 etc.
- Calculate  times:   ,  = 1, … , 

[Wolf et al. Determining Lyapunov exponents from a time series, Physica D 1985]
Today’s lecture
Non-linear model systems
Linear measures
Introduction to non-linear dynamics
Non-linear measures
- Introduction to phase space reconstruction
- Lyapunov exponent
Next lecture
Non-linear measures
- Dimension
- Entropies
- Determinism
- Tests for Non-linearity, Time series surrogates

similar documents