The tools that provide teams within an organisation to identify system events and respond/react to them in as close as real-time manner (or near-real time).

An event is any change or action that occurs within the system

EDAs don’t ignore the fact that data needs to be stored, categorized, and analysed, but they emphasize the timely reaction to events, recognizing that the value of an event might diminish as time progresses. EDAs still value data but in a different way.

The model will shift from one where data is collected and stored, and interested parties decide to read what’s available

To a model where data is captured in the events that arise from actions on, or changes in the state of, the systems

We’ve already identified that tools like MOMs are ideal for this paradigm.

01 An EDA Example

Consider the situation where your bank wants to notify you in realtime that it suspects that there is fraud being committed against your account. The behaviour pattern we are looking to detect is a series of transfers from your account to other accounts outside of the bank. The transfers are small in their amounts of money initially as the criminals probe the account to see if there is any detection, if there is none, then the transfers continue and the amounts transferred increase. The transfers are done quickly, within minutes of one another, decreasing in time to seconds.

Because events can occur at any time, it’s important to ensure we respond to the events in a timely way and when it is right to do so. State machines can help us better under the problem and the timing.

It’s not going to work because the processing of the transfer will occur even if the fraud has been detected.

So we could migrate the model to

And our service architecture could look like this

02 Events

As would be expected events are central to this model, but what is an event in the model?

Each event typically comprises

a key that identifies the event or the entity it pertains to
a payload that holds the actual data the event carries
a timestamp indicating when the event occurred or was recorded
optionally some metadata about the event source
optionally a schema and its associated data

Events are source and destination agnostic. Any such information must be part of the payload.

Typically the sink that consumes an event is called a channel or a pipe.

As events traverse through a network of services, they may be transformed and used in several ways

Enrichment to make it more useful for downstream services
De-enrichment so clutter is removed from the payload to make it more useful to downstream services
Ingested so it is stored in a database
Aggregated to perform calculations
Analysed to look for patterns

03 Synchronicity and queuing

Messaging can be either synchronous or asynchronous

A synchronous send operation doesn't complete until the target of the message has both received it and finished processing it...

on completion, a reply will often be received
generally doesn't involve any middleware
also called "connection-oriented messaging"

Asynchronous messaging has been much more common

Messages are sent to a "queue" or "topic", from which messages are either pushed or pulled to receivers (consumers)
Requires message server middleware

04 Message format and QOS

Messaging systems have generally enabled both text and binary data to be carried

so XML and now JSON messaging have been relatively easy to introduce

Web Services founded on XML as the message format

Messaging servers offer various Quality of Service (QOS) benefits

reliable delivery (e.g. once and once only)
no need for the receiver to be available all the time
failure handling (automatic failure notification messages)
support for security and transactions

05 Event Type

Try NOT to use a single message structure for all message types

Group messages according to some classification model such as

Command messages, to invoke a procedure
Error messages, to indicate something has gone wrong
Reply messages, to indicate a response to the previous request
Data messages, to pass data
Event messages, to indicate a change of state

Separation simplifies the implementation of the consumers and producers

06 EDA Patterns

06-01 Channel Adapter

06-02 Pipes and Filters (sequential flow)

An old concept from the Mainframe and Unix world

A series of independent processing components chained to one another in series or parallel through pipes (channels)

In the following example, each filter can be executed in isolation

Filtering the data and enriching the data can be done in any order, but encryption must be done at the end

06-03 Modelling MOM Patterns (sequential flow)

UML Activity diagrams are useful for capturing the sequence and timing of MOM patterns

Each stream represents a unique thread of control

06-04 Processing Pipeline (service model 1)

If we execute each “filter data” on a separate service then we can increase the throughput of the model (consider 3 services)

The processing time is reduced from 420secs to 360secs (processing is reduced by a ~1/6)

(3 x 30) + (3 x 110) = 360secs. Why?
Answer in the next section

06-05 Processing Pipeline (service model 1) answer

Using three services to filter the data

So its actually 30 + (3 x 110) = 360.

06-06 Processing Pipeline (service model 2)

If we were able to execute each “filter data” on a separate service then we could increase the throughput of the model (consider 3 services)

The processing time is reduced from 420secs to 330secs (In this model, the time taken is reduced by ~1/3)

(3 x 110) = 330secs. Why?
Answer in the next section

DIS - Quick Share

04 Event Drive Architectures