One of the major keys to success in life in general and in marketing especially is timing. If you can manage to approach your users with relevant marketing content exactly when they’re looking for it, you are most likely to receive a response, and most probably a beneficial one.
But that’s easier said than done. How can you reach your customer exactly at the right time? How can you match between who they are, what they’re doing right now, and the next best marketing action?
That’s where the Optimove Realtime Mechanism (ORTM) comes into play.
What is ORTM?
Optimove Realtime is an event-driven mechanism. It enables marketers to define a set of events performed by the user, which will activate an immediate or delayed marketing response in any of various channels.
How Our Event Driven Marketing Software Works
Preparation
Using the Optimove software, the marketer defines key events of interest (e.g., login, poker win, bingo win, specific item added to shopping cart). Once selected, Optimove supplies tags for these events, which the marketer adds to the website at the relevant web page or button.
Realtime Trigger Definition
The marketer defines a set of events as a realtime trigger, and then connects it to a target group with the requested execution channel and marketing content.
Let’s consider an example:
Marketers of a gaming web-site want to reach their users exactly when they are most likely to leave (after 3 consecutive wins.) They define a set of events (realtime trigger) comprised of three consecutive wins within 10 minutes. Users who experience this sequence will receive a pop-up that entitles them to a 50% boost if they continue to play.
This realtime trigger is connected to a target group or groups and is defined with a time-frame, an execution channel or channels, and matching marketing content. The set of trigger, target group, execution channel and marketing content makes up the realtime campaign.
Activation
Once the marketer completed the definition, Optimove takes charge. The relevant users are added to a daily refreshed database that contains all the potential users with their relevant triggers. Optimove then starts to listen to the events coming from the client’s website.
This is the challenging part of the mechanism, as millions of events arrive from users to the ORTM. Most of them are not relevant, but those which are should be processed within the shortest time possible to ensure that the end users that actually performed the entire sequence will receive the marketing content in real time.
Implementation of the Realtime Event Trigger
Let’s see how we implemented this system:
Gateway
Each request from the client’s website which was defined as an event reaches the Gateway. This module is responsible for transferring the request for processing, receiving the response and sending it back to the user’s browser with the relevant content as fast as possible.
Decider
The Decider is the mechanism’s “brain”. Every request coming from the Gateway is checked by the Decider, and if it matches an event and a relevant user a State Machine is created or updated. When an activating event arrives, the request is sent to the third module – the Engager.
Let’s return to the example from the previous section:
A user wins for the first time – at this stage, a request arrives from the website through the Gateway. The Decider then checks if the event and user are related to any trigger. If so, the Decider loads this user into an in-memory data structure (a “State Machine”) and indicates that this user is now in stage 1 out of 3 for this trigger, and has 10 minutes to complete the entire set.
The user wins the second time – Decider updates the State Machine to phase 2.
When the user wins for the third time within the 10-minute time-frame, Decider promotes it to the last phase and activates the trigger by sending a request with the relevant information to the Engager.
Engager
This module receives a request from the Decider with the UserID and TriggerID, and executes the campaign according to the trigger’s definition.
Going back to our example, the Engager will receive the activation request, find that the execution channel is a webpage pop-up and that the content is an HTML file called “wow”, and send a request with this data back to the Gateway and to the user’s browser.
Technological Challenges and Solutions
The ORTM presented us with several technological challenges. The mechanism should handle extremely high concurrency (millions of requests per second), determine which request is relevant, make immediate decisions for each relevant request, and activate a response for the user – all within the sub-second zone.
To meet these challenges, here are some of the guidelines and concepts we used:
Micro-Services
The ORTM contains three separate micro-services – Gateway, Decider and Engager – that communicate using HTTP requests and responses, with each of them responsible for a specific part of the job. This separation creates independency. Since there is no binding between the services, each can work separately on a different environment, enabling us to scale-out only the bottlenecks and as much as we need.
State Machine
The status of each user in each initiated trigger is kept as a single row that represent its current status only. The State Machine’s pros are:
Less data to store.
Data is faster to retrieve since there is only one relevant row for each user and trigger.
No need to check history with every access: the relevant and current data is kept.
Hybrid Data Storage Technologies
The general list of potentially active users and their active triggers is kept in a MongoDB collection, but once a user’s trigger is initiated (first event occurs) the user is loaded to the State Machine whose data is kept in Redis in-memory storage technology that enables an extremely quick access.
Node JS
The micro-services are implemented in NodeJS, an event-driven programming technology that suits the ORTM’s needs precisely. Each request is by definition handled separately, does not have to wait for events such as DB retrievals or API responses to complete, and thus enables other requests to complete faster.
“Slim” Requests
The data moving between the micro-services is minimal:
Gateway → Decider: ‘UserID’, ‘EventID’
Decider → Engager: ‘UserID’, ‘TriggerID’
Engager → Gateway: ‘HTML Content’
This method allows us to keep the network as free as possible. Since only the mandatory data is transferred, the serialization / deserialization is much faster.
A Priceless Result
The ORTM was built for speed, scale and precision. The architecture, data flow, processing and technologies were all put in place to allow the mechanism to cater to millions of concurrent users, while enabling interactions in blazing speeds of <300 milliseconds for popup ads, and even faster for all other channels. It’s a technology perfectly suited for the task of engaging users on the spot – a priceless feature for every marketer.
Yair Thurem holds a BA in Computer Science from Bar Ilan University and an MBA from the Hebrew University in Jerusalem. Yair has more than a decade of experience in the high-tech industry, and has been a team leader at Optimove for the past three years.
