MoEngage Streams

Introduction

Streams allow you to forward events such as user actions, campaign interaction events, and so on available in MoEngage to a predefined API endpoint in near real-time.

With MoEngage Streams, you can configure your API endpoint, define the events that you want to be forwarded, and once done, see a stream of data flowing to your system. You can then utilize the data to enrich your data warehouses and recommendation systems.

Popular Use Cases for MoEngage Streams

The following are a few popular use cases for Streams:

  • Enrich the central data warehouse with MoEngage events such as Notification Clicked Android, Email Clicked, SMS Clicked, and so on.
  • Forward the Conversion Events such as Purchase, Song Played, Bill Payment Done, and so on from MoEngage to the external Analytics tool.
  • Feed campaign interaction events from MoEngage to your recommendation system.
  • Export the notification interaction data from MoEngage to our machine learning (ML) system for optimizing the performance of our algorithms.
  • Send all user events present in MoEngage to an external own data lake.

Enable Streams for Your Account

Please reach out to your Customer Success Manager to get this enabled for your account. Note that Streams is only available as a part of a few pricing plans and you can include this separately when required.

Set Up Streams

Streams can export data to your API endpoint or one of the partners integrated with MoEngage.

Set Up Streams for Exporting Data to Your API Endpoint

To export data from MoEngage to your servers, you must raise a support ticket with the following information:

  1. API endpoint to send the data. It can be something like https://api.example.com/events.
  2. List of events that need to be sent to this API endpoint. A list of campaign events available for export is mentioned in the next section.
  3. List of user attributes and device attributes that need to be sent to the API endpoint with each event.
  4. If you need all campaign interaction events, we will export such events as mentioned in the next section.

You can use the following template to raise a Streams enablement request:

Support Ticket
Hey!

Please enable Streams for my Workspace.
      
Workspace Region: DC-01/DC-02/DC-03/DC-04
Workspace Name: MyWorkspace
Workspace ID: <my-workspace-id>

API Endpoint: https://www.example.com/api?url_param1=vaue1

Headers:
header1: value1
header2: value2

Authentication:
Method: No auth/Basic Auth
[Only for basic auth] Basic Auth username: "Username"
[Only for basic auth] Basic Auth password: "Password"

Events to export: All/List of events
[Optional] List of events: List of events to export

User properties to export: All/List of user properties
[Optional] List of user properties: List of user properties to export

Set Up Streams for Exporting Data to Partners

With Streams, you can export data directly to integrated partners so that you can enrich your marketing/analytics activities on these platforms. As of now, Streams can export data to the following partners:

Please follow partner-specific integration documents for setting this up.

Campaign Interaction Events

The following is a list of Campaign Interaction Events that MoEngage generates, which you will often need to be sent to your API endpoint:

Event Name Event Code Channel

Email Sent

MOE_EMAIL_SENT

Email

Email Deferred

MOE_EMAIL_DEFERRED

Email

Email Delivered

MOE_EMAIL_DELIVERED

Email

Email Dropped

MOE_EMAIL_DROP

Email

Email Bounced

MOE_EMAIL_HARD_BOUNCE

Email

Email Soft Bounced

MOE_EMAIL_SOFT_BOUNCE

Email

Email Opened

MOE_EMAIL_OPEN

Email

Email Clicked

MOE_EMAIL_CLICK

Email

Email Unsubscribed

MOE_EMAIL_UNSUBSCRIBE

Email

Email Spam Complained

MOE_EMAIL_SPAM

Email

SMS Sent

SMS_SENT

SMS

SMS Delivered

SMS_DELIVERED

SMS

Notification Received Android

NOTIFICATION_RECEIVED_MOE

Push

Notification Clicked Android

NOTIFICATION_CLICKED_MOE

Push

Notification Swiped Android

NOTIFICATION_CLEARED_MOE

Push

Notification Sent iOS

n_i_s

Push

Notification Clicked iOS

NOTIFICATION_CLICKED_IOS_MOE

Push

Notification Received Web

NOTIFICATION_RECEIVED_WEB_MOE

Push

Notification Clicked Web

NOTIFICATION_CLICKED_WEB_MOE

Push

Connector Sent

MOE_CONNECTOR_SENT

Connector

Card Sent

MOE_CARD_SENT

Cards

Card Delivered

MOE_CARD_DELIVERED

Cards

Card Viewed

MOE_CARD_VIEWED

Cards

Card Clicked

MOE_CARD_CLICKED

Cards

Mobile In-App Shown

MOE_IN_APP_SHOWN

Mobile In-Apps

Mobile In-App Clicked

MOE_IN_APP_CLICKED

Mobile In-Apps

Mobile In-App Closed

MOE_IN_APP_DISMISSED

Mobile In-Apps

On-site Message Shown

MOE_ONSITE_MESSAGE_SHOWN

On-site Messaging

On-site Message Clicked

MOE_ONSITE_MESSAGE_CLICKED

On-site Messaging

On-site Message Closed

MOE_ONSITE_MESSAGE_DISMISSED

On-site Messaging

User Entered Flow

USER_ENTERED_FLOW

Flows

User Exited Flow

USER_EXITED_FLOW

Flows

User Added to Control Group

MOE_CAMPAIGN_CONTROL_GROUP

All channels
  • With the events above, the following event attributes will be exported by default:
    • campaign_id
    • campaign_name
    • campaign_type
    • campaign_channel
  • The following event attributes are not supported for export in streams currently:
    • email_subject
    • email_click_url
    • inapp_widget_clicked
    • onsite_message_url_clicked

For more information on when the campaign events and attributes are tracked, refer to this article.

Authentication Methods

Streams supports the following types of authentications:

  • No auth
  • Basic Auth: Streams supports the standard Basic Auth implementation. You must provide us with the username and password while configuring Streams. All data related to authentication is always encrypted in our systems.

To support other authentication methods such as API Key, Bearer Token, and so on, you can communicate the same through the enablement ticket and provide us the Header key and static value to be passed. We will not be able to support dynamically refreshing API keys and tokens.

API Request Format

When Streams sends an event to your API endpoint, the request format will be as below: 

Headers:

"Content-Type" : "application/json"

Request Body:

JSON
{
    "app_name": "App Name",
    "source": "MOENGAGE",
    "moe_request_id": "moengage unique request id for each request",
    "events": [{
        "event_name": "Notification Received Android",
        "event_code": "NOTIFICATION_RECEIVED_MOE",
        "event_uuid": "moengage unique id for each event",
        "event_time": 1580967474,
        "event_type": "CAMPAIGN_EVENT",
        "event_source": "MOENGAGE",
        "push_id": "recipient device’s push token",
        "uid": "<MoEngage customer_id>",
        "event_attributes": {
            "campaign_id": "353df897hkbh67658",
            "campaign_name": "Name of the Campaign",
            "campaign_type": "Event Trigger",
            "campaign_channel": "Push"
        },
        "user_attributes": {
            "moengage_user_id": "moe_internal_user_id",
            "user_attr_1": "user_attr_val1",
            "user_attr_2": "user_attr_val2"
        },
        "device_attributes": {
            "moengage_device_id": "moe_internal_device_id",
            "device_attr_1": "device_attr_val1",
            "device_attr_2": "device_attr_val2"
        }
    }, {
        "event_name": "Email Sent",
        "event_code": "MOE_EMAIL_SENT",
        "event_uuid": "moengage unique id for each event",
        "event_time": 1580967474,
        "event_type": "CAMPAIGN_EVENT",
        "event_source": "MOENGAGE",
        "email_id": "recipient email_id",
        "uid": "<MoEngage customer_id>",
        "event_attributes": {
            "campaign_id": "353df897hkbh67658",
            "campaign_name": "Name of the Campaign",
            "campaign_type": "General",
            "campaign_channel": "Email"
        },
        "user_attributes": {
            "moengage_user_id": "moe_internal_user_id",
            "user_attr_1": "user_attr_val1",
            "user_attr_2": "user_attr_val2"
        },
        "device_attributes": {
            "moengage_device_id": "moe_internal_device_id",
            "device_attr_1": "device_attr_val1",
            "device_attr_2": "device_attr_val2"
        }
    }, {
        "event_name": "SMS Delivered",
        "event_code": "SMS_DELIVERED",
        "event_uuid": "moengage unique id for each event",
        "event_time": 1580967474,
        "event_type": "CAMPAIGN_EVENT",
        "event_source": "MOENGAGE",
        "mobile_number": "recipient mobile number",
        "uid": "<MoEngage customer_id>",
        "event_attributes": {
            "campaign_id": "353df897hkbh67658",
            "campaign_name": "Name of the Campaign",
            "campaign_type": "Event Trigger",
            "campaign_channel": "SMS"
        },
        "user_attributes": {
            "moengage_user_id": "moe_internal_user_id",
            "user_attr_1": "user_attr_val1",
            "user_attr_2": "user_attr_val2"
        },
        "device_attributes": {
            "moengage_device_id": "moe_internal_device_id",
            "device_attr_1": "device_attr_val1",
            "device_attr_2": "device_attr_val2"
        }
    }]
}

Streams Data Glossary

info

Information

For the full list of updated events and attributes, refer to Data Exports Glossary.

The keys in the API request and their description are:

Key Description
app_name Your app name in MoEngage.
source Source = MoEngage to identify the requests coming from MoEngage.
event_name Display the Name of the event. This is also the name for the event that you will see on the MoEngage dashboard.
event_code Event code as present in the MoEngage system. This is not visible on the MoEngage dashboard but is the raw event name that we save in the MoEngage database.
event_uuid This is the unique event identifier for each event added by MoEngage. You can use this for de-duplication.
event_time Time of event in UTC, format - epoch time in seconds. This is the time when the event happened.
event_type CAMPAIGN_EVENT or USER_ACTION_EVENT to differentiate between campaign events and user action.
event_source This will have value = MoEngage for you to identify that MoEngage is the source of these events.
push_id
Push token of the device to which the push notification was sent. This will only be available for Push-related events.
email_id
Email Id to which the email was sent. This will only be available for Email-related events.
mobile_number
Mobile Number to which the SMS was sent. This will only be available for SMS-related events.
uid
MoEngage customer_id to uniquely identify a user in MoEngage.
campaign_name
Campaign name in MoEngage.
campaign_id
Campaign ID in MoEngage.
event_attributes
Additional event attributes, dictionary.
campaign_channel
Campaign Type - Push / Email / SMS.
user_attributes
Additional user properties, dictionary.
moengage_user_id
MoEngage internal user ID.
device_attributes
Additional device attributes, dictionary.
campaign_type
Type of campaigns such as Periodic, Active, One-time.
variation_id
Present with event_type = CAMPAIGN_EVENT when the campaign has multiple variations.
locale_id
Present with event_type = CAMPAIGN_EVENT when the campaign has multiple locales.
locale_name
Present with event_type = CAMPAIGN_EVENT when the campaign has multiple locales.
parent_campaign_id
Present with event_type = CAMPAIGN_EVENT when the campaign has multiple locales.
parent_flow_id Present for campaigns that are created inside a MoEngage Flow campaign.
parent_flow_name
Present for campaigns that are created inside a MoEngage Flow campaign.

Whitelist IPs

If your endpoints are in a VPC/are not accessible publicly, you must whitelist the following MoEngage IPs depending on the region of your Workspace:

Data Center Name Dashboard URL IP to Whitelist
DC-01 https://dashboard-01.moengage.com 52.6.248.125
DC-02 https://dashboard-02.moengage.com 18.195.110.23
DC-03 https://dashboard-03.moengage.com 3.6.251.95
DC-04 https://dashboard-04.moengage.com 18.217.73.1

Limitations

Please be aware of the following constraints or limitations while integrating Streams with third-party apps.

  • We can send data to only one static endpoint where the access token or link has no expiry. We cannot integrate it on a dynamic endpoint where the access token is refreshed every few hours.
  • We cannot modify the body or key names while streaming the data from MoEngage.

Frequently Asked Questions

arrow_drop_down What is the throughput, average volume, and batch size that can be expected?

The throughput and average volume of each API request depends on the volume of selected events captured into MoEngage. The default batch size (number of events for each API request) is 100.

arrow_drop_down What is the retry mechanism of Streams?

The retrial mechanism allows MoEngage to hit your provided endpoints in multiple attempts. In case a batch of events fail (anything other than a "2XX" response from the endpoint is considered failed), the entire batch is retried after some time. However, there is a limit assigned to the retrial mechanism. In total, three retry attempts are made with an interval of 30 seconds, 60 seconds, and 120 seconds.

arrow_drop_down If I pause Streams, can I still get the data later?

If you pause Streams, data is not collected for exports and hence cannot be replayed at a later date.

Previous

Next

Was this article helpful?
2 out of 3 found this helpful

How can we improve this article?