Joining Flow

Joining is a real-time customizable step of Promoted’s metrics process. The goal of joining records together is to produce proper training examples, which powers improvements to search and discovery. Promoted joins insertions (the decision to allocate a listing) to impressions (when the listing is visible in the viewport for long enough) to actions (the user engaging with, and ultimately purchasing, the listing). There are myriad user events to consider, so Promoted only joins events most likely to impact conversions. This process, called attribution, is described in the next section. Here is the joining flow:

1. Combine the delivery log
2. Perform joins. Produces JoinedImpressions and ActionPaths.
   1. Insertion → Impression
   2. Impression → Impression Actions
   3. Impression and Select Actions → Post-Select Actions
   4. Post-Select Actions → Post-Purchase Actions
3. Reduce redundant events
4. Apply attribution models to produce AttributedActions.
5. Enrich as necessary

Throughout the joins, Promoted also supports falling back and out of order events.

Joining Diagram

Joining Steps

Combined Delivery Log

When joining metrics, there are two systems to take into consideration: Promoted’s Delivery API and the client’s own SDK serving data. While Promoted’s API is usually the system that serves the recommendations, the request is served by the client’s own code base in certain situations, such as during initial integration or testing. Our SDKs are designed to timeout with a configurable threshold to fallback to your existing results so that the user experience is never interrupted. Promoted resolves this by producing a combined delivery log, which includes whichever system effectively served the request, in addition to data and metrics from the other system that can be used as machine learning features.

Foreign Keys and Falling Back

Joins are performed for impressions, select actions (e.g., click, add to cart), and post-select actions (e.g., purchase). Promoted supports multiple, prioritized foreign keys for event joining. This process is referred to as falling back, and it is beneficial to attributing credit (described below) when actions have different content types from insertions. On food delivery applications, for example, the insertions can be restaurants and items, but the purchase happens on items from those restaurants. In this case, Promoted would fall back to assign credit for touchpoints on the restaurants, not only the dish. This allows Promoted to handle attributing credit to both restaurant and item touchpoints.

Fanning Out

The best joining link is the impressionId or insertionId, but this may not be possible for all clients and all joins. Therefore, Promoted performs a "fan out" to attempt joins on multiple keys in addition to impressionId or insertionId. This combines the userId or anonUserId with the contentId or otherContentId, in addition to the time. Only the highest quality and relevant joins are selected.

Promoted's Approach

1. Promoted takes a configured list of join keys and fans out the joins.
2. For each leaf output, Promoted prioritizes the fanned-out joins and picks the highest one. E.g. prefer direct foreign keys. E.g. prefer matching touchpoints with navigate events.

The relevant data may be passed directly on the record or separately. There are different types of fall back joins, and they may happen across different IDs throughout the purchase process.

Example Joiners

Here are some example joiners for insertion to impression. Time is also used for all joiners except for insertionId.

1. impressionId
2. insertionId
3. (userId, contentId)
4. (anonUserId, contentId)
5. (userId, otherContentId)
6. (anonUserId, otherContentId)

Passing the impressionId or insertionId to Promoted is the best-case scenario and provides a direct join link, but other joiners are often used during initial integration to reduce the amount of required ID passing.

Output

The output of the joining steps are Touchpoints and ActionPaths.

Reducing Redundant Events

After the joining steps are completed and before attribution, Promoted deduplicates and reduces cases where multiple events should not be recorded or joined. Consider the following example:

1. User performs a search query and clicks on an item
   • The impression and action are joined with the corresponding insertion
2. User erroneously clicks the back button and generates a refresh, resulting in a new insertion on that item
3. User clicks on the same item again
   • A new impression and new action are logged and joined with the new insertion, not the original
4. User purchases that item
   • The purchase is joined with the new insertion

Here are the final results:

Promoted will collapse these redundant events with the goal of improving the quality of model training and ML features. Here is the resulting join:

Attribution

Touchpoints

Promoted assigns credit to relevant events that lead up to or impact the user’s decision to make a particular purchase. These events, or touchpoints, are the ways that the user may view or interact with the content. For example, a user may navigate to an ad on social media for a marketplace, download an app, and perform a search query for certain items. A few days later, the user may receive a notification about an item being on sale, and return to view and purchase the item. Promoted wants to specify to its internal model which of these touchpoints mattered most in this conversion flow.

Email Touchpoints

Clients can generate an email delivery log with email insertions for use as touchpoints. This improvement boosts model quality, including our ability to predict incrementally, which is a critical component of Promoted's performance ads systems.

Assigning Fractional Credit

The way that Promoted assigns credit to touchpoints is called the attribution model. Promoted prefers touchpoints that have a navigate (e.g. click). Not all touchpoints deserve the same credit for leading a user to the purchase, especially those falling out of a standard impression-to-click-to-purchase flow. The delay between touchpoint and purchase impacts the credit. The attribution windows need to be configured differently depending on the vertical (e.g. travel). Email touchpoints are supported. Hard-to-measure touchpoints, such TV ads, require special handling since they do not have Promoted events. A user may also interact with an item multiple times before purchasing, leading to uncertainty in which events should be assigned credit. Promoted is able to handle these edge cases where applicable and customize the model for each marketplace.

Promoted offers multiple ways of assigning credit, such as preferring the last touchpoint before the purchase, or giving even credit to all high-priority touchpoints that occurred before the purchase. In addition to navigates and clicks, Promoted places a high value on impressions, using the IAB definition that specifies that at least 50% of the item must be in view for at least one second.

Joining Window

Promoted assigns credit to touchpoints that occur within a custom delivery window of a specified duration of time. The length of the window may depend on the industry, company, or event itself; there may be different windows in joining insertions to impressions, and impressions to actions. Consider the following situations:

Common windows include one day, seven days, and 30 days. Certain events necessitate even longer windows, such as travel cancellations and repeat purchases that may happen months after the initial purchase. Longer join windows are less accurate and more expensive to operate.

Out of Order Support

Events and joins happen in real time. After the delivery log is sent to the client, impression data and other actions are sent back to Promoted, which then joins that data with the insertions.

Some customers perform server-side data batching, which interrupts the flow of real-time processing and may cause the data to be delivered out of order (e.g., receiving clicks after purchases). Although batching is strongly discouraged, Promoted is able to accommodate to ensure the data is ordered properly. Either streaming the data directly or performing short browser-side batching is recommended. Although batching may seem efficient, the lack of real-time data impacts the personalized predictions that power our recommendations.

Out-of-order support also protects against issues stemming from failures of remote procedure calls (RPC) on some devices and other errors.