Promoted optimizes for long-term business outcomes like the gross services value (GSV) and gross merchandise value (GMV), not only proxy engagement metrics like clicks or views. This avoids the trap of having positive initial results but negative revenue impact over time. For example, recommending popular or previously purchased items may lead to a higher click-through rate, but not increase revenue. This feedback loop may recommend items customers would have purchased anyway, reducing the discovery potential of additional items.

Long-term value optimization is implemented in the Blender rules, such as optimizing the utility score definition and learning the optimal mix of diversity and promotion load to maximize future user retention and revenue. Promoted incorporates post-purchase and other events, such as cancellations, refunds, and repeated purchases into the models that output the probabilities behind the utility score.

Post-purchase modeling

Repeated purchases

Promoted joins repeated purchases for the same user and same content directly to past attributed purchases within a join window, on the highest priority join. See the documentation on Attribution and Joining for more details.

👍

Promoted joins to directly to the original purchase and avoids long chains of joins

Instead of joining directly to the insertions associated with the attributed purchases, Promoted joins to the original purchase. This avoids missing joins on older insertions that may have taken place outside of the join window.

In order to avoid a potentially infinite chain of joins, the repeated purchase is always joined to the oldest attributed purchase in the sequence. After a window of about 90 days (customized per client) from the original purchase, repeated purchases will begin a new sequence and no longer be joined to the original purchase. Of course, Promoted accommodates for clients with a longer recurring sequence of purchases.

Simple example

Below is a simple case of post-purchase events joining to a purchase, which is joined to a single insertion by a click.

At time t=0, no purchases are joined. At t=1, a p(purchase|click) training example can be produced, but the post-purchase example has not been observed long enough for use in training. At t=2, a post-purchase event with a value of $$$ is joined to the purchase, and the revenue post-purchase model is used in training. The time-since-insertion feature is set to 2. At t=3, the post-purchase model is trained again with the same example and label of $$$, but with the time-since-insertion feature set to 3.

Complex example

Below is a more complex example of multiple insertions, clicks, purchases, and post-purchase events for the same buyer, user, seller, and item. In this example, assume a minimum observation window of 7.

The original purchase at t=5 is joined to two clicks with 50/50 attribution credit. These clicks are joined to corresponding insertions A and B and their corresponding features to generate two training examples, each with a training weight of 0.5. At t=9, the purchase has been observed for a minimum of 7, so train on both A and B (even though B has only been observed for 5). As time progresses, adjust the label to be the sum of values attributed to both purchases.

Models

Promoted uses the following models to represent long-term optimization, including GSV models:

The Blender ranking combines these and other probabilities (e.g., p(click) and p(purchase|click)) into a single score. An example formula could be, p(click|impression) * p(purchase|click) * ((1 – p(bad|purchase)) * E(revenue|purchase). See Blender documentation for details on how this score is used.

Options for sending data

Post-purchase events are low volume and have long join windows. Promoted prefers clients send post-purchase events via the Metrics API with metadata like purchase values in the event.properties struct. Set the PurchaseID and any other important joining IDs in the properties struct. If it is too cumbersome to use the Metrics API, and constructing an event would be too artificial (for example, if total_spend is the result of a daily pipeline per purchase), then use the Content CMS and upload the latest state per PurchaseID.

Metrics API

Customers send post-purchase events of the appropriate type and metadata like “purchase value."

Advantages:

• Data are available for immediate joining and aggregation per user and per item in the standard way in the Metrics system
• Re-uses the existing Purchase API system for sending data

Disadvantages:

• Potentially more challenging to use for daily batches
• Not clear what type of an event a change in purchase state would be
• Properties struct is not as easily accessible

Content CMS

Customers or Promoted itself can load arbitrary JSON documents to the Content CMS with the contentID set to the purchaseID with an appropriately named source type. Promoted’s data pipelines join the latest state of the Content CMS to purchases in daily batches for use in training and data analytics. The PurchaseID must be set on the corresponding purchase event.

Advantages:

• The latest state can be easily written and read by both the Customer and Promoted
• Updates can be made in batches or per item as they occur
• Flexible re-use of an existing, easy-to-integrate CMS pattern used for Items

Disadvantages:

• The sequence of state changes must be tracked by the versioned feature store
• Not efficient for a large volume of changes or very fast changes
• Does not join in real-time
• More appropriate for “purchase state” versus user-triggered engagement actions
• Cumbersome to use in counter features