Paging is when a previous search or ads ranking result is returned without modifications, typically by navigating backward in a multi-page or infinite scroll design. When paging is missing or broken, previous results are not preserved. Instead, users see new search or feed results when navigating backward, which leads to a frustrating user experience. Implementing preservation is done through paging of past results on back navigation, which aligns with the expectations most users have when navigating search and feed products.

Implementing Paging

Overview

1. Clients perform their own retrieval, by narrowing many candidates down to the top N (e.g., 300–500) to send to Promoted's SDK. These few hundred candidates are called Request Insertions.
2. The Delivery SDK or API will narrow down to a smaller page of items, which are the Response Insertions. They are specified using Request.paging.offset and Request.paging.size.

For this case:

• paging.offset = 0- Indicates which position to return as the first insertion in the Response.insertion list.
• paging.size = 50- The size of the response page size. This is similar to limit in select statements.
• retrievalInsertionOffset = 0 - For use if you want to send a different subset of insertions to Promoted's SDK. Described below.

Caching recently served response items

When the Delivery API receives a Request, the API looks in our page cache by a paging key. This allows Promoted to return already paged items back to the user if they request the same pages again.

Promoted creates the paging key from these Request properties:

• anonUserId
• clientInfo
• useCase
• searchQuery
• blenderConfig
• properties.

Promoted also has a setting for how long to keep recent pages. The default is around 30 minutes, but Promoted can customize paging behavior for your use case.

🚧

If you have Request.properties that will change across pages, contact Promoted to exclude them from the paging key.

The default behavior is to include all Request.properties in the paging key. If a Request.property changes across pages (e.g., the request timestamp), it'll change the paging key and disrupt paging. If you want exclude certain request properties from the paging key, please contact Promoted.

🚧

When a new page is requested from the Delivery API, Promoted will try to serve the next best recommendations to that page, regardless of the page number.

Here's an example:

1. The user starts their request on page 10. Promoted will serve the top results on page 10.
2. The user then hops back to page 0. Promoted will serve the next best items on page 0.

The results for each page are temporarily cached inside Promoted's servers.

If user tries to share a link to a page of results, the items will be different for each user that tries to load the list; each user gets their own personalized results.

Missing Request.insertion.content_ids on subsequent Requests

If, upon repeated user requests, previously allocated contentIds are missing in the new Request.insertion list, the Delivery API's behavior depends on its configuration. This situation arises due to common issues like inconsistent retrieval results or changes in item details.

Configurations and Behaviors

1. Default Setting (limitToRequestInsertions=false):

The Delivery API continues to return the previously allocated contentId in its original position, even if it's no longer in the current Request.insertion. This allows for expanding the list of contentIds across multiple requests. The cached allocation returns what the user saw, so it maintains consistency and allows for users to find the same items quickly, with the tradoff of lowering discovery potential.

Note: If Promoted handles retrieval or the marketplaces use third party ads, limitToRequestInsertions must be false.

Example:

• First Request: contentIds [A, B, C] for positions 0-2. Response: [B, C, A].
• Second Request: contentIds [C, D, E] for positions 0-2. Response: [B, C, A].

2. Alternative Setting (limitToRequestInsertions=true):

Any missing Request.insertion.contentIds are omitted from the response. This limits the content to only those items currently listed in the Request.insertion. This means positions previously allocated (and their corresponding content_ids) may not be on the request and will have a new content_id inserted into that position.

Example:

• First Request: contentIds [A, B, C] for positions 0-2. Response: [B, C, A].
• Second Request: contentIds [C, D, E] for positions 0-2. Response: [D, C, E]. Here, A and B are excluded as they are not in the current insertion list, while C retains its position and D and E fill the available slots.

Sending more than the top few hundred insertions

📘

Contact Promoted if you want to send additional Request Insertions to the Delivery API

Different windows of retrieval candidates can be sent to Promoted's SDK. The window is specified using the retrieval_insertion_offset parameter. The union config option needs to be enabled.

For this case:

• Request.paging.offset = 550- The starting index.
• Request.paging.size = 50- The response page size.
• retrievalInsertionOffset = 500 - The starting index of the window of retrieval insertions being passed into the SDK.

The Delivery API will cache both the earlier range of [0, 500) insertions and the additional [500, 999)insertions.

Linking Pages Together with Paging Keys

Promoted supports two methods of connecting Delivery Response pages into a paging sequence:

• Use an implicit paging key (default). Promoted infers the paging key for the same user, query, and search filters within a limited time. This is easier to integrate initially, but you have less control and may require maintenance to support new search filters or different types of A/B experimentation.
• Set an explicit paging key. You are responsible for all paging key preservation, passing, and logic. This option is the most reliable and has the most control, but it requires some overhead to handle the paging key. Contact Promoted for integration support on this feature.

Paging and Blender

Internally, Promoted implements paging like an additional retrieval system, plus a Blender allocation rule. If a position represented by (pagingKey, positionEnumeration) has an active paging cache entry, then Promoted will allocate the item as specified in the paging cache during Blender (more details). If that item cannot be allocated for any reason, then Promoted will allocate another available item. Blender is otherwise bypassed for Response Insertions allocated using Paging. Any rules that create attributes to be returned in the Response Insertion Properties like isBoosted are not re-created.

Paging and Response Insertion Properties

Response Insertion Properties from the original Response Insertion are returned on future paging cache copies. For example, if isBoosted was set in the original Response Insertion, it will also be set on all future paging cache copies of that Response Insertion. Additionally, the "previouslyAllocated" = true flag is set on all paging cache Response Insertion Properties.

Paging cache Response Insertions are assigned new Insertion IDs. They do not reuse the original Response Insertion ID.

For example, say contentID = ABC was delivered on page 1 with the isBoosted attribute set and a pass-through of the L2 pCTR model inference as "pCTR." The user navigates to page 2 and then back to page 1.

The resulting new paging cache Response Insertion Properties may look like:

Response.Insertion.Properties = {
  "previouslyAllocated": true,
  "isBoosted": true,
  "pCTR": 0.32
}

Paging and Ads

Ads may be a special case of paged delivery. Ads may have user-level and allocation-level frequency limits that can have edge cases when considering paging. Contact Promoted if you require special behavior for ads on paged results.