Promoted Integration Walkthrough - Hipcamp

A step-by-step concrete example of how to integrate with Promoted featuring live customers and concrete, visual examples.

Promoted’s search and discovery is unparalleled. How does it work?

Below is a complete walkthrough of Promoted's systems, tracking the progress of a search query submitted on Hipcamp’s mobile app. Hipcamp is a Promoted customer that makes it easy to find and book campsites, cabins, and other lodging, strengthened by Promoted’s search and discovery optimization. This overview details the entire process of finding and booking lodging, from opening the Hipcamp app through completing a booking. Although these steps are not exhaustive — and some steps and features are either simplified or skipped (e.g., Hipcamp does not use native ads) — the overall process below provides an in-depth perspective useful to anyone considering Promoted's services. You can follow the descriptions along with an abbreviated technical diagram at the bottom of the page.

To continue without an example, you can jump to Getting Started.

Delivery

  1. To begin, imagine a user opening Hipcamp's mobile app. The app opens to the explore tab, populating the display with customized campsite recommendations. Implicitly, this means a search query is automatically being generated and answered using Promoted, based on the user’s location, past detinations, past engagement trends, and other available signals. The types of signals and availability of data depend on whether the user is new or returning, and Promoted optimizes both scenarios. More detail on this later.

    Opening the Hipcamp app and scrolling through the explore tab

    Opening the Hipcamp app and scrolling through the explore tab

  2. The user performs a custom search for campsites at a particular location and date, potentially using other filters like amenities and price preferences.

    Searching for campsites at Yosemite National Park in August for two guests

    Searching for campsites at Yosemite National Park in August for two guests

  3. The search query is sent to the controller layer in Hipcamp’s server. The server may use a search retrieval system, such as Elastic or Algolia, to fetch a large set of 500 to 1000 eligible candidates that could be shown to the user, based on the search query. The candidates have individual contentIDs and other metadata, such as a search relevancy score. All of that associated data is sent to Promoted.

    Candidate items are gathered by Hipcamp and sent to Promoted

    Hundreds of candidate items are gathered by Hipcamp and sent to Promoted

    Promoted sends and receives data using an efficient, secure AWS PrivateLink connection between virtual private clouds, with latency so low it's as if you were hosting the service yourself. Promoted's Delivery APIs uses the same lightweight Protobuf schemas for Request, Response, and Insertion, and the Metrics API logs the Protos as records. Although JSON is also supported, the Protobuf format is lighter and faster. See more details on Promoted's public schema repository on Github.


    message Insertion {
      ...
      // Optional.  Contains protobuf-serialized {Platform}Insertion.
      // Supports JSON-like struct or binary proto.
      Properties properties = 20;
    }
    
    // Platform-specific message.  This is the message serialized on `payload_bytes`.
    message MymarketInsertion {
      MymarketContent content = 1;
      ...
    }
    
    message Properties {
      oneof struct_field {
        // Optional.  Contains protobuf serialized bytes.
        bytes struct_bytes = 1;
        // Optional.  Can be converted to/from JSON.
        google.protobuf.Struct struct = 2;
      }
    
  4. Now it's time for Promoted to process and deliver results. There are multiple steps.

    Promoted communicates with Hipcamp's backend to process the data and provide rankings. See the [technical diagram at the bottom of the page](doc:promoted-integration-walkthrough#technical-diagram) for more detail and with labeled steps.

    Promoted communicates with Hipcamp's backend to process the data and provide rankings. See the technical diagram below for more detail and with labeled steps.

    1. Promoted denormalizes the data. Promoted fetches all information about the candidate items, including query information, viewing frequency, and other historical or semantic information, such as the number of images in the listing, the text description, and user engagement, including the click and cancelation history. Additionally, Promoted looks at the historical information of the user, including the entire user interaction sequence proceeding the search.
    2. Hipcamp also sends geographical features to Promoted, including the latitude and longitude of the listings, search query, and user, when applicable. Step 7 details the log containing this information. This data is sent via JSON using a string representation, and Promoted extracts the numerical information. Instead of a bounding box, Hipcamp provides a center and radius to that limits the retrieval listings to the circle.

      For records lacking exact coordinates, Promoted uses any available proxies for distance to calculate the approximate or relative location based on data from the viewport; there is high value in the relative positioning and proximity of listings even when exact measurements are unavailable. Promoted also handles the relative shift and revised endpoints from the original search to a new search. For example, if a user searches for a campsite in a particular town then manually shifts the map, Promoted is able to see that adjustment in the viewport and optimize recommendations accordingly.
    3. With this raw information, Promoted performs a feature transformation that includes text embeddings, image embeddings, and distribution features. An example of a distribution feature is “price as a percentile" — this allows for measurement of price relative to the other items being considered for allocation.
    4. Promoted scores the listings using L2 ranking, a technique involving deep neural networks and other advanced machine learning. This is used to generate a variety of scores, or probabilities, from the feature sets about each item, specific to this context and user. Scores may include p(click), the probability of a click occurring, p(booking | click), the probability of a booking being made given a click, and p(booking confirmed | click), the probability that the booking is successfully completed given a click. For other clients, Promoted also includes semantic relevance scores. These ML-generated scores go to the allocator.
    5. The allocator maximizes the long-term success of the end-user and the platform. In order to allocate, or rank, each item, Promoted combines the probabilities using a Blender allocation formulas, and assigns each item a single combined value called the Blender utility score. A formula could be as simple as p(click) * p(booking | click), or it may contain additional variables, weights, and filters. Formulas are refined over time.
      Hipcamp's Blender configuration is shown below, with some constants and variables obfuscated for confidentiality. The actual ranking is decided by the SORT_ALLOC() allocation rules. Each rule is followed in order until the LIMIT() is reached. This means that the first few listings in the ranking may follow a different rule from the following items. For example, Hipcamp uses a single challenger slot toward the beginning of the ranking order; a specific Blender rule is followed in order to prioritize the challenger more than the distance parameter. Besides the challenger and maximizer, the rules prioritize distance, ext/substring match, and click/purchase predictions.

    [  
    // Values are calculated to be used in allocation rules
    // Based on the features in each listing
    "clean_long_quote_match = IF(FEATURE(CLEAN_QUERY_TITLE_MATCH) > #, IF(FEATURE(CLEAN_QUERY_NUM_WORDS) > #, #, IF(FEATURE(QUERY_HAS_QUOTES) > 0, #, 0)), 0);",  
    "title_query_substring = IF(FEATURE(\"titleQueryWordsSubstring\") > #, 1, 0);", 
    "gb_and_near = IF(FEATURE(\"search.search_scope=geocenter\") == 1, IF(FEATURE(\"search.filter_country_code=XX\") == 1, IF(FEATURE(HIPCAMP_DISTANCE_SCORE_1) == 0, IF(FEATURE(HIPCAMP_DISTANCE_SCORE_2) \< #, 1, 0), 0), 0), 0);",  
    "challenger = IF(FEATURE(\"total_bookings\") \< #, #, #);",  
    "Hipcamp_Threshold = IF(FEATURE(\"search.filter_country_code=XX\") == 1, #, IF(FEATURE(\"search.filter_country_code=XX\") == 1, #, IF(FEATURE(\"search.filter_country_code=XX\") == 1, #, #)));",  
    "maximizer = IF(FEATURE(\"experiment(Hipcamp_Maximizer)=TREATMENT\") == 1, IF(FEATURE(\"Hipcamp_Value_1\") > Hipcamp_Threshold, 1, 0), 0);",  
    "score = FEATURE(PREDICTION_SCORE_1) \* FEATURE(PREDICTION_SCORE_2);",  
    
    // Allocation rules combine the above values into a single Blender score
    "SORT_ALLOC(gb_and_near, clean_long_quote_match, title_query_substring, score) LIMIT(#);",  
    "SORT_ALLOC(challenger, gb_and_near, clean_long_quote_match, title_query_substring, score) LIMIT(#);",  
    "SORT_ALLOC(gb_and_near, clean_long_quote_match, title_query_substring, score) LIMIT(#);",  
    "SORT_ALLOC(maximizer, gb_and_near, clean_long_quote_match, title_query_substring, score) LIMIT(#);",  
    "SORT_ALLOC(gb_and_near, clean_long_quote_match, title_query_substring, score);"  
    ]
    
    1. Once every item has a Blender utility score, we can begin the allocation. Each item is either inserted at a particular position in the ranking order, or it is not ranked at all. As only a small number of listings are shown to the user on the mobile app, most listings are not shown, even if they are given to Promoted. Note that Promoted supports real-time optimization, which means additional items will be dynamically ranked when the user refreshes or loads more items at the bottom.
    2. In the allocation process, Promoted allows for special rules that increase item diversity, filter results, and promote certain types of listings. This is incorporated in a Blender rule that contributes to the utility score. In this case, the third position is a “challenger slot”— a position in the ranking that has a particular rule. Hipcamp uses it to emphasize listings from new hosts on the platform.
    Hipcamp has a challenger slot that promotes listings from hosts who are new to the platform

    Hipcamp has a challenger slot that promotes listings from hosts who are new to the platform

  5. At the end of the allocation, only a portion of all items being considered will end up being inserted and visible to the user. Two things will happen after allocation is concluded.

    1. All items — including those considered but not inserted or ranked — are logged to Promoted’s server.
    2. The ranked items, or response insertions, are sent in a delivery response back to Hipcamp’s server. Promoted's entire processing and delivery sequence only takes about 10 to 50 ms for any request Hipcamp sends. For clients that have very large requests with complex machine learning, the process could take up to 100 ms.
  6. Hipcamp now send the ranked items to the user interface on the mobile device. The user, whose query was processed by Promoted faster than they blinked, can now scroll through the results. The next step details how this information is logged.

    Hipcamp displays the allocated items in the order of Promoted's ranking

    Hipcamp displays the allocated items in the order of Promoted's ranking.

  7. The delivery log, containing the request, response, and execution, is recorded. An Introspection report is created for the listing, which includes details from the delivery log in format easier to read and debug. This allows Hipcamp's engineers to gain a clear understanding of why these results showed up for the user's particular query.

    The structure of Promoted's delivery log is broken down into three parts below. The full delivery log combines all of these blocks, with some additional data, into one long log file. Click here for an example of a real, raw delivery log, hosted on Github. Note that the full log contains all data pertaining to the query — it is easily over a million characters long.

    1. The request block contains the request insertions sent by Hipcamp to Promoted, as well as all details about the user and environment.

      /*** REQUEST ***/
      
      {
        platform_id= ...,
        user_info={...},
        search_query=...,
                   
        // Additional fields for timing, client, and device information
            
        request_id = ...,
      	paging={...},
                
        insertion = {
          content_id=..., 
          position=...,
          properties={...},
      		retrieval_rank=..., 
      		retrieval_score=...
      	},
      
        properties = {
        	{
          	...
          	key=search, 
          	value={
          		struct_value={
          			fields=[
          				{key=search_source_lat, value=...}, 
      						{key=filter_query, value=...},
      						{key=filter_lng, value=...}, 
      						{key=search_source_lng, value=...}, 
      						{key=filter_country_code, value=...}, 
      						{key=filter_lat, value=...}, 
      						{key=search_source_to_center_distance_mi, value=...}, 
      						{key=filter_radius, value=...},
      						...
      						// Other search filters here
      			]}}
      },
      
    2. The response block contains the allocated response insertions that Promoted has optimized.

      /*** RESPONSE ***/
      {
        insertion=[{
          insertion_id=..., 
          content_id=..., 
          position=..., 
          properties={...},
          retrieval_rank=..., 
          retrieval_score=...
      	}], 
          paging_info=..., 
          request_id=...,
      },
      
    3. The execution block contains details about the ML model and its predictions, as well as the Blender rules and calculations resulting in the ranking order.

      /*** EXECUTION ***/
      {
        execution_server={
        	predictor_stage=..., // ML model details,
          blender_session_log=..., // Blender rules and scores
        }
      }
      

Congratulations! You've just seen the full life of a Hipcamp search query. But wait — there's much more to discuss about Promoted's backend and how Promoted collects and joins metrics.

Metrics

There are other ways to send data to Promoted outside of a search query. On the technical diagram, steps 1, 2, and 3 are marked in a blue circle.

  1. As the user is interacting with Hipcamp, their progress in booking a listing can be fully measured in discrete client events. This includes a visibility event (when the listing becomes visible), a booking, and everything in between, such as likes and impressions. This process is different on a mobile app from the web. Ultimately, Promoted wants to learn how to generate and optimize the client events (user responses) to achieve long-term goals. Hipcamp forwards these events to Promoted’s server.

    1. On the Hipcamp app, engagement events like taps, swipes, and other gestures are usually counted as "navigates," which is analogous to "clicks" on the web. Promoted provides iOS and Android SDKs for integration into mobile apps.

      Impressions and navigates are recorded as the user interacts with the app.

      Impressions and navigates are recorded as the user interacts with the app.

    2. On the Hipcamp website, engagement events are similar to those on the mobile app with some different metrics like clicks or mouse movement. Promoted provides a web SDK for integration.

      Impressions, clicks, and scrolls are recorded as the user interacts with the webpage.

      Impressions, clicks, and scrolls are recorded as the user interacts with the webpage.

  2. Promoted assembles the real-time data, tracking the path of the user's journey. The goal is to produce proper training examples to power improvements to search and discovery. Promoted joins insertions (the decision to allocate a listing) to impressions (when the listing is visible in the viewport for long enough) to actions (the user engaging with, and ultimately booking, the listing). There are myriad user events to consider, so Promoted only joins events most likely to impact conversions, picked out by the attribution model. This is the joining flow:

    1. Promoted combines the delivery log from Promoted’s Delivery API and Hipcamp's own system. While Promoted’s API is usually the system that serves the ranking requests, a request may be served by Hipcamp in certain situations. Promoted resolves this by producing a combined delivery log, which includes whichever system effectively served the request, in addition to data and metrics from the other system that can be used as ML features.

    2. Joins are performed on the data. Impressions are joined, followed by select actions (e.g., click, add to cart), and post-select actions (e.g., purchase). Promoted performs a "fan out" to attempt joins on multiple keys, and only the highest quality and relevant joins are selected. Joining by the impressionId or insertionId are preferred, as they provide a direct link.

      Promoted combines the delivery logs, performs the joins, and deduplicates the results before running the attribution model and sending the final output to the database.

      Promoted combines the delivery logs, performs the joins, and deduplicates the results before running the attribution model and sending the final output to the database.

    3. The outputs of the joining steps are touchpoints and ActionPaths. After reducing any duplicate or redundant events (e.g., the user reloads a page, generating potentially unnecessary duplicates), Promoted assigns fractional credit to user interaction events to denote their impact in the conversion flow. Important steps concerning this process are detailed below. For more detail, see Attribution and Joining Logic.

      1. These user interaction events, or touchpoints, may lead up to or impact the user’s decision to make a booking. Assigning credit to the touchpoints is called attribution. For example, suppose this user interacted with Hipcamp's social media ad, downloaded the app, and received push notifications before returning to search and book. Promoted wants to specify to its internal model which of these touchpoints mattered most in this conversion flow. Promoted assigns credit to touchpoints that occur within a specific, custom delivery window for Hipcamp.
      2. Promoted prefers touchpoints that have a navigate (on mobile) or click (on web), as well as impressions. Not all touchpoints deserve credit for leading a user to the purchase, especially those falling out of a standard impression-to-click-to-purchase flow. For example, a simple Hipcamp search query that the user performed months before booking would not be counted. Promoted offers multiple ways of assigning credit, such as preferring the last touchpoint before the purchase, or giving even credit to all high-priority touch points that occurred before the purchase.
      3. Different content types pose a challenge to properly attributing credit, so Promoted "falls back" and supports foreign keys for this event joining, shown in the previous diagram. For example, Hipcamp's insertion is on a campground, but the purchase is on a specific site within that campground — so Promoted uses Hipcamp's provided insertionId for a direct join link. For cases that lack the insertionId, Promoted is able to use the userIds and contentIds (in conjunction with time) to fan out a series of joins and choose the highest priority.
      4. When receiving data from Hipcamp, batching or unexpected delays may interrupt the flow of real-time processing and cause data to be delivered out of order. This could result in receiving a booking before the impression and clicks. Promoted accommodates to ensure the data is ordered properly.
    4. Promoted reduces cases where multiple redundant events should be treated as fewer (e.g., the user clicks the back button and re-performs clicks or navigates during a booking sequence). Promoted collapses these redundant events with the goal of improving the quality of model training and ML features.

  3. A machine learning example is created using the data. The labels in the model are output (i.e., impression, click, and booking), and the features are the input. The features include everything we know about the item, including the user and context details. This example goes into the machine learning pipeline, which runs every day with the latest data. If a new model outperforms the current production model, the new model is published to production. Relevant scores are generated, including p(CTR), the probability of a click given an ad impression, and p(CVR), the probability of any conversion (typically, a sale) given an ad click or impression.

Asynchronous Logging

Some of the item and user information that Promoted processes is asynchronous. This includes additional attributes about the listing that aren't found on the delivery log, such as image and text-based features — like the number of lodging sites a listing has available. This information is sent by Hipcamp to Promoted’s content management system in a simple HTTP request as a key-value pair, where the key is an ID, like the contentID or userID, and the value is some JSON document. This and other information is then hashed down, processed, and made available for delivery for any kind of allocation decision, including machine learning features, allocation rules, and logging.

User CMS Request

This is a real Hipcamp user CMS request, with identifiable and confidential information obfuscated.

{
    "user_id": "4c9w228w-d446-9fg6-a4fc-c6mx11bk2f41",
    "properties": {
        "completed_bookings_invited_to_last_30_days": 0,
        "completed_bookings_last_30_days": 1,
        "completed_bookings_total": 4,
        "first_booking_shelter_type": "structure",
        "future_bookings_count": 0,
        "hipcash_balance": 0,
        "is_host": false,
        "is_staff": false,
        "last_booking_date": "2024-04-15T12:30:48-06:00",
        "last_booking_site_type": ["tiny-home"],
        "last_stay_date": "2024-06-10",
        "preferred_site_type": "structure",
        "self_reported_camping_frequency": null,
        "self_reported_important_accommodations": ["tents", "cabin", "treehouse", "rv-or-trailer"],
        "self_reported_important_activities": ["hiking", "horseback-riding", "snow-sports", "wildlife-watching"],
        "self_reported_important_basic_amenities": [],
        "self_reported_important_core_amenities": ["pets", "toilet", "shower"],
        "self_reported_important_terrains": ["beach", "cave", "forest", "hot-spring"]
    },
    "last_updated": 1413768042238,
    "is_deleted": false,
    "is_backfill": false
}

Content CMS Request

This is a real Hipcamp content CMS request, with identifiable and confidential information obfuscated. Some key-value pairs are removed or shortened for brevity.

{
    "content_id": "1b4e2a5d-8c1f-4f7a-b9d7-23a91c6e8e54",
    "properties": {
        "acceptance_rate": 98,
        "accommodations": ["tents", "vehicles"],
        "acreage": 20,
        "any_instant_bookable": true,
        "availability_next_30_days": 27,
        "average_nightly_price_last_365": 91.5,
        "average_order_value_last_365": 128.8,
        "average_trip_length_last_365": 1,
        "booking_cancellation_rate_last_365": 0.042,
        "bookings_last_365": 75,
        "commitment_rate": 99.6,
        "elevation": "355",
        "favorites_count": 2,
        "full_name": "Sunny Grove Camp",
        "hashid": "u3mr7kp5",
        "highlight_slugs": ["pets_off_leash", "great_for_groups"],
        "highlights_full_text": ["pets_off_leash_Off-leash friendly_Pets can be off-leash at some sites.", "great_for_groups_Great for groups_Recent Hipcampers say this Hipcamp is great for larger groups."],
        "id": "0f8b647e-9a1d-4c3f-b7e1-c36f2a1472a5",
        "land_url": "https://www.hipcamp.com/link-to-campsite",
        "lat": 45.20384,
        "lng": -78.48932,
        "max_capacity": 15,
        "max_rv_length_meters": 10.5,
        "next_weekend_availability": 4,
        "overview": "Welcome large gatherings! From graduations and weddings to family reunions, our venue is ideal for your special occasion. Whether you're looking to rent the campground for a day or entire weekend, we're flexible. Note: Kindly plan your arrival and camp setup between 2 PM and 10 PM. Nestled on our serene 20-acre farm between Toledo and Fort Wayne, close to the Mountain View River, our campsite spans 12 acres of wooded terrain adorned with mature red oak, walnut, and elm trees. While we offer seclusion surrounded by natural landscapes, amenities like shops and gas are closeby. Explore live music and baseball games in Toledo, and explore downtown Toledo, Fort Wayne, and Auburn within a 15-minute drive. For local delights, Beanstalk Farms, featuring a charming antique market on Sundays (7 AM - 12 PM), is a pleasant walk down our rustic cobblestone road. Contact us today to plan your unforgettable event in the great outdoors with us!",
        "price_per_night": "55.0",
        "recommends_count": 32,
        "recommends_percentage": 100,
        "response_rate": 50,
        "review_one": "It's great",
        "review_two": "Wonderful place",
        "site_count": 6,
        "status": "live",
        "this_weekend_availability": 3,
        "total_bookings": 52,
        "weather_condition_code_3_days_from_now": "Thunderstorms",
        "weather_historical_avg_precipitation_amount_10_weeks_from_now": 0.37,
        "weather_max_uv_index_2_days_from_now": 8,
        "weather_moon_phase_2_days_from_now": "waxingGibbous",
        "weather_moonset_7_days_from_now": "2024-06-26 17:02:47",
        "weather_precipitation_type_5_days_from_now": "clear",
        "weather_snowfall_amount_5_days_from_now": 0,
        "weather_sunrise_civil_5_days_from_now": "2024-06-24 05:29:11",
        "weather_temperature_max_celsius_7_days_from_now": 30.5,
        "weekend_availability_next_30_days": 8,
        "weeknight_price_discount": true,
        "went_live_at": "2022-09-14T15:50:12-04:00"
      
        // Additional weather data and other metrics removed for brevity
    },
    "last_updated": 1718842267589,
    "is_deleted": false,
    "is_backfill": false
}

Technical Diagram

This is the condensed structure of a Delivery API request and response, showing the process of a search query. The numbers in yellow correspond to the steps in the Delivery section above, and the numbers in blue correspond to Metrics.

Abbreviated Delivery API showing the process of a query and Promoted's request and response. See [Delivery System Diagrams](doc:delivery-system-diagrams) for more detail.

Abbreviated Delivery API showing the process of a query and Promoted's request and response. See Delivery System Diagrams for even more detail and additional steps.


What’s Next