Promoted’s search and discovery is unparalleled. How does it work?

Below is a complete walkthrough of Promoted's systems, tracking the progress of a search query submitted on Hipcamp’s mobile app. Hipcamp is a Promoted customer that makes it easy for campers to find and book campsites, cabins, and other lodging, strengthened by Promoted’s search and discovery optimization. This overview details the entire process of finding and booking lodging, from opening the Hipcamp app through completing a booking. Although these steps are not exhaustive — and some steps and features are either simplified or skipped (e.g., Hipcamp does not use native ads) — the overall process below provides an in-depth perspective useful to anyone considering Promoted's services.

To continue without an example, you can jump to Getting Started.

Table of Contents

Data Overview

Before diving into a search query, let's address these questions: How will Hipcamp send data to Promoted? What does Promoted do with that data? There are three main types of data to send:

1. Retrieval information (Delivery API)

Promoted needs retrieval information from Hipcamp's search and feed system, as well as the real-time user and context information (including the search query), to optimally rank listings and achieve short-term and long-term business goals. This is sent through the Delivery API or Delivery SDKs.

2. User engagement metrics (Metrics API)

User engagement data includes clicks, navigates, impressions, purchases, and other ways the user engages with Hipcamp's app or website. This data can be sent in real time or in small batches. Promoted forms sequences of engagement data, representing the full path of a user's actions throughout time. The sequences are used in Promoted's model to predict clicks and conversions and optimize ranking. Engagement data is sent through the Metrics API. Promoted also provides Client Metrics SDKs and Server-side Metrics SDKs for ease of logging.

3. User and content updates (Content API and CMS)

Promoted's content management system (CMS) handles updates to the user's information (including both campers and hosts) and the details of the campsite catalog. The CMS section includes examples of User CMS and Content CMS requests, which can be sent in real time or batched. User and content data are used as features in Promoted's model and contribute to accurate ranking, so it's important to keep this information up to date as it inevitably changes over time. This data is sent via the Content API to the CMS.

Delivery

Hipcamp wants to maximize revenue and maintain a high-quality experience for hosts and campers. Promoted does both. This delivery section focuses on the campers, while going through the technical detail behind Promoted's systems. Each numbered step is marked in the technical diagram at the bottom of the page. Please follow along.

Search query and candidate results

1. To begin, imagine a user opening Hipcamp's mobile app. The app opens to the explore tab, populating the display with customized campsite recommendations. Implicitly, this means a search query is automatically being generated and answered using Promoted, based on the user’s location, past destinations, past engagement trends, and other available signals. Results are fully customized to the user and context for an improved experience. The types of signals and availability of data depend on whether the user is new or returning, and Promoted optimizes both scenarios. See Metrics for more detail.

   

2. The user performs a custom search for campsites at a particular location and date, potentially using other filters like amenities and price preferences.

   

3. The search query is sent to the controller layer in Hipcamp’s server. The server may use a search retrieval system, such as Elastic or Algolia, to fetch a large set of 500 to 1000 eligible candidates that could be shown to the user, based on the search query. Promoted's system also works well for a smaller number of candidates, such as in areas with fewer hosts available. Promoted conforms to Hipcamp's system and requirements — not the other way around. This prevents unexpected results and avoids integration or incompatibility issues. The candidates have individual contentIDs and other metadata, such as a search relevancy score. All of that associated data is sent to Promoted.

   

   Promoted sends and receives data using an efficient, secure AWS PrivateLink connection between virtual private clouds, with latency so low it's as if Hipcamp were hosting the service itself. See the SOC2 technical diagram for more detail.

   Promoted's Delivery API uses the lightweight Protobuf schema for the request, response, and insertions. Although JSON is also supported, the Protobuf format is lighter and faster. See more details on Promoted's public schema repository on Github.

   Protobuf Schema

   JSON

   message Insertion {
     ...
     // Optional.  Contains protobuf-serialized {Platform}Insertion.
     // Supports JSON-like struct or binary proto.
     Properties properties = 20;
   }
   
   // Platform-specific message.  This is the message serialized on `payload_bytes`.
   message MymarketInsertion {
     MymarketContent content = 1;
     ...
   }
   
   message Properties {
     oneof struct_field {
       // Optional.  Contains protobuf serialized bytes.
       bytes struct_bytes = 1;
       // Optional.  Can be converted to/from JSON.
       google.protobuf.Struct struct = 2;
     }
   

Promoted internal delivery flow

4. Now it's time for Promoted to process and deliver optimized results to the user. There are multiple steps, which correspond to the numbering on the technical diagram.

   

   1. Cleints have flexibility in how data is sent to Promoted, and Promoted handles all the processing on its end. First, Promoted denormalizes the data. Promoted fetches all information about the candidate items, including query information, viewing frequency, and other historical or semantic information, such as the number of images in the listing, the text description, and user engagement, including the click and cancelation history. Additionally, Promoted looks at the historical information of the user, including their entire sequence of interactions preceding the search. See ML features for more details on the data.

   2. Promoted fully supports geolocation-based search and feed. Hipcamp sends geographical features to Promoted, including the latitude and longitude of the listings, search query, and user, when applicable. Step 7 details the delivery log containing this information. This data is sent via JSON using a string representation, and Promoted extracts the numerical information. Instead of a bounding box, Hipcamp provides a geographical center and radius. See the Geolocation documentation for more details.

      👍

      Promoted handles records that are missing coordinates.

      For records lacking exact coordinates, Promoted uses proxies for distance to calculate the approximate or relative location based on data from the viewport.

      Promoted also handles the relative shift and revised endpoints from the original search to a new search. For example, suppose a camper searches for a campsite in a particular town, then manually shifts the map. Promoted sees that adjustment in the viewport and optimizes recommendations accordingly.

   3. Machine learning processing happens automatically on Promoted's end. With this raw information, Promoted performs a feature transformation that includes text embeddings, image embeddings, and distribution features. An example of a distribution feature is “price as a percentile" — this allows for measurement of price of each campsite relative to the other items being considered for allocation.

   4. Promoted scores the listings using L2 ranking, a technique involving deep neural networks and other advanced machine learning. This is used to generate a variety of scores, or probabilities, from the feature sets about each listing, specific to this context and user. The scores may include p(click), the probability of a click occurring; p(booking | click), the probability of a booking being made given a click; and p(booking confirmed | click), the probability that the booking is successfully completed given a click. Promoted can also provide semantic relevance scores. These ML-generated scores are a core part of the next step — the allocator.

   5. The allocator maximizes the long-term success of the hosts, campers, and platform, without sacrificing performance or quality. In order to allocate, or rank, each item, Promoted combines the probabilities into a single score, which is then fed into the final allocation rules. The formula that generates this score is refined over time, and could be as simple as p(click) * p(booking | click), or may contain additional variables, weights, and filters.
      
      Hipcamp's Blender configuration is shown below, with some constants and variables obfuscated for confidentiality. The actual ranking is decided by the SORT_ALLOC() allocation rules. Each rule is followed in order until the LIMIT() is reached. This means that the first few van listings ranked may follow a different rule from the following vans. In fact, Hipcamp uses a single challenger slot toward the beginning of the ranking order; a specific Blender rule is followed to prioritize the challenger more than the distance parameter. Besides the challenger and maximizer, the rules prioritize distance, ext/substring match, and click/purchase predictions.

      Blender Rules

      Blender rules

      [  
      // Values are calculated to be used in allocation rules
      // Based on the features in each listing
      "clean_long_quote_match = IF(FEATURE(CLEAN_QUERY_TITLE_MATCH) > #, IF(FEATURE(CLEAN_QUERY_NUM_WORDS) > #, #, IF(FEATURE(QUERY_HAS_QUOTES) > 0, #, 0)), 0);",  
      "title_query_substring = IF(FEATURE(\"titleQueryWordsSubstring\") > #, 1, 0);", 
      "gb_and_near = IF(FEATURE(\"search.search_scope=geocenter\") == 1, IF(FEATURE(\"search.filter_country_code=XX\") == 1, IF(FEATURE(HIPCAMP_DISTANCE_SCORE_1) == 0, IF(FEATURE(HIPCAMP_DISTANCE_SCORE_2) \< #, 1, 0), 0), 0), 0);",  
      "challenger = IF(FEATURE(\"total_bookings\") \< #, #, #);",  
      "Hipcamp_Threshold = IF(FEATURE(\"search.filter_country_code=XX\") == 1, #, IF(FEATURE(\"search.filter_country_code=XX\") == 1, #, IF(FEATURE(\"search.filter_country_code=XX\") == 1, #, #)));",  
      "maximizer = IF(FEATURE(\"experiment(Hipcamp_Maximizer)=TREATMENT\") == 1, IF(FEATURE(\"Hipcamp_Value_1\") > Hipcamp_Threshold, 1, 0), 0);",  
      "score = FEATURE(PREDICTION_SCORE_1) \* FEATURE(PREDICTION_SCORE_2);",  
      
      // Allocation rules combine the above values to sort the listings
      "SORT_ALLOC(gb_and_near, clean_long_quote_match, title_query_substring, score) LIMIT(#);",  
      "SORT_ALLOC(challenger, gb_and_near, clean_long_quote_match, title_query_substring, score) LIMIT(#);",  
      "SORT_ALLOC(gb_and_near, clean_long_quote_match, title_query_substring, score) LIMIT(#);",  
      "SORT_ALLOC(maximizer, gb_and_near, clean_long_quote_match, title_query_substring, score) LIMIT(#);",  
      "SORT_ALLOC(gb_and_near, clean_long_quote_match, title_query_substring, score);"  
      ]
      

      👍

      Native ads and promotions are fully supported.

      Promoted implements native ads through additional blender rules that combine the probabilities with the bid value and other ads metrics. Hipcamp does not use native ads, so they are not included in this example.

   6. Now we can begin the allocation. Each item is either inserted at a particular position in the ranking order, or it is not ranked at all. Each item is either inserted at a particular position in the ranking order, or it is not ranked at all. Only a small percentage of all items in retrieval will be ranked and shown to the user initially, and Promoted supplies high-quality results and dynamic ranking of items when the user refreshes the page or loads additional items. The list view also works seamlessly with the map view, and as long as geolocation data is being sent to Promoted, the list will update immediately when the user adjusts the map or filters.

   7. In the allocation process, Promoted allows for special rules that increase item diversity, filter results, and promote certain types of listings. In the Blender example above, the third position is a challenger slot — a position in the ranking that has a particular rule. Hipcamp uses it to emphasize listings from new hosts on the platform.

Receiving and displaying results

5. At the end of the allocation, only a portion of all items being considered will be inserted and visible to the user. Two things will happen after allocation is concluded.
   1. To improve the model for future ranking, all items are logged to Promoted’s server — including listings considered but not inserted.
   2. The ranked items, or response insertions, are sent in a delivery response back to Hipcamp’s server. Promoted's entire processing and delivery sequence only takes about 10 to 50 ms for any request Hipcamp sends. For clients with large requests and complex machine learning, the process could take up to 100 ms.
6. Hipcamp sends the ranked items to the user interface on the mobile device. The user, whose query was processed by Promoted faster than they blinked, can now scroll through the results. The next step details how this information is logged.

Delivery log

7. Promoted provides unparalleled insight into the delivery process, so engineers gain a clear understanding of why the results are ranked in that order. The delivery log is recorded on Promoted's servers and contains the request, response, and execution information in significant detail. The raw logs take time to inspect, so Promoted provides a corresponding Introspection report that boils down the log into an organized Google Sheet. Both the full log and the Introspection report are sent to a shared Slack channel, so Hipcamp's engineers and Promoted's engineers can discuss the reports collaboratively. The structure of Promoted's delivery log is broken down into three parts, shown below. The full delivery log combines all of these blocks, with some additional data, into one text file.

   📘

   The full log is a large file that contains all data pertaining to the query.

   It is easily over a million characters long. Promoted's Introspection report breaks down the log into a digestible format. Click here for an example of a real, raw delivery log on Github. The log is obfuscated for confidentiality.

   1. The request block contains the request insertions sent by Hipcamp to Promoted, as well as all details about the user and environment. Promoted does not force a particular schema, so clients may customize the structure and content.
      
      

      JSON

      /*** REQUEST ***/
      
      {
        platform_id= ...,
        user_info={
          log_user_id="7p3x441t-a675-2hg9-b3fd-d8pk88vj3e52", 
          is_internal_user=false,
          ignore_usage=false, 
          anon_user_id="4c9w228w-d446-9fg6-a4fc-c6mx11bk2f41", 
          retained_user_id="5r2y336s-e123-7jk8-c9ab-f7ml99xc5g73"
        },
        search_query="Yosemite",
      
        timing={
          event_api_timestamp=1718129155532, 
          log_timestamp=1718129155532, 
        }, 
      
        client_info={
          client_type=PLATFORM_SERVER, 
          traffic_type=PRODUCTION
        }, 
      
        device={
          device_type=DESKTOP, 
          ip_address="192.168.87.245", 
          browser={
            user_agent="Mozilla/5.0 AppleWebKit/537.36 Chrome/125.0.0.0 Safari/537.36",
          }
        }, 
      
        use_case=SEARCH, 
                   
        // Potentially additional key-value pairs
            
        request_id = "2m1n624t-a382-7kr5-c2ab-e3qr99zx4p88",
        paging={
          paging_id="3m5q839r-e754-4jh2-b8fe-f9pg33cl7k92", 
          size=42, 
          offset=0
        },
                
        insertion = {[
          {
            content_id="9r8s723u-f921-3lm7-a5de-h7st45dp9o23", 
            position=0,
            properties={...},
            retrieval_rank=0, 
            retrieval_score=0.836
          },
          {
            content_id="8w6x428y-h721-5rs8-f4de-k9ab88pq3g99", 
            position=0,
            properties={...},
            retrieval_rank=1, 
            retrieval_score=0.797
          },
          {
            content_id="5t2u521v-g643-8pq9-d3bc-j2xy77mn5o55", 
            position=0,
            properties={...},
            retrieval_rank=2, 
            retrieval_score=0.520
          },
          {...}, // Additional insertions skipped for brevity
        ]},
      
        properties = {
        	{
          	key=search, 
          	value={
              struct_value={
                fields=[
                  {key=filter_query_city, value="Yosemite"},
                  {key=filter_query_state, value="CA"},
                  {key=search_source_latitude, value=46.59384},
                  {key=search_source_longitude, value=-183.9834},
                  {key=filter_latitute, value=52.80410345}, 
                  {key=filter_longitude, value=-293.92398}, 
                  {key=filter_radius, value=150,
                  {key=search_source_to_center_distance_mi, value=105}, 
                  {...}
                  // Other search filters here
      			]}}
      },
      

      
      

   2. The response block contains the allocated response insertions that Promoted has optimized.
      
      

      JSON

      /*** RESPONSE ***/
      {
        insertion=[{
            insertion_id="6d5x729y-f567-3hg8-e5ab-i8pq44cl6o62", 
            content_id="3a2b613c-h432-6jk7-b2de-j7st99mn4r73", 
            position=0, 
            properties={...},
            retrieval_rank=..., 
            retrieval_score=...
          },
          { 
            insertion_id="2m8n430t-a391-7rs8-f4de-k9ab77pq3h99", 
            content_id="7p9q427r-g754-4tu9-d3bc-j2xy88mn5k55", 
            position=1, 
            properties={...},
            retrieval_rank=..., 
            retrieval_score=...
          },
          {...},
      ], 
          paging_info={
            paging_id = "384298492395032",
            cursor = 40
          }, 
          request_id="5t7u831v-g622-8pq9-d3bc-j2xy99mn5o88",
      },
      

      
      

   3. The execution block contains details about the ML model and its predictions, as well as the Blender rules and calculations resulting in the ranking order.
      
      

      JSON

      /*** EXECUTION ***/
      {
        execution_server=API,
        execution_insertion= {
          {...}                   // Many fields skipped for brevity
          predictor_stage = {
            models = [{
              model_id = daily_mtml_navigate_imp_seq_ea,
              model_type = NN_MTML,
              prediction_type = NAVIGATE_IMP,
              name = "hipcamp-mtml-navigate-imp-auto-seq-20240605-6729",
              feature_id = 30005,
              is_assigned = true
            },
          blender_session_log={...}, // Blender rules and scores
        }
      }
      

      
      

Congratulations! You've just seen the full life of a Hipcamp search query using Promoted. Read on — there's much more to discuss about Promoted's backend and how Promoted collects and joins metrics.

Metrics

Eventing

There are other ways to send data to Promoted outside of a search query. This data can be sent in real time or in small batches using the LogRequest or the Snowplow API for web. See Promoted's documentation on sending engagements from your server for examples. On the technical diagram, steps 1, 2, and 3 of eventing are marked in a blue circle.

1. As the user interacts with Hipcamp, their progress in booking a listing can be fully measured in discrete client events. This includes a visibility event (when the listing becomes visible), a booking, and everything in between, such as likes and impressions. This process is different on a mobile app from the web. Ultimately, Promoted wants to learn how to generate and optimize client events (user responses) to achieve long-term revenue and user retention goals.

   📘

   Promoted never collects metrics directly from the user

   Many events, including the purchase confirmation flow, occur on the server side. These events are not sent directly to Promoted but are first sent to Hipcamp, which then forwards these events to Promoted.

   1. On the Hipcamp app, engagement events like taps, swipes, and other gestures are usually counted as "navigates," which is analogous to "clicks" on the web. Promoted provides iOS and Android SDKs for integration into mobile apps.

      

   2. On Hipcamp's website, engagement events are similar to those on the mobile app with some different metrics like clicks or mouse movement. Promoted provides web SDKs for integration.

      

Joining

2. Promoted assembles the real-time data and maps the path of the user's journey. The goal is to produce proper training examples to power improvements to search and discovery. Promoted joins insertions (the decision to allocate a listing) to impressions (when the listing is visible in the viewport for long enough) to actions (the user engaging with and ultimately booking the camp). There are myriad user events to consider, so Promoted only joins events most likely to impact conversions, picked out by the attribution model. This is the joining flow:

   1. Promoted combines the delivery log from Promoted’s Delivery API and Hipcamp's system. While Promoted’s API is usually the system that serves the ranking requests, a request may be served by Hipcamp in certain situations, mainly in early integration or for experiments. Promoted resolves this by producing a combined delivery log, which includes whichever system effectively served the request, in addition to data and metrics from the other system that can be used as ML features. This way, no data is lost — Promoted extracts any potential improvements, whenever and wherever possible.

   2. Joins are performed on the data. Impressions are joined, followed by select actions (e.g., click), post-select actions (e.g., purchase), and post-purchase actions (e.g., cancelations). Promoted performs a "fan out" to attempt joins on multiple keys, and only the highest quality and relevant joins are selected. Joining by the impressionId or insertionId are preferred, as they provide a direct link.
      

      

   3. The outputs of the joining steps are touchpoints and ActionPaths. Promoted assigns fractional credit to user interaction events to denote their impact in the conversion flow. The next step explains this process. For more detail, see Attribution and Joining Logic.

   4. These user interaction events, or touchpoints, may lead up to or impact the user’s decision to make a booking. Assigning credit to the touchpoints is called attribution. For example, suppose this user interacted with Hipcamp's social media ad, downloaded the app, and received push notifications before returning to search and book. Promoted wants to specify to its internal model which of these touchpoints mattered most in this conversion flow. Promoted assigns credit to touchpoints that occur within a specific, custom delivery window for Hipcamp.

      1. Promoted prefers touchpoints that have a navigate (on mobile) or click (on web), as well as impressions. Not all touchpoints deserve credit for leading a user to the purchase, especially those falling out of a standard impression-to-click-to-purchase flow. For example, a simple Hipcamp search query that the user performed months before booking would not be counted. Promoted offers multiple ways of assigning credit, such as preferring the last touchpoint before the purchase, or giving even credit to all high-priority touch points that occurred before the purchase.
      2. Different content types pose a challenge to properly attributing credit, so Promoted "falls back" and supports foreign keys for this event joining, shown in the previous diagram. For example, Hipcamp's insertion is on a campground, but the purchase is on a specific site within that campground — so Promoted uses Hipcamp's provided insertionId for a direct join link. For cases that lack the insertionId, Promoted is able to use the userIds and contentIds (in conjunction with time) to fan out a series of joins and choose the highest priority.

         👍

         Promoted handles unexpected delays and duplicate events

         Delays: When receiving data from Hipcamp, batching or unexpected delays may interrupt the flow of real-time processing and cause data to be delivered out of order. This could result in receiving a booking before the impression and clicks. Promoted accommodates to ensure the data is ordered properly.

         Redundant events: Promoted reduces cases where multiple redundant events should be treated as fewer (e.g., the user clicks the back button and re-performs clicks or navigates during a booking sequence). Promoted collapses these redundant events with the goal of improving the quality of model training and ML features.

ML Features

3. With the goal of further improving results, a machine learning example is created using the data. The labels in the model are output (i.e., impression, click, and booking), and the features are the input. The features include everything we know about the item, including the user and context details, in addition to user sequences, counts, and other metrics defined below.

   1. Promoted generates a user sequence for each user, which represents all of the user’s engagement events, like impressions and navigates, throughout time. The sequence is accumulated in real time and then transformed and used as a feature in our model. This functionality is similar to AWS personalize, but is built as a native part of our architecture for increased customization, efficiency, and performance. See our diagram on ML Feature Types for more details. Below is a shortened example of a real user sequence, with some information obfuscated. In this sequence, the user performed initial navigates as well as a checkout, then returned to checkout and purchase on a different item a few days later.

      
      

      action type	relative time	content name	response position	request retrieval rank	timestamp	content id
      NAVIGATE	9 days, 19:58:35	The Cove On Sunny Island	3	46	2024-06-19 00:07:52	9a2d1f80-6433-4f99-bd6f-67a3f9b3c46d
      NAVIGATE	9 days, 19:55:17	Triple Light Lake Front Camping	0	23	2024-06-19 00:11:10	eab0f0c4-9a4d-46b3-a62c-e85d8c9424e6
      NAVIGATE	8 days, 5:56:32	The Cove On Sunny Island			2024-06-20 14:09:55	9a2d1f80-6433-4f99-bd6f-67a3f9b3c46d
      CHECKOUT	8 days, 5:54:32	The Cove On Sunny Island			2024-06-20 14:11:55	9a2d1f80-6433-4f99-bd6f-67a3f9b3c46d
      NAVIGATE	8 days, 5:53:41	Oak Grove Ranch	4	54	2024-06-20 14:12:46	a0b6e1c7-cc67-4239-a5e8-6a4d3b3f0d42
      NAVIGATE	6 days, 21:24:58	Tri-spring Purple Orchard	1	101	2024-06-21 22:41:29	1a5c3d8f-63f5-4c24-9b3a-1d4c1e60e2ee
      CHECKOUT	6 days, 20:41:57	Tri-spring Purple Orchard	1	101	2024-06-21 23:24:30	1a5c3d8f-63f5-4c24-9b3a-1d4c1e60e2ee
      PURCHASE	6 days, 20:36:19	Tri-spring Purple Orchard	1	101	2024-06-21 23:30:08	1a5c3d8f-63f5-4c24-9b3a-1d4c1e60e2ee

      
      

   2. Real-time counts of user, item, and query data (or some intersection of these) over specified durations are also used as features. These counts are also used to compute rates and ratios. For example, actual features include the total number of “favorites” by the user, the ratio of 7-day to 30-day impressions per item, or a 7-day total of clicks per item on the same device type (e.g., mobile) as the user. The introspection report and feature importance report lists these features.

   3. Before being used as input, features are automatically transformed either for necessity (if they cannot be used directly in the model) or for increased performance. Strings are transformed into words or text embeddings, and numbers are used both without transformation and as ordinals. Promoted also has transformers for distance, time (including the time-to-now), radians, and many others.

   4. This example goes into the machine learning pipeline, which runs every day with the latest data. If a new model outperforms the current production model, the new model is published to production. This is immediately communicated in a shared Slack channel, so Hipcamp and Promoted are always in sync.

Content Management System

When Hipcamp or a Hipcamp host updates a listing, that information needs to be communicated to Promoted. Keeping Promoted up to date means that users will always have the best results. These updates may include image or text-based features, like an update to the daily availability of a campsite. Hipcamp sends this information to Promoted's content management system via the Content API, using a simple HTTP request as a key-value pair, where the key is an ID, like the contentID or userID, and the value is some JSON document. This information is then hashed down, processed, and made available for delivery, machine learning features, allocation rules, and logging. It is also possible to attach the updates to delivery requests instead of using the Content API — while this is less efficient, Promoted provides flexibility.

These are a few ways of sending the data:

• Database hook. When a change is made to one of Hipcamp's database records, a function is triggered that calls the Content API to communicate the change. This is common for changes to catalog items, like details of a listing being updated.
• Daily batch update. Hipcamp's larger analytics changes use a daily update containing a large batch of data. For example, Hipcamp uses this for batch updates in weather predictions.

User CMS Request

This is a real Hipcamp user CMS request, with identifiable and confidential information obfuscated.

{
    "user_id": "4c9w228w-d446-9fg6-a4fc-c6mx11bk2f41",
    "properties": {
        "completed_bookings_invited_to_last_30_days": 0,
        "completed_bookings_last_30_days": 1,
        "completed_bookings_total": 4,
        "first_booking_shelter_type": "structure",
        "future_bookings_count": 0,
        "hipcash_balance": 0,
        "is_host": false,
        "is_staff": false,
        "last_booking_date": "2024-04-15T12:30:48-06:00",
        "last_booking_site_type": ["tiny-home"],
        "last_stay_date": "2024-06-10",
        "preferred_site_type": "structure",
        "self_reported_camping_frequency": null,
        "self_reported_important_accommodations": ["tents", "cabin", "treehouse", "rv-or-trailer"],
        "self_reported_important_activities": ["hiking", "horseback-riding", "snow-sports", "wildlife-watching"],
        "self_reported_important_basic_amenities": [],
        "self_reported_important_core_amenities": ["pets", "toilet", "shower"],
        "self_reported_important_terrains": ["beach", "cave", "forest", "hot-spring"]
    },
    "last_updated": 1413768042238,
    "is_deleted": false,
    "is_backfill": false
}

Content CMS Request

This is a real Hipcamp content CMS request, with identifiable and confidential information obfuscated. Some key-value pairs are removed or shortened for brevity.

{
    "content_id": "1b4e2a5d-8c1f-4f7a-b9d7-23a91c6e8e54",
    "properties": {
        "acceptance_rate": 98,
        "accommodations": ["tents", "vehicles"],
        "acreage": 20,
        "any_instant_bookable": true,
        "availability_next_30_days": 27,
        "average_nightly_price_last_365": 91.5,
        "average_order_value_last_365": 128.8,
        "average_trip_length_last_365": 1,
        "booking_cancellation_rate_last_365": 0.042,
        "bookings_last_365": 75,
        "commitment_rate": 99.6,
        "elevation": "355",
        "favorites_count": 2,
        "full_name": "Sunny Grove Camp",
        "hashid": "u3mr7kp5",
        "highlight_slugs": ["pets_off_leash", "great_for_groups"],
        "highlights_full_text": ["pets_off_leash_Off-leash friendly_Pets can be off-leash at some sites.", "great_for_groups_Great for groups_Recent Hipcampers say this Hipcamp is great for larger groups."],
        "id": "0f8b647e-9a1d-4c3f-b7e1-c36f2a1472a5",
        "land_url": "https://www.hipcamp.com/link-to-campsite",
        "lat": 45.20384,
        "lng": -78.48932,
        "max_capacity": 15,
        "max_rv_length_meters": 10.5,
        "next_weekend_availability": 4,
        "overview": "Welcome large gatherings! From graduations and weddings to family reunions, our venue is ideal for your special occasion. Whether you're looking to rent the campground for a day or entire weekend, we're flexible. Note: Kindly plan your arrival and camp setup between 2 PM and 10 PM. Nestled on our serene 20-acre farm between Toledo and Fort Wayne, close to the Mountain View River, our campsite spans 12 acres of wooded terrain adorned with mature red oak, walnut, and elm trees. While we offer seclusion surrounded by natural landscapes, amenities like shops and gas are closeby. Explore live music and baseball games in Toledo, and explore downtown Toledo, Fort Wayne, and Auburn within a 15-minute drive. For local delights, Beanstalk Farms, featuring a charming antique market on Sundays (7 AM - 12 PM), is a pleasant walk down our rustic cobblestone road. Contact us today to plan your unforgettable event in the great outdoors with us!",
        "price_per_night": "55.0",
        "recommends_count": 32,
        "recommends_percentage": 100,
        "response_rate": 50,
        "review_one": "It's great",
        "review_two": "Wonderful place",
        "site_count": 6,
        "status": "live",
        "this_weekend_availability": 3,
        "total_bookings": 52,
        "weather_condition_code_3_days_from_now": "Thunderstorms",
        "weather_historical_avg_precipitation_amount_10_weeks_from_now": 0.37,
        "weather_max_uv_index_2_days_from_now": 8,
        "weather_moon_phase_2_days_from_now": "waxingGibbous",
        "weather_moonset_7_days_from_now": "2024-06-26 17:02:47",
        "weather_precipitation_type_5_days_from_now": "clear",
        "weather_snowfall_amount_5_days_from_now": 0,
        "weather_sunrise_civil_5_days_from_now": "2024-06-24 05:29:11",
        "weather_temperature_max_celsius_7_days_from_now": 30.5,
        "weekend_availability_next_30_days": 8,
        "weeknight_price_discount": true,
        "went_live_at": "2022-09-14T15:50:12-04:00"
      
        // Additional weather data and other metrics removed for brevity
    },
    "last_updated": 1718842267589,
    "is_deleted": false,
    "is_backfill": false
}

Testing and Experimentation

A/B Testing

Hipcamp needs to know that Promoted's search and feed optimizations are actually increasing revenue per user. Promoted provides a robust testing infrastructure and supports multiple types of A/B testing, both before and after launch. Hipcamp can either run its own experiments and share the assignments with Promoted (see Cohorts) or request Promoted manage the experiment assignments. This includes overlapping experiments, where Hipcamp tests Promoted while also testing other, non-Promoted changes. Hipcamp also performs holdback testing (also called a "backtest") to verify that the expected impact is still being realized. Lastly, Promoted is able to limit A/B testing to a subset of users based on their geographic location or surface for insight into specific groups or features. See Promoted’s A/B testing documentation for specifics.

Cohorts

During experiments, users are split into different groups in order to compare results. Campers may be placed into the TREATMENT arm or CONROL arm, and some experiments may have additional TREATMENT arms.

Hipcamp can choose to exclude some users from the experiment. Excluding some users may be necessary to test a specific feature or audience, but it also means less data is available for analysis. By default, every user is excluded unless they are added to an experiment arm in one of two ways:

1. Clients may batch upload the assignments to Promoted. This is useful if a client prefers to pre-generate lists of users for each group.
2. Clients can use a triggering event, called a cohort assignment event, that assigns the users into either TREATMENT or CONTROL. Specifying a cohort assignment (e.g., a user logging in, submitting a search query, etc.) automates the grouping process and ensures certain thresholds are met before assignment.

The following table, visible in Promoted's introspection report, contains details about the cohort and experiment the user belongs to on Hipcamp. Cleints have full access to these experiment details through Introspection and the delivery log.

Technical Diagram

This is the condensed structure of a Hipcamp Delivery API request and response, showing the process of a search query. The numbers in yellow correspond to the steps in the Delivery section, and the numbers in blue correspond to Metrics.

Back to top of page