Data Partner Integration Guide for Audiences

Document created by catherine.racette on Nov 1, 2017Last modified by catherine.racette on Nov 6, 2017
Version 3Show Document
  • View in full screen mode

Introduction

A data partner can integrate with the PubMatic Audience platform using one of the
following methods:

  • Server-To-Server (S2S) Data Transfer
  • Browser-side Real-time (HTTP 302 Redirect) Data Transfer

Server-To-Server (S2S) Data Transfer

The server-to-server data transfer mechanism is a highly-efficient way to pass data to PubMatic, as the actual data transfer happens directly between servers. The server-to-server data transfer mechanism can be executed by performing the following steps in the specified sequence:

  • Phase 1 - UID Sync-up
  • Phase 2 - Actual Data Transfer

The following figure depicts the process of the server-to-server data transfer mechanism between a data provider and the PubMatic system.

Phase 1- Unique Identifier (UID) Sync-up

This phase contains the following steps:

  1. A user visits a website using a browser window.
  2. If the data partner is associated with this website, this visitor (or browser) is assigned a unique identifier (UID) by the data partner. Similarly, since the publisher is also associated with PubMatic, this visitor is also mapped with another UID by the PubMatic system. Since there are two different UIDs for the same visitor, they need to be mapped with each other. (See next step for details.)
  3. If the data partner discovers the visitor first, then it needs to call the PubMatic UID sync Data API URL, and then the PubMatic system will pass the PubMatic-specific UID for that visitor to the data partner. However, if the PubMatic system discovers the visitor first, the it sends the PubMatic-specific UID for that visitor directly to the data partner. This syncing must be performed in real time.

Note:

  • The data partner needs to maintain the mapping between its UID and the PubMatic-specific UID for all visitors.
  • The data partner needs to provide PubMatic with a pixel similar to <img src="http://partner.com/xxxx?id=UID" width=1 height=1>, which will be invoked by the PubMatic system to pass the UID of the visitor.
  • The data partner also needs to provide the segment ID-to-segment name mapping to PubMatic only once at the start of the integration.

Phase 2 - Data Transfer

This phase contains the following steps:

  1. After the UID sync-up between the data partner and PubMatic UIDs, the PubMatic system should receive the data parter's audience taxonomy. This is a one-time process of importing this data into the PubMatic system.
  2. PubMatic shares a Data Ingestion API URL with the data partner. The data partner then needs to pass the audience segment data to this API in batches. This API typically requires the PubMatic-specific UID and the list of audience segment IDs as input.
  3. For users whose audience profile has changed since the last data transfer, the data partner needs to transfer the audience profile information to the PubMatic system in batches.

Data Ingestion API Format (Server-to-Server)

The PubMatic Data Ingestion API URL for server-to-server data transfer:

http://aud.pubmatic.com/AdServer/Artemis?dpid=[data_partner_ID_assigned_by_PubMatic_at_the_
time_of_integration]&userid=[PubMatic_User_ID]&segid=[comma_separated_list_of_segment_IDs]

&ip=[last_known_user_IP_address]&country=[country_code]&addseg=[comma_separated_list_of_
segment_IDs
_to_be_appended]&remseg=
[comma_separated_list_of_segment_IDs_to_be_removed]

In the API URL above, the data partner should replace the following variables with actual values during each request:

 

  • [PubMatic_User_ID] should be replaced with the PubMatic user ID, i.e., the user ID passed to the data partner by PubMatic during the UID sync up phase.
  • [comma_separated_list_of_segment_IDs] should be replaced with a comma-separated complete list of IDs of all the audience segments to which the user currently belongs.
    Note: You should use the segid parameter to send the complete list of IDs for the segments to which the user currently belongs. This means that any subsequent request for that user with the segid parameter will overwrite the user’s already existing segments’ list. It is incorrect to send segid and (addseg or remseg) in the same request.
  • [last_known_user_IP_address] should be replaced with last known IP address of the user. If it is not known, then this parameter should be excluded from the request. This parameter is optional if the Country Code is sent in the request.
  • [country_code] should be replaced with the two-character code of the country to which the user belongs (for example, US, IN, CA). If it is not known, then this parameter should be excluded from the request. This parameter is optional if the IP address is sent in the request.
  • [comma_separated_list_of_segment_IDs_to_be_appended] should be replaced with a comma-separated list of IDs of any new audience segments which need to be appended to the user’s already existing list of segment IDs.
    Note: You can either send the complete list of segments for the user using the segid parameter or just send the list of segments to be appended to the user’s existing segments’ list using this addseg parameter. It is incorrect to send both segid and addseg in the same request.
  • [comma_separated_list_of_segment_IDs_to_be_removed] should be replaced with a comma-separated list of IDs of any audience segments which need to be removed from the user’s already existing list of segment IDs.
    Note: You can either send the complete list of segments for a user using the segid parameter or just send the list of segments to be removed from the user’s existing segments’ list using this remseg parameter. It is incorrect to send both segid and remseg in the same request.

The value of [data_partner_ID_assigned_by_PubMatic_at_the_time_of_integration] will be shared by PubMatic with the data partner at time of integration.

 

Sample server-to-server integration requests are as follows:

 

  1. http://aud.pubmatic.com/AdServer/Artemis?dpid=89&userid=B8CB98E6-0FF0-4C2B-A161-4F89155C28F&segid=123,98901,4532&ip=74.125.236&country=US
    In this request, 89 indicates the identifier assigned by PubMatic to the data partner, B8CB98E6-0FF0-4C2B-A161-4F89155C28F indicates the PubMatic user ID and the user belongs to three segments with IDs - 123, 98901 and 4532. The last known IP of the user was 74.125.236 and the user is from US.
  2. http://aud.pubmatic.com/AdServer/Artemis?dpid=89&userid=B8CB98E6-0FF0-4C2B-A161-4F89155C28F&ip=74.125.236&country=US&addseg=3242&remseg=123
    In this request for the same user, segment ID 3242 needs to be appended to the user’s existing list, and segment ID 123 needs to be removed from it, that is, the final user’s list of segments is 98901, 4532 and 3242.

Note: It is mandatory to send either the IP address or the Country Code in the API request to PubMatic.

Note: The data partner should periodically transfer data (typically once in every 3 hours) using this API URL to PubMatic.

Browser-side Real-time (HTTP 302 Redirect) Data Transfer

Data partners who do not support the server-to-server data transfer mechanism can pass the data to PubMatic using the browser-side real-time data transfer mechanism for each visitor. This implementation is depicted in the following figure.

  1. A user visits the website using a browser, which upon loading, executes the PubMatic pixel.
  2. The PubMatic pixel invokes the data partner's pixel or URL to request the audience segment data for this visitor. To reduce the latency, this request is sent to the data partner only once in a pre-configured time interval (e.g., once per day) per user.
  3. The data partner's system then redirects to the PubMatic Data Ingestion API using the HTTP 302 redirect. This API URL will be shared by PubMatic with the data partner before the start of the integration. While redirecting to the API URL, the data partner's system should include all the IDs of the audience segments to which the visitor belongs.
    Note: The data partner also needs to provide the segment ID-to-segment name mapping to PubMatic only once at the start of the integration.

Data Ingestion API Format (Browser-Side)

The PubMatic Data Ingestion API URL for the browser-side real-time data transfer:

http://aud.pubmatic.com/AdServer/Artemis?dpid=[data_partner_ID_assigned_by_PubMatic_at_the_
time_of_integration]&segid=[comma_separated_list_of_segment_IDs]&addseg=[comma_separated_

list_of_segment_IDs_to_be_appended]&remseg=[comma_separated_list_of_segment_IDs_to_be_

removed]

In the above API URL, the data partner needs to replace the following variables with actual values during each request:

  • [comma_separated_list_of_segment_IDs] should be replaced with a comma-separated list of IDs of all the audience segments to which the user currently belongs.
  • [comma_separated_list_of_segment_IDs_to_be_appended] should be replaced with a comma-separated list of IDs of the audience segments which need to be appended to the user’s existing list of segment IDs.
    Note: You can either send the complete list of segments for a user using the segid parameter or just send the list of segments to be appended to the user’s segments’ list using this addseg parameter. It is incorrect to send both segid and addseg in the same request.
  • [comma_separated_list_of_segment_IDs_to_be_removed] should be replaced with a comma-separated list of IDs of the audience segments which need to be removed from the user’s existing list of segment IDs.

    Note: You can either send the complete list of segments for a user using the segid parameter or just send the list of segments to be removed from the user’s segments’ list using this remseg parameter. It is incorrect to send both segid and remseg in the same request.

The value of [data_partner_ID_assigned_by_PubMatic_at_the_time_of_integration] will be shared by PubMatic with the data partner at time of integration.

 

Sample browser-side real-time data transfer requests:

  1. http://aud.pubmatic.com/AdServer/Artemis?dpid=89&segid=123,98901,4532
    In this request, 89 indicates the identifier assigned by PubMatic to the data partner and the user belongs to three segments with IDs - 123, 98901 and 4532.
  2. http://aud.pubmatic.com/AdServer/Artemis?dpid=89&addseg=3242&remseg=123 
    In this request for the same user, segment ID 3242 needs to be appended to the user’s existing list, and segment ID 123 needs to be removed from it, that is, the final user’s list of segments is 98901, 4532 and 3242.

Bulk Audience Data Transfer

PubMatic provides the option to send data in bulk using a flat file using the following specifications.

 

Every row in file should use the following format:
[DataProvider_ID] [TAB] [PubMatic_User_ID] [TAB] [Segment_IDs] [TAB] [Last_Known_User_IP_Address] [TAB] [Country_Code]

 

[if !supportLists]      [endif]  

[if !supportLists]      [endif]   [DataProvider_ID]: ID assigned by PubMatic for this data provider at the time of integration.

[if !supportLists]      [endif]   [PubMatic_User_ID]: Unique identification string to identify a user in PubMatic. This will be the PubMatic user ID based on cookie sync with PubMatic.

[if !supportLists]      [endif]   [Segment_IDs]: Comma-separated list of segments to which user belongs. To remove a user from all audiences to which the user previously belonged, do not pass any segment ID. Audiences in the latest record will be stored for the user ID, i.e. we overwrite the audiences of a user. Please make sure you have all the audiences that each user belongs in comma separated format and not just a delta.

Note: These segment IDs should already be registered by the DMP with PubMatic. These IDs come from the DMP’s platform and are not assigned by PubMatic on registration.

  • [Last_Known_User_IP_Address]: Last-known IP address of the user. If it is not known, this parameter should be excluded from the request. This field is optional if the [Country_Code] is provided.
  • [Country_Code]: Refers to the two-character code of the country to which the user belongs (for example, US, IN, CA). If it is not known, then this parameter should be excluded from the request. This field is optional if the [Last_Known_User_IP_Address] is provided.

 

Note: It is mandatory to provide either [Last_Known_User_IP_Address] or [Country_Code].

The file name should follow a pattern such as <DMPID>_<TIMESTAMP>_<MD5SUM>.tsv' and while transfer is in progress, suffix the file with '.UPLOADING' and rename it back once completed. We support compression formats like gzip or bzip2 etc. If you are compressing data file, the filename would of course be ".tsv.gz" or the like.


Before sending data for specific segment IDs in the TSV file, the DMP/publisher should have already registered those segments with PubMatic. The DMP/publisher can either email PubMatic any new segment names and segment IDs to register with PubMatic or use the PubMatic Audience Registration API for this purpose.

 

Next steps:

  1. Contact your account manager for FTP server details and login credentials.
  2. Ensure the segments are registered in PubMatic.
  3. Send a test file and confirm when the data is sent.
  4. If everything works, PubMatic will confirm integration is complete.

Attachments

    Outcomes