How to validate installs and in-app event data from MMPs like Appsflyer / Adjust / Branch.io ?

What if I am not sure if MMPs like Adjust or Appsflyers have correctly tracked all installs and in-app events for the App, and wanted to confirm it has been recorded 100%?

This happens especially when startups do not have a well-deployed user analysis system, the logic to confirm it, is to find a baseline or trusted data source that has "100%" install data,

I will recommend using Firebase as the data source, all MMPs have provided a Push API(Appsflyer) or Callbacks(Adjust) that allows you to access the raw data flow.

Data difference between Firebase "first_open" event and MMPs "install" event

Firebase first_open = install + reinstall from MMPs

Normally, the Firebase first_open event number will be higher than the install event from MMPs, cause the first_open event fires every time your App first launches (no matter whether it was first installed on a device or deleted and reinstalled again on the same device). This is different for MMPs, Adjust or Appsflyer has a setting of "Re-attribution window", the default setting is 90 days.

The re-attribution window starts when the app is first installed. During the window duration, reinstalls can't be attributed as new installs. Reinstalls are ignored unless they are a result of a user engaging with a retargeting campaign. This is referred to as a re-attribution. Learn More

previous_first_open_count parameters in first_open event

The Firebase first_open event is an Automatically collected event and has an event parameter previous_first_open , which shows on the device level, how many times the app has been installed and first launched before, if it's a brand new install, the previous_first_open_count = 0, and increases by the time that the device deletes the app and reinstalls it.

So number of first_open (where previous_first_open_count = 0) will be closer to install data from MMPs.

What happens to previous_first_open_count when an iOS device does not get ATT opt-in, that didn't accept Apple's iOS 14 app-tracking prompt?

Although Firebase can't get access to IDFA because did not get ATT opt-in, Firebase can still get IDFV, a single device identifier that is from the same publisher/vendor, so the previous_first_open_count can still work for all iOS devices.

By blow SQL you can see the number of first_open events grouped by previous_first_open_count and by OS systems in BigQuery.

SELECT
  device.operating_system AS os,
  event_params.value.int_value AS previous_first_open_count,
  COUNT(*) AS count_of_events
FROM
  `your_project.your_dataset.events_*`, /*replace with your project id and dataset */
  UNNEST(event_params) AS event_params
WHERE
  event_name = 'first_open'
  AND event_params.key = 'previous_first_open_count'
GROUP BY
  os, previous_first_open_count
ORDER BY
  os, previous_first_open_count

You will sample results like this

Validate MMP install event by using ADID and IDFV as the identifier to cross-compare the first_open event from Firebase

Pull ADID and IDFV from first_open event of Firebase in BigQuery

The ADID in the Firebase event is stored as advertising_id, and IDFV is stored as vendor_id, both are parameters of the device column, refer to [GA4] BigQuery Export schema.

So by blow SQL, you can extract first_open event with advertising_id , IDFV, and previous_first_open_count, by date and timestamp.

SELECT
  event_name,
  device.operating_system,
  device.advertising_id,
  device.vendor_id,
  TIMESTAMP_MICROS(event_timestamp) AS event_timestamp,
  DATE(TIMESTAMP_MICROS(event_timestamp)) AS event_date,
  device.mobile_brand_name,
  (SELECT value.int_value FROM UNNEST(event_params) WHERE key = 'previous_first_open_count') AS previous_first_open_count
FROM
  `your_project.your_dataset.events_*`,  /* replace with your project id */
WHERE
  event_name = 'first_open'
  AND (DATE(TIMESTAMP_MICROS(event_timestamp))  = '2023-12-03' or DATE(TIMESTAMP_MICROS(event_timestamp))  = '2023-12-04')  /*set time period*/
ORDER BY
  event_timestamp

You can edit time limit for a specific date range, and save results to google sheet for further comparison.

Access install raw data from MMPs

Here I listed examples of export install raw data from Appsflyer and Adjust

Export install postback data from Appsflyer

Appsflyer provides a "Raw Data Export" under "Report" section, you can choose installs under postback, just remember:

1. For iOS and Android, you will need to export saparately under each App.

2. The duration of time on Appsflyer is following your own timezone settings, but Firebase install data time stamp is UTC+0, so you may need to match with timezone.

another way is to use Appsflyer Push API to record install data, same as Adjust callback.

Export install postback data from Adjust

Adjust does not have a raw data export function on the dashboard, so if you want to export raw install data, you can contact your account manager and specify the date range the account manager will export for you.

Or, we can use Adjust callback functions to record the install data manually, just you can't access historical data from callbacks.

You can use make.com to create a simple webhook that can receive callbacks from Adjust, and save it to a Google sheet, not a single line of code needs to be written, all done with a few simple clicks.

And when you configure the Adjust install callbacks, remember to add the "User Data" section of parameters so the ADID and IDFV will be included in the callback.

Validate install data from MMP by comparing ADID and IDFV to the first_open event from Firebase

Now you have the first_open event and install event for a certain period of time, you can use a simple lookup function in Google Sheets to find which ADID or IDFV are missing and provide it to developers to further locate problems.