CDP implementation: data collection

CDP implementation: stages of implementation

The valantic blog series on CDP implementation provides you with detailed insights into the different stages of the project. After project organization, the specification and the basic configuration, stage three in the implementation process follows: data collection.

Creating a data repository

In the prior specification phase, all participants defined which data is required for the selected use cases and in which form and frequency it should be transmitted to the customer data platform. In the basic configuration, it was determined whether the source systems are directly integrated into the CDP or whether some kind of middleware is required. At this point – stage three – there is clarity about the required source systems and data sets. The following data types can be distinguished:

• Customer master data: This includes all master data of the customers that are required for the personalization of the defined use cases – for instance, a unique ID per customer.

• Consents: These are the required consents of the customers in order to be allowed to process the corresponding content.

• Events: Digital as well as offline touchpoints of the customers are stored as so-called events in the CDP at a certain point in time.

• Catalogue data: Catalogue data describes the storage of further objects that are related to customers and events and can change regularly. Examples are product data from the shop or the PIM system.

In addition to the data types just mentioned, the following options for collecting data are integrated in most CDPs:

• (REST)-API: This http-based and very flexible way of collecting data is offered by most CDPs and usually provides different endpoints for reading and writing the individual data types.

• Database-based: Since most CDPs are cloud-based SaaS solutions, the database must be externally accessible. The information is then regularly retrieved via SQL. The structured query language (SQL) serves as the standard language for communication between the various databases. The SQL functions as a kind of interface in which external queries from users are translated into commands that the database can understand.

• File-based: In this case, files are regularly retrieved, such as CSV files, which are then stored via a URL, an SFTP server or in the cloud.

• Web/ App SDK-based: With most customer data platforms, a software development kit (SDK) is also offered to track the behavior of customers. Important: To be compliant with the GDPR, the SDK should only be loaded after the cookies have been accepted.

Agile collaboration for a more effective implementation

For the implementation of the data repository, an agile approach is recommended, as already described in the first part of our blog series. That’s why, for the integration of the various source systems, a close exchange with the internal development department is advisable. In this way, tests can be carried out in great detail, and data already supplied can be checked for correctness and completeness. The duration of the implementation process is highly dependent on the number of source systems and the complexity of the use cases, which is why the timeframe can vary from a few weeks to several months. An agile implementation approach offers the advantage that less complex use cases can be implemented at an early stage.

Once the basic configuration is complete and the implementation of the data sets is underway, the next stage of the project can begin: configuration of the use cases.