Conception and Implementation of API Products for Anonymized Vehicle Data
Since 2017, Mercedes-Benz AG and pentacor GmbH have been collaborating to realize numerous API and API management projects.
Since March 2023, we have been supporting the company in developing new versions of API products for providing anonymized vehicle data. The data is collected by various sensors in connected vehicles in the field and analyzed by business departments or external partners to ultimately provide location-based aggregated data in an anonymized form. This includes information about the road conditions (such as waviness or surface texture), traffic signs, potential hazards, and weather data. These anonymized data is provided as digital “Data-as-a-Service” products for consumers such as countries and municipalities.
Previously, Mercedes-Benz AG and pentacor, in collaboration with Microsoft, designed and implemented the new API management platform for managing public APIs. The API management platform went live in November 2022 and has since been maintained by pentacor.
In addition, pentacor has been involved in API management in the past, providing internal and public APIs for use by third-party applications and developers. Thus, we have extensive experience and a broad knowledge of APIs and API management.
This experience we bring to the internationally positioned project team. As part of the project, pentacor is responsible for the conception and implementation of the infrastructure and backend systems of the APIs and their integration into the API management platform.
We value very open communication and make our work transparent, so that the entire team and especially the client have full insight into our work and access to all data.
We are responsible for the complete conception and architecture of the platform and the individual services that we develop. The project is realized using agile methods with Scrum, while also taking into account predefined milestones.
Initial Situation and Objective
The implementation of the backend systems for providing the anonymized vehicle data is a new development aimed at replacing the previous implementation to meet quality and performance requirements.
Our newly developed solution is based on Microsoft Azure, particularly leveraging Azure Data Explorer as a powerful data platform for storing and retrieving data via services deployed in Azure Kubernetes Service. Event Hubs and Storage Accounts are used for flexible data ingestion. Configurable mappings in the data platform for the data input and output enable extensive decoupling from the data providers.
Key quality objectives focus on reliability, performance, and scalability. Additionally, extensibility is an important aspect of the design and realization of the backend platform, as Mercedes-Benz AG not only offers a range of API products for accessing anonymized vehicle data but also plans a series of further data products. There is also a focus on reusing common aspects of backend services across individual products while supporting product-specific features, such as data filtering.
Challenges in the Project
The new solution for providing anonymized vehicle data was implemented in parallel to the development of the previous version, requiring constant adaptation to new and changing requirements. The new version also aimed to improve the consistency and developer or consumer experience of the APIs.
We repeatedly faced challenges where the “Platform-as-a-Service” standard components provided by Microsoft Azure did not fully meet the specific requirements of Mercedes-Benz. Together with the customer, we developed creative adjustments and extensions to successfully align the requirements and objectives with the generic platform components.
Results
In implementing the API products for providing anonymized vehicle data, we were able to benefit from the technical foundation we had created with the work on the API management platform itself. Nonetheless, we consider both as largely independent projects to separate specific concerns. However, the used tool-chain benefited from the commonalities between the API management platform and the API products. In both cases, a high degree of automation was achieved through Azure DevOps Pipelines and the consistent use of Infrastructure-as-Code with Terraform. The use of these concepts was the key for creating a familiar development environment for the team that enables effective and efficient work and benefits from previous experiences.
The fact that API management platform and API product development are driven forward by the same team enables the evolution of API management platform base functions at the same time and simplifies the coordination with stakeholders.
For the anonymized vehicle data API products, the go-live was planned in multiple waves. In the first wave in mid-October 2023, a total of five products with their corresponding services went live.
All services and the required infrastructure, consisting of Azure PaaS components, can be provided independently but are based on a common core implementation that allows product-specific features to be implemented solely through configuration. This contributes to ensuring that all API products have identical basic functionality, and only the product-specific data filtering or processing planned for the future will require dedicated implementations, for which integration points are already provided.This ensures that all API products have identical basic functionality and that the product-specific data filtering or processing planned for the future will require dedicated implementations, for which the integration points are already provided.
Throughout the project, the main focus was on a holistic approach to architecture and realization, including comprehensive documentation. The platform’s services are tested at all levels, from whitebox unit tests to blackbox system integration tests at the platform level. The pentacor team is also responsible for automating build, deployment, and release processes and for monitoring the backend systems.
Technical Basis
At the heart of the API backends are PaaS components such as Azure Data Explorer, with additional Event Hubs and Storage Accounts for data ingestion from data providers. As additional persistence layer an Azure SQL database is also used for managing geographic areas for data scoping.
The services themselves are implemented in Java as Spring Boot applications and are deployed as containerized applications in a Kubernetes cluster based on Azure Kubernetes Service in combination with the Azure Application Gateway Ingress Controller. Azure-provided tools and services are also used for logging, monitoring, and alarming.
For configuring the platform’s infrastructure, we use Terraform and Helm for deploying applications in Kubernetes, both automated through Azure DevOps Pipelines.
Conclusion
The existing API products for providing anonymized vehicle data were optimized through redesign and a new implementation to improve performance, developer experience, and consistency of the individual products. This laid the foundation for future extensions of the core functions and the efficient implementation of further data-driven API products. The basis for this was laid with a clean decoupling from the data providers as well as a stable and standardized data architecture.
Since 2017, Mercedes-Benz AG and pentacor GmbH have been collaborating to realize numerous API and API management projects.
Since March 2023, we have been supporting the company in developing new versions of API products for providing anonymized vehicle data. The data is collected by various sensors in connected vehicles in the field and analyzed by business departments or external partners to ultimately provide location-based aggregated data in an anonymized form. This includes information about the road conditions (such as waviness or surface texture), traffic signs, potential hazards, and weather data. These anonymized data is provided as digital “Data-as-a-Service” products for consumers such as countries and municipalities.
Previously, Mercedes-Benz AG and pentacor, in collaboration with Microsoft, designed and implemented the new API management platform for managing public APIs. The API management platform went live in November 2022 and has since been maintained by pentacor.
In addition, pentacor has been involved in API management in the past, providing internal and public APIs for use by third-party applications and developers. Thus, we have extensive experience and a broad knowledge of APIs and API management.
This experience we bring to the internationally positioned project team. As part of the project, pentacor is responsible for the conception and implementation of the infrastructure and backend systems of the APIs and their integration into the API management platform.
We value very open communication and make our work transparent, so that the entire team and especially the client have full insight into our work and access to all data.
We are responsible for the complete conception and architecture of the platform and the individual services that we develop. The project is realized using agile methods with Scrum, while also taking into account predefined milestones.
Initial Situation and Objective
The implementation of the backend systems for providing the anonymized vehicle data is a new development aimed at replacing the previous implementation to meet quality and performance requirements.
Our newly developed solution is based on Microsoft Azure, particularly leveraging Azure Data Explorer as a powerful data platform for storing and retrieving data via services deployed in Azure Kubernetes Service. Event Hubs and Storage Accounts are used for flexible data ingestion. Configurable mappings in the data platform for the data input and output enable extensive decoupling from the data providers.
Key quality objectives focus on reliability, performance, and scalability. Additionally, extensibility is an important aspect of the design and realization of the backend platform, as Mercedes-Benz AG not only offers a range of API products for accessing anonymized vehicle data but also plans a series of further data products. There is also a focus on reusing common aspects of backend services across individual products while supporting product-specific features, such as data filtering.
Challenges in the Project
The new solution for providing anonymized vehicle data was implemented in parallel to the development of the previous version, requiring constant adaptation to new and changing requirements. The new version also aimed to improve the consistency and developer or consumer experience of the APIs.
We repeatedly faced challenges where the “Platform-as-a-Service” standard components provided by Microsoft Azure did not fully meet the specific requirements of Mercedes-Benz. Together with the customer, we developed creative adjustments and extensions to successfully align the requirements and objectives with the generic platform components.
Results
In implementing the API products for providing anonymized vehicle data, we were able to benefit from the technical foundation we had created with the work on the API management platform itself. Nonetheless, we consider both as largely independent projects to separate specific concerns. However, the used tool-chain benefited from the commonalities between the API management platform and the API products. In both cases, a high degree of automation was achieved through Azure DevOps Pipelines and the consistent use of Infrastructure-as-Code with Terraform. The use of these concepts was the key for creating a familiar development environment for the team that enables effective and efficient work and benefits from previous experiences.
The fact that API management platform and API product development are driven forward by the same team enables the evolution of API management platform base functions at the same time and simplifies the coordination with stakeholders.
For the anonymized vehicle data API products, the go-live was planned in multiple waves. In the first wave in mid-October 2023, a total of five products with their corresponding services went live.
All services and the required infrastructure, consisting of Azure PaaS components, can be provided independently but are based on a common core implementation that allows product-specific features to be implemented solely through configuration. This contributes to ensuring that all API products have identical basic functionality, and only the product-specific data filtering or processing planned for the future will require dedicated implementations, for which integration points are already provided.This ensures that all API products have identical basic functionality and that the product-specific data filtering or processing planned for the future will require dedicated implementations, for which the integration points are already provided.
Throughout the project, the main focus was on a holistic approach to architecture and realization, including comprehensive documentation. The platform’s services are tested at all levels, from whitebox unit tests to blackbox system integration tests at the platform level. The pentacor team is also responsible for automating build, deployment, and release processes and for monitoring the backend systems.
Technical Basis
At the heart of the API backends are PaaS components such as Azure Data Explorer, with additional Event Hubs and Storage Accounts for data ingestion from data providers. As additional persistence layer an Azure SQL database is also used for managing geographic areas for data scoping.
The services themselves are implemented in Java as Spring Boot applications and are deployed as containerized applications in a Kubernetes cluster based on Azure Kubernetes Service in combination with the Azure Application Gateway Ingress Controller. Azure-provided tools and services are also used for logging, monitoring, and alarming.
For configuring the platform’s infrastructure, we use Terraform and Helm for deploying applications in Kubernetes, both automated through Azure DevOps Pipelines.
Conclusion
The existing API products for providing anonymized vehicle data were optimized through redesign and a new implementation to improve performance, developer experience, and consistency of the individual products. This laid the foundation for future extensions of the core functions and the efficient implementation of further data-driven API products. The basis for this was laid with a clean decoupling from the data providers as well as a stable and standardized data architecture.
Get in touch
