In an era where data privacy and security have become paramount concerns, federated learning offers an elegant solution that preserves individual data privacy while harnessing the collective power of distributed datasets. This cutting-edge method allows organizations and machines to work together, training robust and accurate models across a network of participants while keeping sensitive information where it belongs—locally.
In this third article of our article series “Closer look on…” we will delve into the principles, applications, challenges, and the transformative potential federated learning methods holds for the manufacturing industry and which role it plays within the MARS project.
Not only it is a decentralized machine learning approach that enables multiple parties to train a shared model without sharing their raw data but also instead of centralizing data on a server, each participant computes a local model update based on its data and shares only this update with a central aggregator. This process is iterative, with model updates being aggregated and refined over multiple rounds.
Application fields
In scenarios where data privacy, security, or regulatory compliance is essential, federated learning methods are particularly valuable as it keeps data localized, reduces privacy risks, and allows for collaborative model training while preserving individual data privacy. It finds applications in various fields, including healthcare, IoT, and edge computing, and addresses challenges like communication overhead and non-IID data distribution through ongoing research efforts.
Challenges of federated learning methods
Federated learning methods offer a promising approach to collaborative model training while addressing data privacy concerns, but they come with several challenges. Communication overhead can be a significant issue, leading to increased latency and bandwidth usage, especially when dealing with numerous nodes or devices. Non-Independently and Identically Distributed (non-IID) data distribution can hinder model convergence, and ensuring efficient convergence remains a challenge. Security and privacy risks persist, with potential vulnerabilities to attacks on federated learning systems. Customizing aggregation methods for specific use cases can be complex, and managing the heterogeneity of devices and resource constraints in edge computing environments can impact efficiency. Incorporating differential privacy mechanisms introduces noise into local updates, and compliance with data privacy regulations is complex. Nonetheless, ongoing research and advancements continue to address these challenges, including the use-cases that are going to be created within MARS aim to make federated learning more practical and effective for various applications while preserving data privacy and security.
Although AI in manufacturing often involves equipment that exhibits inhomogeneities across sites, the ambition of MARS is to leverage federation to model behaviour that emerges from the general characteristics of the domain of each problem (such general characteristics that appear on vibration signals), with customization and personalization to achieve local fine-tuning and optimization when needed.
Role within MARS
Within MARS, the objective is to harness the capabilities of federated learning to expedite the deployment of advanced AI-driven solutions for achieving zero-defect manufacturing. This approach involves training local AI models independently on individual nodes and subsequently aggregating these models into a unified one using federated learning algorithms. The resultant unified model is then distributed back to the respective nodes.
This consolidated model serves a dual purpose within the domain of process planning:
- Optimization: In the context of process optimization, the model considers both stringent and soft criteria, providing an assessment of process stability.
- Control: For process control, the model offers decision support by suggesting parameter adjustments that optimize cost and resource utilization while ensuring the process remains stable and efficient
Current status
During the first year of the MARS project, various AI models have been employed and trained on the data that are being collected from the pilot cases. A prototypical implementation of a Federated Learning Network has demonstrated the ability to aggregate these models using various federation strategies and distribute back models that scored at least as good as the original models. This scenario is now expanded to include data inhomogeneities, such as different parameterization of experiments (e.g., vibration data points collected from one axis on one site vs data points collected on three axes on another one), or special characteristics of the machines that lead to different outputs when fed with the same input vectors.
The current approach is to separate models in parts that correspond to global behaviour and ones that handle local characteristics and then have the first ones being federated via an appropriate strategy and the latter ones being locally optimized. Such decouplings are based on characteristics exhibited by each model or algorithm. Convolutional Neural Networks (CNNs) for example, tend to use the first layers to extract general information about the input, such as edges in a picture or spikes in a vibration signal, while the last layers are more sensitive to the specifics and local characteristics of the data and problem. Under this approach, the first layers of the CNN could be federated and the final ones be locally optimized to address the problem at hand. For example, a site may append layers that use the federated layer to detect areas of volatility in the vibration signal, while another site may append layers to detect equipment wear based on the analysis of the vibration.
The MARS consortium is currently working on using the Federated Learning Methods within the framework of the MARS project to realise and support the fundamental idea of building a distributed manufacturing platform. Stay tuned as there will be more on this topic in the first project results later this year.