Industries
Banking
Banking Services Transformation: Unlocking Exponential Value
Solution
TCS MasterCraft™
TCS MasterCraft™: Enabling Cognitive and Intelligent Automation
Highlights
- Synthetic data usage started for testing, later for privacy, and then for artificial intelligence (AI) and machine learning (ML) modeling. There is a growing demand for using synthetic data to better maintain privacy while using the closest form of real data.
- AI-based generative adversarial network (GAN) models are gaining greater importance in synthetic data generation. The architecture of GAN models consists of two parts: discriminator and generator. The basic data-based model is called Tabular GAN (TGAN), and industries have started piloting this model.
- These models mainly use the sample real data as seed and convert the data into one hot encoding form using the Gaussian mixture or random distributions. The transformed one hot encoded trained data model generates the synthetic data model as and when executed.
On this page
Decade of sysnthetic data
Synthetic data can be used as an alternative or supplement to real-world data when real-world data is not readily available or confidential and cannot be used directly. Synthetic data usage initially started for testing purposes, later for privacy, and then for AI/ML modeling. Given the growing importance of data and decision-making based on data, the present decade is called AI’s “decade of data” however, in the future, it may be called the “decade of synthetic data.”
Synthetic data creation
- Data generation for testing: This test data is mainly created using business rules or simulation-based scenarios.
- Data for AI modeling or alternative for performance checking: This data is generally created using Gaussian model algorithms with sample input seeded data. However, these algorithms are random and cannot be 100% accurate.
The industry is concerned about users' exposure to production data in view of the importance of data privacy and international regulations on data. Hence, there is a growing demand for synthetic data usage to better maintain privacy and use the closest form of real data. GenAI-based GAN models are gaining more importance in synthetic data generation, and research is evolving in this area.
The differential privacy framework enables data privacy using a randomized technique. It also facilitates synthetic data generation that maintains statistical similarity with real data; however, due to its inherent randomness, it is unable to handle large volumes of data.
GenAI-based models
GenAI-based models have emerged as a new synthetic data generation method. Once the model is trained, data can be generated based on patterns in large volumes of real data.
The architecture of GAN models consists of two parts: discriminator and generator. While GenAI models for image and video content have been doing wonders in the industry, data-based GenAI models are still evolving. The basic data-based model is called Tabular GAN (TGAN), and industries have started piloting this model. These models mainly use the sample real data as seed and convert the data into one hot encoding form using a Gaussian mixture or random distributions. The transformed one hot encoded trained
Types of TGAN models
TGAN has different architecture models, such as Conditional TGAN (CTGAN) and Wasserstein GAN (WGAN). Currently, the CTGAN model is the most popular, as it internally uses a Gaussian mixture model for one hot encoding of the data.
The CTGAN model supports numerical and categorical data only. However, the CTGAN model must be combined with other text data generator algorithms to have a synthetic data platform to support different data types. There are open-source models like "Faker"; however, they might not support domains and geographical data. Although some large language models (LLMs) support text data, there is still considerable scope for improvement. Hence, it is recommended that an orchestrated platform with CTGAN and proven random text-based algorithms be built to generate synthetic data for all various data types. The buzz is that soon, newer models will emerge to support a single platform for synthetic data generation.
