Most state-of-the-art deep learning models utilize large-scale datasets. However, such scale comes with computational cost, as data must be stored, preprocessed, and trained on. To reduce such cost, this paper suggests generating a smaller dataset by minimizing the difference in gradients between the original large dataset and the synthesized small dataset.
This paper may be interesting to you if you:
Consider an image classification problem with $C$ classes. We hope to generate a small dataset $S$ from a large dataset $T$, such that the neural network $\phi_{\theta^S}$ with parameters $\theta^S$ trained with the small dataset has comparable performance to that of the neural network $\phi_{\theta^T}$ with parameters $\theta^T$ trained with the large dataset.