Using advanced neural networks, researchers have developed a pioneering AI-based system to unmask art forgeries by Wolfgang Beltracchi, helping art experts pinpoint suspicious works with unprecedented precision.
Research: Art Forgery Detection using Kolmogorov Arnold and Convolutional Neural Networks
*Important notice: arXiv publishes preliminary scientific reports that are not peer-reviewed and, therefore, should not be regarded as conclusive, guide clinical practice/health-related behavior, or treated as established information.
In an article recently submitted to the arXiv preprint* server, researchers at the University of Zurich presented an art authentication framework to identify the German former art forger Wolfgang Beltracchi.
They shifted the focus from traditional artificial intelligence (AI) methods by developing a multiclass image classification model using an efficient neural network (EfficientNet) and training it on a carefully compiled dataset of Beltracchi's forgeries and known works. The results were compared with Kolmogorov Arnold networks (KAN), which had yet to be previously tested in the art domain.
Overall, there was general agreement between the predictions of the different models regarding flagged forgeries, which were further analyzed through detailed art-historical visual analysis, focusing on stylistic elements like brushstrokes and composition.
Related Work
Past work has shown that art authentication benefits significantly from interdisciplinary approaches, combining art historical analysis with technical scientific methods. Various studies employed AI to authenticate artworks, focusing mainly on digital images.
Techniques ranged from using optical coherence tomography to analyze cracks in paintings to machine learning algorithms and vision transformers on datasets of artists like Vincent van Gogh.
Additionally, a case study by Art Recognition demonstrated the application of a vision-based algorithm trained on Max Pechstein’s works to analyze brushstrokes and identify Wolfgang Beltracchi's forgeries.
Methodology for Art Authentication
AI-aided art authentication is typically a binary image classification task, where class 0 represents the artist of interest and class 1 includes forgers, imitators, and similar artists.
This study shifted the focus to a specific forger, Wolfgang Beltracchi, alongside several authentic artists, necessitating a multiclass classification approach. Each class comprises genuine artists whose works were forged by Beltracchi, with an additional class for the forger himself.
The methodology involved preprocessing the paintings by splitting them into center-cropped patches of 256x256 pixels, a simple yet powerful data augmentation technique. This approach allowed the model to concentrate on finer details during training.
To address the issue of superficial artifacts resulting from digital encoding, a Gaussian blur was applied to enhance image clarity. Additionally, patches with minimal information, such as background-only images, were filtered out based on entropy measures calculated across the red, green, and blue (RGB) channels.
The study evaluated state-of-the-art convolutional neural networks (CNNs) such as EfficientNet against the newly introduced KAN for model comparison. EfficientNet is designed to scale efficiently across depth, width, and resolution and has demonstrated strong performance in various image classification tasks, including art authentication.
The researchers fine-tuned the EfficientNet model for their specific dataset and conducted experiments using different network versions to assess performance variability.
The dataset consisted of works from 11 artists known to have been forged by Beltracchi, along with the forger's artworks, totaling 1,334 paintings. Images were primarily sourced from Wikiart, focusing on those with a minimum width of 768 pixels to ensure quality.
The dataset was split into training, validation, and test sets while providing a uniform distribution of artists across these subsets. Ultimately, 10% of the paintings were allocated to the test set. The remainder was divided into 80% for training and 20% for validation, with ten random seeds used for repeated dataset splits to ensure the statistical robustness of the results.
Enhancing Forgery Detection Techniques
The authors assessed two preprocessing techniques in this evaluation, determined the optimal model size, and compared CNN's EfficientNet to KAN. The entropy-based patch filtering method was tested by training EfficientNet B0 on ten training set splits using entropy thresholds of 0, 2.5, and 3.
Results showed that while the differences in accuracy were not significant, an entropy threshold of 2.5 yielded the highest accuracy across different training set selections. Consequently, this threshold was employed in subsequent experiments.
EfficientNet B0 was trained with an entropy threshold of 2.5, once on original images and again on slightly blurred images. The results indicated a slight improvement in validation accuracy with blurring, although the differences were again insignificant.
The evaluation of different versions of EfficientNet revealed that EfficientNet B2 achieved the highest accuracy, while EfficientNet B0 offered a good tradeoff between accuracy and computational cost.
Furthermore, the authors compared the EfficientNet architecture and a simple KAN model with varying configurations. The results indicated that a medium-sized KAN model of 120x32x12 provided the best performance, but the accuracies were generally lower than those of EfficientNet. The authors acknowledged the limited exploration of the KAN parameter space, indicating that further tuning might improve its performance.
Additionally, they addressed the challenge of selecting an authentic training set from a pool containing potential forgeries, emphasizing the need to manage label noise effectively.
Statistical analysis confirmed the models' effectiveness in identifying potential forgeries by Wolfgang Beltracchi, with both EfficientNet B0 and KAN consistently predicting certain paintings as forgeries. The study highlights that a small percentage of mislabeled data (around 1% of the dataset) was flagged as questionable, and these patches warrant further investigation.
The findings, framed within art history, highlighted stylistic traits associated with Beltracchi's work, indicating that the algorithm could help address dataset biases and improve forgery detection methods.
Conclusion
To sum up, the authors contributed to art authentication research by demonstrating the effectiveness of CNN and KAN models in detecting forgeries by famous artists, helping experts focus on questionable attribution areas. They found that EfficientNet models accurately identified artworks attributed to Beltracchi based on specific stylistic features.
The study emphasized the importance of refining KAN architectures and mitigating the mislabeling of training sets. Future work will aim to optimize KAN architectures, improve dataset resolution, and incorporate feedback from art experts to refine forgery detection algorithms.
*Important notice: arXiv publishes preliminary scientific reports that are not peer-reviewed and, therefore, should not be regarded as conclusive, guide clinical practice/health-related behavior, or treated as established information.
Journal reference:
- Preliminary scientific report.
Boccuzzo, S., et al. (2024). Art Forgery Detection using Kolmogorov Arnold and Convolutional Neural Networks. ArXiv. DOI:10.48550/arXiv.2410.04866, https://arxiv.org/abs/2410.04866