A new study reveals how AI-powered zero-shot classification rivals traditional methods in organizing art data, making investment analysis and auction pricing more efficient than ever.
Research: Zero-Shot Classification of Art With Large Language Models. Image Credit: Peshkova / Shutterstock
Art has emerged as a significant investment asset. This has led to growing interest in art price prediction as a tool for assessing potential returns and risks. However, organizing and annotating the data required for price prediction is challenging due to the substantial human costs and time involved.
Traditional machine learning (ML) methods, such as Logistic Regression, Random Forest, and XGBoost, require thousands of labeled examples for training, making data preprocessing particularly labor-intensive.
To address this, researchers applied a technique known as "zero-shot classification," which leverages a large language model (LLM) to classify data without the need for pre-prepared training data.
The research team explored the feasibility of automatically determining artwork types—such as paintings, prints, sculptures, and photographs—by optimizing several large language models, including the LLM "Llama-3 70B," which was optimized to a 4-bit format for local execution, significantly reducing hardware requirements.
Other models tested included GPT-4o, GPT-4o mini, Llama-2, and Gemma, with Llama-3 70B achieving the highest accuracy among local models at 0.9, outperforming OpenAI's GPT-4o by 0.025.
Furthermore, GPT-4o mini performed only marginally worse than GPT-4o, but at a cost that is approximately 30 times lower per million tokens, making it a cost-effective alternative for large-scale applications.
This approach enables performance comparable to conventional machine learning methods while notably reducing the human effort and time required for data organization. Llama-3 70B's accuracy was only 0.051 lower than the best-performing traditional ML methods, demonstrating that zero-shot classification is a viable alternative.
Although machine learning models such as Random Forest and XGBoost still slightly outperformed LLMs, the latter provided a major advantage in automating annotation, reducing manual data labeling efforts.
Notably, LLMs primarily based their classification on the artwork's medium description and dimensions, while artist names played a lesser role—an insight that could inform future model refinements.
These results could enhance accessibility to art analyses and price evaluation, expanding opportunities not only for investment but also for research, automatic categorization of auction data, and broader applications in art market analytics.
Source:
Journal reference: