In a paper published in the journal Scientific Data, researchers introduced the global climate change mitigation policy dataset (GCCMPD); developed using a semi-supervised hybrid machine learning (ML) approach, the GCCMPD drew from international and regional policy sources, covering 73,625 policies across 216 entities. It provided a detailed classification of indicators and sectoral policy instruments, including objectives, target sectors, instruments, legal compulsion, and administrative entities.
Machine learning processes for identifying duplicate policies. Image Credit: https://www.nature.com/articles/s41597-024-03411-z
The dataset integrated expert knowledge-based dictionary mapping, probability statistics, advanced natural language processing (NLP), and the dataset aligned with the Intergovernmental Panel on Climate Change (IPCC) emission sector classification. The GCCMPD aimed to help policymakers, researchers, and social organizations understand climate activities across countries and sectors.
Related Work
Past work has highlighted the substantial growth in climate-related policies since the first World Climate Conference in 1979, with international governance milestones such as the Kyoto Protocol 1997 and the Paris Agreement in 2015 establishing national mitigation targets and mechanisms. National, subnational, and sectoral policies have also expanded, addressing environmental protection, economic development, equity, and sustainable development.
Quantitative policy studies have emerged to assess or simulate policy impacts empirically. However, these studies often focus on a single country or industry or compare a few countries, requiring limited policy data.
GCCMPD Development Overview
The development of the GCCMPD involved a meticulous process spanning data identification, collection, processing, and verification. Diverse global and regional sources were integrated, including the Climate Change Laws of the World (CCLW), the International Energy Agency (IEA) Policies and Measures Dataset, and the Climate Policy dataset (CP), to ensure comprehensive coverage of climate-related policies. This dataset encompasses laws, regulations, and strategies for climate change mitigation and adaptation, offering a detailed classification of multiple indicators crucial for policy analysis.
Data processing entailed harmonizing classifications across datasets and utilizing NLP techniques for refinement. The dataset's indicators, such as sector, instrument, and objective, were categorized based on authoritative classifications like those from the IPCC Fifth Assessment Report (AR5). Manual verification ensured consistency and accuracy, supplemented by searches for and translations of missing policy content. The analysts curated this dataset meticulously through these steps to provide reliable and detailed information for analysis.
Verification procedures included using advanced algorithms like Best Match 25 (BM25) to check for duplicates and manually verifying policy characteristics. Indicators such as sector, instrument, and objective were meticulously categorized based on established classifications, ensuring consistency and accuracy. Additionally, efforts were made to address missing or incomplete policy content through manual searches and translations, enhancing the dataset's reliability and comprehensiveness.
Finally, dataset expansion employed advanced NLP and ML techniques to overcome the limitations of manual processing. Climate bidirectional encoder representations from transformers (ClimateBERT), a model fine-tuned for climate-related texts, were utilized for multilabel and single-label classification of policy indicators. Named entity recognition (NER) was employed to identify jurisdictions, while text similarity methods ensured accurate identification of duplicate policies. Topic modeling with bidirectional encoder representations from transformers topic (BERTopic) provided insights into policy evolution, expanding the dataset's analytical capabilities and robustness.
Enhancing Data Quality
Several factors influence the data quality of the GCCMPD, including the comprehensiveness of data sources and the reliability of annotation and training sets. Manual retrieval from high-quality literature, Google searches, and authoritative datasets was conducted to ensure high-quality data, focusing on comprehensive policy details, the policy year, and entity information.
Doctorate students in climate and energy fields compared manual annotations and checks, ensuring objectivity through dictionary mapping. Authoritative data from IEA, CP, and CCLW served as benchmarks, with high recall rates confirming the effectiveness of dictionary mapping. However, categories with lower recall rates and precision issues due to classification discrepancies required further manual verification and annotation.
State-of-the-art models were used to enhance model training capabilities, with improvements including translating non-English policies into English and thoroughly checking and completing policy content. Manual inspections summarized policy goals, purposes, and objectives and addressed policy duplication for accuracy.
The comparison between dictionary mapping and manual verification highlighted high recall and precision areas, with some categories needing refinement. Public release of the GCCMPD data will allow user interactions to enhance data sources and correct errors.
Despite these efforts, the dataset needs adaptation policies that are vital for countries with high climate vulnerability and mitigation costs. Future research should incorporate an adaptation policy dataset to address this critical aspect of climate policy analysis.
Conclusion
To sum up, PhD students in climate and energy fields played a crucial role in comparing manual annotations and checks, ensuring objectivity through dictionary mapping. This process highlighted the importance of meticulous manual verification and the need for consistency across various data sources.
The high recall rates demonstrated the effectiveness of these efforts, although certain categories still required further refinement. Overall, these steps significantly enhanced the reliability and accuracy of the GCCMPD, providing a robust foundation for future climate policy analysis.
Machine Learning Identifies Seismic Precursors, Advancing Earthquake Forecasting Capabilities