Large Language Models News and Research

RSS

Unveiling Bias in Base Language Models: A Comprehensive Assessment and Mitigation Study

Meta researchers address the crucial need for fair generative large language models (LLMs), focusing on base models. The study introduces novel definitions of bias, employing a demographic parity benchmark, and expands evaluation metrics and datasets. Results reveal varying toxicity and bias rates, highlighting the importance of intersectional demographics, prompting, and ongoing dataset evolution for effective mitigation strategies in language models.

7 Dec 2023

MEDITRON: Advancing Medical Reasoning with Open-Source Large Language Models

This article introduces MEDITRON, an open-source Large Language Model (LLM) tailored for medical reasoning, featuring 7 billion and 70 billion parameters. Built on Llama-2 and leveraging a curated medical corpus, MEDITRON addresses access disparities to medical knowledge. The paper outlines an optimized workflow for scaling domain-specific pretraining, the MEDITRON models' engineering, and their superior performance on medical benchmarks, marking a significant stride toward accessible and capable medical LLMs.

1 Dec 2023

Bridging the AI Translation Gap in Healthcare: A Quality Management System Approach

Researchers propose leveraging a Quality Management System (QMS) tailored to healthcare AI as a systematic solution to bridge the translation gap from research to clinical application. The QMS, aligned with ISO 13485 and risk-based approaches, addresses key components enabling healthcare organizations to navigate regulatory complexities, minimize redundancy, and optimize the ethical deployment of AI in patient care.

30 Nov 2023

Emotional Pathways in User Experiences: Integrating ChatGPT into Knowledge Work

This study, published in AISeL, explores the user experience of integrating AI technologies like ChatGPT into knowledge work. Through interviews with 31 users, distinct phases were identified, ranging from pre-use curiosity and anxiety to the establishment of a tight intertwinement with ChatGPT as a collaborative assistant. The findings emphasize the emotional dimensions of AI adoption and raise important considerations for individuals, organizations, and society regarding potential dependencies, deskilling, and the evolving role of AI in the workplace.

22 Nov 2023

AI in Scientific Research: From Hypothesis Generation to Robotic Experimentation

This article explores the expanding role of artificial intelligence (AI) in scientific research, focusing on its creative ability in hypothesis generation and collaborative efforts with human researchers. AI, particularly large language models (LLMs), aids in proposing hypotheses, identifying blind spots, and collaborating on broad hypotheses, showcasing its potential in various fields like chemistry, biology, and materials science.

21 Nov 2023

AI-Generated Summaries of Preprints: Balancing Access and Accuracy in Scientific Literature

This article discusses bioRxiv's collaboration with ScienceCast, an AI startup, to use large language models for multi-level summaries of scientific preprints. While aiming to enhance accessibility, the pilot reveals challenges in accurately summarizing complex technical content, with scientists noting inaccuracies. The future outlook suggests potential benefits as AI capabilities advance, but concerns around precision and the need for a balance between automation and human oversight persist.

19 Nov 2023

CHARTing the Course for Transparent Evaluation: Establishing Reporting Standards for AI-Driven Chatbots in Healthcare

The paper addresses concerns about the accuracy of AI-driven chatbots, focusing on large language models (LLMs) like ChatGPT, in providing clinical advice. The researchers propose the Chatbot Assessment Reporting Tool (CHART) as a collaborative effort to establish structured reporting standards, involving a diverse group of stakeholders, from statisticians to patient partners.

16 Nov 2023

MetaQA: Revolutionizing Geospatial Data Search Using AI and Language Models

This study introduces MetaQA, a groundbreaking data search model that combines artificial intelligence (AI) techniques with metadata search to enhance the discoverability and usability of scientific data, particularly in geospatial contexts. The MetaQA system, employing advanced natural language processing and spatial-temporal search logic, significantly outperforms traditional keyword-based approaches, offering a paradigm shift in scientific data search that can accelerate research across disciplines.

15 Nov 2023

‌Roleplay and Simulation: A Framework for Understanding Conversational AI in Generative Language Models

Researchers propose viewing large language models (LLMs) in conversational AI through the lens of roleplay and simulation. This metaphorical framework helps avoid anthropomorphic misattributions, enabling nuanced interpretations of LLM behavior and fostering responsible development within ethical constraints.

12 Nov 2023

DAIL: Enhancing In-Context Learning with Data Augmentation through Language Models

Researchers propose an innovative approach, Data Augmentation for In-Context Learning (DAIL), addressing a common challenge in In-Context Learning (ICL) where annotated demonstrations are scarce. Leveraging large language models' self-generated content familiarity, DAIL employs self-paraphrasing and majority voting, surpassing standard ICL techniques, especially in low-resource scenarios.

9 Nov 2023

RoboGen: Automating Robotic Skill Learning with GPT-4

Researchers have developed RoboGen, a generative framework that utilizes GPT-4 to automate robotic skill learning. This system generates diverse tasks, scenes, training supervision, and skill learning environments, offering a scalable and automated approach to training robots with minimal human involvement.

7 Nov 2023

EGS Framework Enhances Communication Using Large Language Models

The Explore-Generate-Simulate (EGS) framework, powered by large language models (LLMs) helps improve communication in various scenarios. EGS assists individuals in crafting effective messages by exploring diverse advice, generating candidate messages, and simulating audience reactions, ultimately leading to enhanced communication and control in goal-directed interactions.

5 Nov 2023

Exploring the Threat of Embedding Space Attacks on Large Language Models

Researchers propose essential prerequisites for improving the robustness evaluation of large language models (LLMs) and highlight the growing threat of embedding space attacks. This study emphasizes the need for clear threat models, meaningful benchmarks, and a comprehensive understanding of potential vulnerabilities to ensure LLMs can withstand adversarial challenges in open-source models.

2 Nov 2023

Evaluating Large Language Models: A Comprehensive Survey

This paper presents a comprehensive survey of large language model (LLM) evaluation across various dimensions, including knowledge, reasoning, alignment, safety, and specialized domains. It covers a wide range of evaluation benchmarks and methodologies to assess the capabilities, ethical considerations, robustness, and application-specific performance of LLMs, aiming to guide the development of LLMs that are beneficial and trustworthy.

2 Nov 2023

Evaluating Language Models with SKILL-MIX: A Novel Approach

This article discusses a novel evaluation method called "SKILL-MIX" for assessing the capabilities of Large Language Models (LLMs). SKILL-MIX challenges LLMs by generating text that combines random subsets of skills and topics, providing a more comprehensive assessment of their abilities, and addresses the limitations of traditional evaluation methods in the evolving landscape of language models.

1 Nov 2023

Instilling Human Values and Moral Codes in AI Systems

This article explores the challenges and approaches to imparting human values and ethical decision-making in AI systems, with a focus on large language models like ChatGPT. It discusses techniques such as supervised fine-tuning, auxiliary models, and reinforcement learning from human feedback to imbue AI systems with desired moral stances, emphasizing the need for interdisciplinary perspectives from fields like cognitive science to align AI with human ethics.

1 Nov 2023

Open-Source AI: Fostering Collaboration and Responsibility

This article delves into the importance of embracing open-source principles in AI technology. It emphasizes the advantages of open-source models in fostering collaboration, accessibility, and accountability while addressing the challenges posed by proprietary AI technology. The author advocates for a responsible and inclusive approach to open-source AI to promote innovation and societal welfare.

31 Oct 2023

JudgeLM: Scalable Language Models for Evaluating Large Language Models

This research introduces JudgeLM, a scalable solution for evaluating Large Language Models (LLMs) like GPT-3 and GPT-4. By fine-tuning LLMs to judge the quality of LLM-generated responses, the study addresses the challenges of evaluating open-ended tasks while providing cost-effective, unbiased, and efficient evaluation mechanisms, promising a significant impact on the development of LLMs.

31 Oct 2023

Achieving Human-Like Systematic Generalization in Neural Networks through Meta-Learning

This research presents a novel approach, Meta-Learning for Compositionality (MLC), that enhances the systematic generalization capabilities of neural networks. Through meta-learning, MLC guides neural networks to exhibit human-like compositional skills, addressing challenges related to systematicity in neural networks.

29 Oct 2023

Woodpecker: Correcting Hallucinations in Multimodal Large Language Models

Researchers introduce Woodpecker, a novel training-free approach designed to address hallucinations in Multimodal Large Language Models (MLLMs). By systematically extracting key concepts, formulating questions, and utilizing visual knowledge validation, this method effectively corrects hallucinations in MLLM-generated responses.

27 Oct 2023

Large Language Models News and Research

Unveiling Bias in Base Language Models: A Comprehensive Assessment and Mitigation Study

MEDITRON: Advancing Medical Reasoning with Open-Source Large Language Models

Bridging the AI Translation Gap in Healthcare: A Quality Management System Approach

Emotional Pathways in User Experiences: Integrating ChatGPT into Knowledge Work

AI in Scientific Research: From Hypothesis Generation to Robotic Experimentation

AI-Generated Summaries of Preprints: Balancing Access and Accuracy in Scientific Literature

CHARTing the Course for Transparent Evaluation: Establishing Reporting Standards for AI-Driven Chatbots in Healthcare

MetaQA: Revolutionizing Geospatial Data Search Using AI and Language Models

‌Roleplay and Simulation: A Framework for Understanding Conversational AI in Generative Language Models

DAIL: Enhancing In-Context Learning with Data Augmentation through Language Models

RoboGen: Automating Robotic Skill Learning with GPT-4

EGS Framework Enhances Communication Using Large Language Models

Exploring the Threat of Embedding Space Attacks on Large Language Models

Evaluating Large Language Models: A Comprehensive Survey

Evaluating Language Models with SKILL-MIX: A Novel Approach

Instilling Human Values and Moral Codes in AI Systems

Open-Source AI: Fostering Collaboration and Responsibility

JudgeLM: Scalable Language Models for Evaluating Large Language Models

Achieving Human-Like Systematic Generalization in Neural Networks through Meta-Learning

Woodpecker: Correcting Hallucinations in Multimodal Large Language Models

Trending Stories

Latest AI News