Decoding Human-Machine Interactions: A Curated Dataset for Adaptive Human-Machine Interfaces

By Dr Silpaja Chandrasekar, PhDReviewed by Susha Cheriyedath, M.Sc.Nov 30 2023

In a paper published in the journal Scientific Data, researchers presented a curated dataset of human-machine interactions gathered through a specialized application employing formally defined User Interfaces (UIs), aims to decode user behavior and advance adaptive Human-Machine Interfaces (HMIs).

The dataset underwent rigorous processing, ensuring cleanliness and completeness, alongside a thorough data profiling analysis. Researchers generously included access to the collection code, data profiling procedures, and usage guidance for crafting adaptive UIs, catering to professionals and data analysts delving into HMI  research and development.

Background

Combining manufacturers' focus on intuitive task completion and anticipating user needs, alongside understanding user motivations like autonomy and competence, fuels the drive toward enhanced User Experience (UX) within HMIs. This amalgamation enables the creation of interfaces that prioritize seamless interactions and dynamically adapt to users' evolving requirements, leveraging artificial intelligence techniques to elevate user satisfaction and overall efficiency in HMI interactions.

Data Collection and Analysis Methodology

The data collection process involved an industrial mixing machine from the food sector used by multiple operators across shifts for mixture creation. Interactions adjusted parameters influencing the final product. Operators accessed a mobile app via a Quick Response (QR) code, recording sequence steps and time intervals.

Twenty-seven operators, aged 23 to 45, consented to record interactions. Over 151 days, 10,608 interactions were captured, anonymized, and devoid of sensitive information.

Data acquisition followed a methodology starting with UI description in a UIDL format, representing the entire HMI through JSON. The HMI, built using Next.js and Chakra UI, was responsive and tailored for tactile devices. The researchers modeled an interaction process as a Finite State Machine (FSM), outlining the adjustment of parameters in a mixture creation process.

During active experiment phases, non-intrusive capturing stored interactions in a database, recording user identity, EPOCH timestamps, and interacted element identification. Following the definition of event sequences, the subsequent data processing aimed to generate valid interaction sequences for each user by employing a specific algorithm. Out of 10,608 interactions, the system generated 1358 good sequences, each marked by particular initiation and conclusion events determined in the FSM.

This rigorous methodology encompassed diverse stages, from experimental setup and UI description to interaction capture and sequence extraction, yielding a comprehensive set of 1358 valid interaction sequences for further analysis and interpretation.

Dataset Validation, Analysis, and Usability

Technical Validation: The dataset underwent meticulous validation to ensure its consistency, completeness, and suitability for analysis. It involved cleaning the data by removing duplicates and rectifying errors. After processing the data, researchers conducted a comprehensive data profiling analysis to validate the resulting dataset. This validation aimed to ascertain the dataset's consistency by confirming that interaction elements corresponded to those in the UI JavaScript Object Notation (JSON) file and that users' IDs existed in the users.csv file.

Moreover, researchers used the International Business Machines Application Programming Interface (IBM API) of Data Quality for Artificial Intelligence (AI) to assess the dataset's quality quantitatively.

This toolkit provides a range of quality estimation metrics, assigning a score between zero and one to identify data issues, with one indicating no problems detected. Key metrics offered insights into the dataset's quality and fitness for various analyses.

Distribution Analysis: Researchers scrutinized the distribution of sequences across different machine services, users, and periods. Notably, the analysis revealed varied engagement levels among services and users. Density-based clustering algorithms were employed to analyze the hourly distribution of interactions for operators, identifying clusters representing intervals of frequent interactions.

Insights and Usability: Insights drawn from the distribution analysis can aid data scientists in leveraging the dataset for applications like recommendation systems or adaptive user interfaces. Understanding user behavior across services, users, and timeframes becomes pivotal for model development or system enhancements.

Usage Notes: The dataset facilitates the development of Adaptive UI (AUIs). Critical dimensions for AUI design are linked to dataset fields, emphasizing the dataset's reusability. Defining adaptation goals—enhancing performance or providing instructional support—guides the adaptation techniques. The research focused on improving performance by identifying recurring interaction patterns and time intervals, resulting in a reduction of over 40% in operator interaction time. A simplified JSON-based interface descriptor further enhances the dataset's usability, streamlining modifications for generating adaptations.

Conclusion

To sum up, the dataset underwent rigorous validation, ensuring its integrity and readiness for analysis. Researchers ensured the dataset's consistency and suitability by employing meticulous cleaning processes and leveraging quantitative assessments using the IBM API for AI. Distribution analysis highlighted varying engagement levels across machine services and users, pivotal insights for diverse applications such as recommendation systems or adaptive user interfaces.

Understanding user behavior through this dataset becomes a cornerstone for enhancing models or systems. Its usability and critical dimensions for AUI design emphasize its potential for performance enhancements and instructional support, exemplified by a 40% reduction in operator interaction time. This dataset, accompanied by a simplified JSON-based interface descriptor, offers a versatile resource for further research and adaptation techniques.