Predictive Models Illuminate Wordle Gameplay Patterns and Word Difficulty

By Aryaman PattnayakReviewed by Susha Cheriyedath, M.Sc.Sep 29 2023

In a study submitted to the arxiv* server, researchers developed mathematical models to predict the number of Wordle game results submitted on a given date and the probability distribution of guesses for a specific word.

*Important notice: arXiv publishes preliminary scientific reports that are not peer-reviewed and, therefore, should not be regarded as definitive, used to guide development decisions, or treated as established information in the field of artificial intelligence research.

The popular online Wordle game involves guessing a secret 5-letter word in as few attempts as possible based on colored letter feedback. The researchers obtained the daily Wordle results submitted from January to December 2022.  They found an initial surge in popularity during January and February 2022. They modeled only the declining period from March onwards to focus on the stable player base. The AutoRegressive Integrated Moving Average Model (ARIMA) showed lower error after comparing exponential decay and other ARIMA models.

The study employs the ARIMA (9,0,2) model, meaning the current value depends on the previous nine days, with no differencing for stationarity and two days of moving average error terms. Diagnostic tests validated the model parameters.  The ARIMAX model incorporates a binary weekday/weekend indicator variable to improve accuracy. Weekends showed 4.59% fewer average results. This boosted performance and dropped the error rate. Using the fitted ARIMAX model, the predicted number of Wordle results on March 1, 2023, is 12,884. The mean absolute error on test data is 664, indicating reasonable precision.

Modeling Word Difficulty

The study also develops a neural network model to predict the probability distribution of guess counts for a given word. The percentage of players guessing the word in one try, two tries, etc., indicates difficulty.

They initially hypothesized that word attributes like letter frequency affect hard mode use. However, the data showed minimal correlation, likely due to stable player proportions. The neural network model employed backpropagation learning on features like letter frequencies and vowels/consonants. Performance improved significantly after expanding the input features. 

The model predicted probability distributions on test data close to the actual outcomes. The error rate was around 21%, mainly from inaccuracies predicting one and 6+ guess counts. Applied to the word "eerie" on March 1, 2023, the model forecasts an average of 4.8 guesses. This aligns with the word's unusual letters with repeated vowels.

Using Machine Learning

The study first averaged guess counts to categorize word difficulty as a metric. They improved this by clustering words into five levels using the K-means algorithm. Model selection methods optimized five clusters. The clusters identified cut-offs at 3.59, 3.97, 4.28, 4.59, and 5 guess averages from easiest to most difficult. "Eerie" falls in the most challenging 5.0 level with its 4.8 guess average.

The researchers extracted each level's average features. More difficult levels exhibited rare letters, fewer vowels, more consonants, and fewer unique characters. This confirms correlations between attributes and difficulty.

Evaluating Model Robustness

Multiple methods assessed the models' robustness. The ARIMAX model showed a low mean absolute error on test data. Cross-validation also achieved reasonable performance from the neural network model.

A sensitivity analysis simulated the impact of a sudden increase in poor Wordle players. The ARIMAX model proved resilient, slightly changing the March 1 prediction to 12,194. Overall, the suite of models provides a practical toolkit for dissecting Wordle gameplay and forecasting outcomes. Limitations include small sample sizes and difficulty quantifying some linguistic features.

Interesting Data Insights

Analyzing the dataset revealed intriguing Wordle player patterns. The proportion attempting hard mode climbed then stabilized at around 8% of loyal fans. This core group enjoys the daily challenge. The models facilitate anticipating Wordle results based on date or word attributes. While imperfect, they offer helpful analytical foundations for future work. Their novel application of techniques like clustering demonstrates the value of machine learning for gaming language data.

With widespread everyday use, Wordle represents a cultural phenomenon. As well as prediction, optimizing enjoyment for players requires understanding motivations. Models can supplement, but not replace, the human delight in puzzles. The researchers' work provides the first steps in this data-driven exploration of gaming psychology.

Future Outlook

The Wordle prediction models presented offer functional foundations built upon several promising research directions:

•  The models can be expanded and refined as more gameplay data becomes available. Fitting to larger datasets will likely improve accuracy and allow more flexibility in techniques like neural network architectures. Capturing evolving player patterns will require continuous model updates.
• Further feature engineering can uncover linguistic attributes that correlate with difficulty. The study identified promising trends, but many intuitive features like letter positions still need to be explored. Advances in representing language data can boost model performance.
• Combining the prediction models can offer more holistic insights into gameplay. Using data forecasts and word difficulty categories together could reveal temporal patterns in game design. Integrative modeling will deliver enhanced analytics.
• Player demographics and motivations should be investigated to complement predictive modeling. Surveys and interviews can uncover why core users persist at Wordle daily. Understanding human factors will assist designers in optimizing enjoyment.
• The models' capabilities can be expanded to other gaming contexts. Wordle's simplicity offers an ideal testbed for developing forecasting methods. Applying similar approaches to complex games could enable user assistance, automated game testing, and design improvements.

While the current study focused on prediction, future work should also explore prescriptive insights from the models. Algorithmic recommendations could make Wordle more engaging and accessible to wider audiences. Ultimately, modeling should enhance human creativity rather than replace it.

*Important notice: arXiv publishes preliminary scientific reports that are not peer-reviewed and, therefore, should not be regarded as definitive, used to guide development decisions, or treated as established information in the field of artificial intelligence research.