Course: Design of Educational Games - Spring 2017
 

Role: Learning Engineer, Programmer

Tools: GameMaker Studio 2

Methods: Cognitive task analysisInterviews, paper prototyping, play testing

I've long been a game enthusiast, and this is my first attempt at making an educational ​game. If you are familiar with the brilliant game Papers, Pleaseyou may notice a striking resemblance! The underlying mechanics of Papers, Please seemed perfect for my game idea. I had limited time in class and so I translated many of these mechanics into my game, Dungeon Data. The game largely revolves around inspecting documents for errors based on definitions and principles found in an exploratory data analysis course. The time limitations clearly show in the upper screen that was never completed. Additionally, some of the assets are taken directly from the publicly available beta for modding Papers, Please. 

Aesthetics
Dungeon Data focuses on three core aesthetics from the MDA framework.

1. Fantasy: The game takes place in a fictional dungeon run by the dungeon master. The player is a low-level grunt who has been hired to work in the research and development department of the dungeon. Part of the job is to make sure the pilot study of other monsters is deemed acceptable. Furthermore, while not fully implemented, the player must also manage their office and staff to keep the operation running. Players hear stories and quips from the other monsters that build an imaginary world outside of the office. A desired, but not implemented feature also included orc uprisings and human invasions to build some depth to the world that the player resides in.

2. Narrative: Underlying the game is a story found in the discussions with other monsters. The story in the game as-is does not contain much depth due to time constraints. The beginnings of an orc uprising are mentioned in passing, as well as human invasions. Furthermore, the player’s actions in response to a monster’s dialog may trigger various events that are reflected in the daily newspaper. For example, a ghost gala is mentioned in the first level where a monster makes a vague threat. The newspaper the next day reflects a disaster occurring at the gala. The story is told primarily through references to outside events. Ideally, player actions could take them down various paths and story-lines adding replay value to the game.

3. Challenge: The core aesthetic of the game comes from the challenge of checking documents for errors. Incorrect actions are punishable with fines and citations. In its current state the game primarily focuses on error feedback. The full intention was to also include a quota that the student must fulfill successfully in the day rather than imposing a time limit on the error checking. If the student has not had enough success within a particular number of documents, then they must work over time which affects their office maintenance. Game over states may arise from further failure during overtime, the office eventually falling apart, or other story elements.

Mechanics

The mechanics of the game are based primarily on handling documents. The player has access to two manuals and the research documents that monsters can give them. The documents are draggable and can be placed either in the booth or examined on the desk. Players call in monsters from a queue one at a time. Ideally the documents produced are randomly generated, or at least the errors and correct responses are randomly assigned which is easier than randomly generating text. Currently, the answers and sequence are fixed. After examining the documents the player makes a decision on whether to accept the document as correct, or reject it as flawed using a stamp. After handing the document back they can call in the next monster. Incorrect responses can lead to penalties after the third error. Players may choose to incorrectly mark documents for story or moral reasons such as helping out a monster in need. After a level is complete the game goes to an office management phase where money earned can be applied to maintenance, worker wages, and upgrades. This feature is not currently implemented.


One major mechanic that was desired but left out for time reasons was to require players to provide an explanation when they marked documents as reject. The idea was to have the rules be clickable in an inspection mode. This would allow the player to highlight the part of the document that has an error and provide a rationale for that error from the rule book. This is a fairly complicated addition that didn’t make it into the final game but is planned for future development.

Dynamics

The dynamics of the game are largely tied to the challenge aesthetic. Players want to make assessments quickly and accurately. Taking out time pressure, which was an early mechanic, reduces the chance of players possibly using a strategy for speed at the cost of thoroughly investigating documents. Scanning for discrepancies is still a major strategy and can lead to mistakes if the player is not careful. This is apparent in the third level which introduces a check on whether the written variable matches the document. Quick scans of instructions and documents led some players to not understand why they made a mistake. The quick scan strategy is the biggest challenge within maintaining the aesthetics of the game.

The narrative provides some possible dynamics that are unintended. The story is interesting and driven by decisions regardless of whether your decision is correct. This might put forth some incentive to see other story lines at the expense of answering problems correctly. This isn’t so much of a problem for learning because the student may know where and why they made purposeful mistakes. The real challenge is in accurately predicting mastery if the game promotes the possibility for purposeful mistakes.

Educational Objectives

Given the results of a CTA with novices the game focused on the following educational objectives:

1. Examining general features of the data: While the players don’t look at raw data, they still need to examine the variables and accompanying research questions. A core task  was recognizing that the description of the data matched what the researcher recorded as their goal.


2. Identifying relevant variables: Similar to the previous objective, participants need to recognize that the variables are appropriately labeled. This does not mean their type, but that the monsters selected the appropriate variables for their study among the possibilities. Within the game the monsters have a document that states the variables they are investigating. The player must then recognize whether the stated variables in a supplementary document matches the research question in the research document.


3. Identify variable types: This asks whether the player can distinguish categorical from quantitative variables. As a task, the player examines the supplementary document provided by the monster and checks whether the variable type listed matches the type of variables described in the research document.


4. Identify study type: This asks whether the player can distinguish exploratory studies from observational studies. As a task, the supplementary document provides the monsters stated version of what kind of study they are conducting. Comparing this information with the problem description in the research document provides the source for answering whether the stated study type matches the description of the problem

Learning Principles

1. Sufficient practice: This is yet another ideal explanation of the game that is not quite implemented in its current state. Ideally, the game tracks and predicts student mastery of topics. This could lead to some issues with making predictions about student knowledge based on purposeful errors. One way around this problem is to only track errors on a first play-through and then after the student has mastered the material they are free to play around in a level and see various story lines. That is, a first play through may be a set story where correctness is important, and after completing the game they may explore various branching story lines. The use of mastery learning can make it so that new levels cannot be unlocked until previous skills on a particular level have been completed with a sufficient amount of success.

2. Feedback with Explanations: The game handles feedback by handing out citations whenever an incorrect response is made. The first two errors are warnings, and the third  penalizes money which is used in the office management part of the game. The citations provide explanations for why errors occurred allowing players to reflect on the mistake that they made. 

3. Self-explanations: A core learning objective desired but not ultimately achieved was to have students provide explanations for their assessments. If implemented, the students would use the rule book or statistics manual to highlight the rule or definition that is being violated. They would then also highlight that part of the document where the discrepancy exists. This form of self-explanation may help students reflect more deeply on their understanding of the material.

4. Worked Examples: Worked examples may help the student reduce cognitive load when they lack mastery of the material. All of the documents that monsters provide are essentially worked examples with potential mistakes. The players aren’t asked to do any computations or analyses themselves. They just need to examine worked examples. The trick there is in noting that they are worked examples with mistakes, so there is an additional step in that the players have to analyze the example. Even with this step, giving the research problems in completed form helps reduce the load of generating the material themselves. Additionally, the complexity of the examples increases as mastery gained so as to not overwhelm students with the analysis task.

5. Increasing Complexity and Segmentation: The game does not throw players into the deep end. The educational objectives could be seen as a sequence of steps when performing exploratory data analysis. The students do not work on future steps until they have mastered the current step. Each level adds one or two new pieces which helps maintain the challenge aesthetic while not overwhelming the student. This objective can also be built into the narrative aesthetic. Perhaps too many mistakes in the monster’s research led to new requirements that you have to check. Other possibilities include promotions to higher positions with more responsibilities, or disasters occurring because nobody was checking the implications of a study. 


6. Hints: Hints are partially implanted in the current version of the game. I was still considering whether it was worth implementing a hint button that may lead to a monster giving you some advice. This advice would largely revolve around pointing the student to the
appropriate place in the statistics guide to find the help they need. The current implementation includes these instructions in a document that introduces the goals of each level. It simply states that if you are having difficulty understanding the material you can use the guidebooks available to search for information. This information is primarily in the form of definitions given the simple learning objectives. They are limited to a few sentences, but I can imagine adding greater depth and examples in the descriptions.

Dungeon Data

A document simulator for teaching introductory statistics