Digital Serious Games (DSG) and Game-Based Learning (GBL) receive increasing highlight by being aligned to the new generations’ characteristics, the called digital natives, such as connectivity, speed and random cognitive processing (Prensky, 2012) . As (Ryoo, 2011) points out, games and GBL employ a narrative designed to engage students in their own learning.
Research shows that GBL and principles of gamification are efficient to improve student motivation and learning outcomes, being more often taken into consideration in the creation of modern Virtual Learning Environments (VLE) (Stefan, 2015) . Video games are participatory and experimentation spaces that invite players to live avatars with individuals and different capacities.
However, feelings of despondency for persistence in the GBL approach can occur due to lack of student’s relation with the game. In this perspective, elements that can help support ubiquitous learning (u-learning), by promoting learner autonomy and the integration with its context (Barbosa, 2007) can be helpful.
Nowadays the Virtual Worlds (VW) are being increasingly used for numerous purposes, such as providing motivation and enhancing users’ learning in educational environments. The VW simulates the real world, with 3D objects, avatars and scenarios, with creativity possibilities, such as flying and touching from distance. Features such as the possibility of collaboration and student active participation, encouraging exploration and problem solving, are being offered through these virtual environments (Ávila, 2013) . As examples of VW platforms there are Second Life (SL), OpenSimulator (OpenSim) and Open Wonderland. According to (Silva, 2012) most of the VW initiatives used in education occur in SL, which is a stable environment with many features. However, it is a proprietary platform that requires financial investment for use. OpenSim is an option to SL, compatible with both structural and resources features, which is free of charge.
In this scenario, the objective of this research is to present possibilities of motivational influences due to a GBL approach on Computer Science formal higher education, in the context of Computer Networks’ (CN) discipline, through the development and application of a sensitive to student’s context DSG, integrated into a 3D VW built on OpenSim platform. It provides a personalized game play profile to the students’ Cognitive Style (CS) and Expertise Level (EL), focusing on checking how their perception about the approach was.
In the following sections, there presented: the related work on Section 2; JASPION game on Section 3; forms of assessment for DSG/GBL approaches on Section 4. On Section 5 it is performed data analysis and interpretation of results, and finally, Section 6 presents the conclusions and the final considerations of this study.
2. Related Work
To show the relevance and contextualize this research we sought for studies that also developed and applied DSG in our scope, in order to strengthen the foundation, the motivation and the justification for the study application.
The related works were divided into two groups: a) games about Computer Networks and b) games developed in Virtual Worlds. On sub item c) is showed the JASPION’s differential.
2.1. Games about Computer Networks
CyberCIEGE was developed in Tybolt Engine by the Naval Postgraduate School from the United States. The game is based on exploring and teaching network security concepts through 20 scenarios, each describing a different concept. It takes place in a 3D environment with users, systems, and real-time economic engine. As (Irvine, 2005) affirms, the aim of the game is to keep users happy and productive in a virtual organization while security measures to protect valuable organizational information assets are adopted.
2.2. Games Developed on Virtual Worlds
JETS was developed in OpenSim and integrated into the VLE MOODLE by SLOODLE plugin. The game simulates the software testing industry in a systems’ development company. Through the questionnaire tool (available on MOODLE) the teacher can edit the challenges of the phases. In the game is discussed the software testing strategies topic (Silva, 2012) .
Asthma, also developed in OpenSim, by (Gomes, 2013) , deals with the health problem of asthma, aiming to impart knowledge about security procedures, as some of the basic steps to take at home to prevent the causes of asthma attacks. The game was developed for primary school children and is composed of questions, in which are given four possible answers. For each correct answer the player receives a word that at the end of the game can be used to form a sentence about the subject, as an incentive for trying to answer all questions.
GrAFiCA, developed in SL by (Terzidou, 2012) , comes from the acronym: Grouping and selecting, Attire, Find, Collaborate and Answer, which also indicates the phases of the game. Players form groups, where each one corresponds to a particular color and an avatar costume to be easily differentiated, aiming to support internal group interactions during the game. The game is also made of questions, in multiple-choice format, and is hidden randomly in virtual objects that decorate the themed buildings in the game space. The main objective is to collect as many points as possible during a specific period of time, answering knowledge questions. The winner is the group with the highest score, determined by the sum of correct answers given by its members during the game.
2.3. Differential of JASPION
This research aims to explore the existence of motivational signs in students through the development and implementation of a ubiquitous serious game for CN learning, wrapped in a narrative as CyberCIEGE, integrated into a 3D VW as GrAFiCA, and using the OpenSim platform as JETS and Asthma. The differential of JASPION, while incorporating the two strands (games for CN education and games developed in VW), is the ubiquity aspect, through the collection and treatment of student’s context information (cognitive style and level of expertise), thus offering a personalized gaming experience.
3. JASPION?a Game in a Virtual World Sensitive to Student’s Context
According to (Savi, 2008) the past 30 years’ generations were born and grew up in a world surrounded by digital technologies, being accustomed from childhood to surf in cyberspace, with a hypermedia and interactive language, making it difficult to adapt them to the traditional twentieth century education system. (Prado, 2015) states that the information flow received by technological means and this “constant connection” is linked to a significant change on concentration capacity of the current generation.
Soflano (2015) points out that there seems to be genuine advantages from adaptation of teaching materials for different ways of learning, and so it might be possible that GBL applications tailored to individual characteristics can improve learning outcomes. Thus, customization based on learner profile, its knowledge, the context that surrounds it and how it relates to the environment becomes a differential.
So we start from the presumption that is possible to improve motivation, and therefore the student’s interest, in a dynamic courseware (educational game), which is adapted to its CS and EL, offering digital pedagogic resources aligned to its reality.
In this sense, the game proposed in this study seeks to check students’ motivational promotion and interest about the CN subject matter. Ubiquity is handled by the game’s pervasive access (through internet by various computing devices) and adaptation to the user’s context.
Context issues addressed in this study (CS and EL) are presented as follows.
3.1. Cognitive Style Context
Although the significant changes observed on digital natives’ Cognitive Style, such as processing information at higher speed, ease of performing many tasks in parallel, preferably graphic instead of texts and random access to sequential; besides being extremely connected, active and fluent in several technologies (Prensky, 2012) , it is important to consider individual aspects of each student, following the assumptions of new educational paradigms and u-learning.
CS can be understood as relatively stable forms regarding characteristics of the cognitive structure of a person, defined according to (Bariani, 1998) by biological factors, influenced by culture and modified from direct or indirect influence of new events.
The research of (Franciscato, 2010) , among other contributions, identified four predominant CS among its participants: Divergent, Holistic, Reflective and Serialist. This parameter and its instrument used to identify the CS is considered for adaptation to student’s CS on JASPION. The game adaptation occurs through Pedagogical Supporting Materials (PSM) presented on the phases, having as parameter the resources’ format available on the VLE MOODLE version 1.9 analyzed on the mentioned work, as follows: text, online text, image, video, and slides.
3.2. Expertise Level Context
According to (Possobom, 2014) the EL refers to one’s knowledge scale on a given area, taking into account the expertise that it already has or not. This meets the problems faced by students on VLE where content is standardized for all users, discouraging them to perform activities that are not in accordance with their level of knowledge.
(Hunicke, 2004) asserts that video games can be discouraging when too easy and, on the contrary, frustrating when too difficult. While some games allow players to adjust the difficulty (easy, medium, hard), its overall challenge level is often static in front of the player input. This lack of flexibility can lead to imbalances between the player’s ability and the game difficulty.
(Possobom, 2014) in her research adapted a VLE according to student’s expertise level on Computer Networks. The instrument used by the author is integrated to the one mentioned above (Franciscato, 2010) forming the questionnaire that identifies user game profile on JASPION.
3.3. Game Narrative
The origin of the name (JASPION) alludes the main character, inspired by the popular Japanese TV series “The Fantastic JASPION”. Analogously to the series, the plot revolves around the hero character named JASPION, which is sent to “Network Planet” to unravel mysteries and puzzles and protect the Earth from the threat of Satan Goss.
Thus, the game has the narrative distinction, with the creation of a fictional story to involve the student in an adventure with suspense. The player is challenged with a series of quizzes with themes related CN area, where to address them, choosing the correct answer, it can proceed to the next stage.
The challenges corresponding to the game phases were created in the shape of objective questions with five answer options and were extracted from free online databases of Brazilian’s public concurrences admissions. The questions were mapped according to CN discipline of Computer Science’s degree from Federal University of Santa Maria, Brazil, with the regent class teacher support. It was also related the compatibility issue with each of the three difficulty levels, aiming to match student’s expertise context.
3.4. Game Design and Mechanics
The game starts with the student logging into the VW. In the first view it finds a spacecraft and other images inserted on totems alluding the story, and watch an introductory one minute video, which tells the game story. The student is orient- ed to continue walking with its avatar till it finds a questionnaire to respond, within the VW itself.
Taking into consideration that users have three levels of expertise (Possobom, 2014) : Basic 1) Intermediate 2) and Advanced 3); and four classifications of Cognitive Style (Franciscato, 2010) : 1) Serialist, 2) Holistic, 3) Reflective and 4) Divergent; it was defined 12 gameplay profiles, that arise from the intersection of these two classifications. The questionnaire replied by the student defines its gameplay profile. So, after submitting the questionnaire, in a few seconds the user’s avatar is automatically redirected to its ideal game. That is possible by the existence of a sensor inserted in a nearby 3D virtual object, which makes a periodic scan of the avatar’s presence and with the result (gameplay profile) “teleports it” to the custom game.
At each phase the player’s avatar enters a room, receives a question and five response options, which may be presented in one or five “doors” (3D object with a texture that shows an alternative(s) to answer the question, and blocks the access to a tunnel that leads to the next phase). To decide for one of the alternatives, the player clicks on the desired option or directs its avatar in front of the door that contains it, depending on the type of question (multiple or single door). Examples of JASPION’s phases are showed in Figure 1.
(Stefan, 2015) affirms that motivation is stimulated by various incentives such as public recognition and periodic rewards. In this sense, users are encouraged to accomplishment by receiving a measure of their progress on JASPION, and if the player’s answer is incorrect, a negative feedback text is generated in the form of dialogue from Satan Goss, a NPC (Non Player Character). Similarly, when responding correctly, it generates a positive textual feedback.
Altogether there are 19 phases in the game (six for each difficulty level and the final stage), always accordingly to the user CS. When solving the six sequential level phases the student receives a notification that successfully finished it (the level) and is directed to the next level (containing six more phases). However, when solving the six stages of the last level (advanced), the player is taken to the final phase, which contains a more complex challenge.
In this path, student finds several supporting materials aimed at reaffirming the immersion in the game’s context and at supporting phase’s resolution, available for consultation as student’s will. These supporting materials are divided into: Game Supporting Materials (GSM), that seek to immerse the player on the story and make the game self-instructive giving tips on how to move
Figure 1. Examples of JASPION’s phases with one door (left) and with multiple doors (right).
around and where to go, totalizing six items; and Pedagogical Supporting Materials (PSM), which are complementary reading and didactic materials, aiming to help the student to meet the challenges. There was at least two PSM available on each phase, totalizing more than 38 items.
It is always available an option to leave (quit) the game at any time. The user’s last position in the VW (checkpoint) is registered on the database, allowing it to resume the phase where left off when accessing it again. Thus, the student is empowered to pause the approach and is free to continue it when having available time or interest.
Reinforcing the gamification characteristic, with every login the player gets three “lives”, a symbol (a transparent object with three green circles attached on avatars’ back) representing the number of chances it has to miss the phase answer without being kicked out of the VW. That’s also possible to see in Figure 1. The user who loses these three chances is automatically logged out, receiving the “game over” message on screen. However, it is free to start from the last checkpoint or give up the approach.
4. Ways of DSG and GBL Approaches’ Assessment
As ways of DSG evaluation it’s observed, according to (Marascin, 2013) , that many approaches use questionnaires, taking the participants’ answers in a direct relation to its conduct. That is, prioritize declarative aspects and are located in an exterior to the game. Few studies use methods that follow the playing operation procedures itself.
The assessment of an educational technology, such as DSG, can consider several criteria. Among them, (Laguardia, 2007: p. 2) mention “the didactic use, cognitive possibilities offered, the technical and aesthetic aspects to adhere that its users, economic profitability and ergonomic design.”
In the systematic mapping carried out in the survey of (Krassmann, 2015) it was observed two main evaluation focuses used on DSG that have similar scope to JASPION, which are summarized in the following paragraphs.
1. Analysis of cognitive advances: seeks to represent the learning elapsed with the approach, noting parameters before and after the game use, intending to check its influence as a pedagogical tool (Zhang, 2013; Raman, 2014; Potter, 2014) . In that research this evaluation axis was denominated “Pedagogical Vision”.
2. Analysis of acceptability/applicability of the GBL/DSG approach: seeks to represent student’s opinion about the game, that is, analyze the approach impact within its users (Rodríguez-Cerezo, 2014; Adamo-Villani, 2013; Terzidou, 2012) . In that research this evaluation axis was called “Student’s Vision”.
However, it is noted that the learning scope’s assessment (axis 1-Pedagogical Vision), usually by tests’ application before and after the game use (pre and post- tests) can generate threats to validity, as faced by other researches in the area, such as the works of (Rausis, 2011) and (Savi, 2011) . The first highlights the lack of a base about the real students’ knowledge before and after the game application, which can hinder a better comparison of differences in results. “There is also the fact that the game has been executed with a small group of participants within the same department, and the same instructor, reducing the possibility of generalizing the results” (Rausis. 2011: p. 52) . The second points out that the ability of the teacher who creates pre and post-tests can interfere in the instruments’ quality and therefore influence the measurement result. “Deviations in the results may also occur if pre-test and post-test are very similar” (Savi, 2011: p. 201) .
Given the above, the Pedagogical Vision, elucidated as one of the main DSG’s evaluation axis, was disregarded in this work, in order to avoid the impasses mentioned. Thus, it was decided to adhere to the second assessment main focus: analysis of acceptability/applicability of GBL/DSG approach―Student’s Vision, considered feasible to meet the research objectives.
With this focus we intend to obtain the audience’s perception, analyzing possible motivation evidences by measuring student’s involvement with the DSG approach, starting from the premise that “not enough that a game is didactically suitable and promote learning, it also need to be able to motivate students to study and provide a good experience” (Savi, 2011: p. 100) . (Sweetser, 2005) points out that if the players do not like the game they will not play it.
(Aarseth, 2003) affirms that there are three main ways to acquire knowledge about a game: a) design, rules and mechanics’ study; b) watch the users playing and read their reports and comments; or c) the researcher itself play the game.
Based on item b suggestion (Aarseth, 2003) , JASPION’s evaluation observes the target audience playing, using Human-Computer Interaction’s (HCI) evaluation techniques, through computer screen video recording of user’s interaction with the system, on the assumption that “not always users realize or can express their experience with the system” (Preece, 2015) . We also recover the user’s feedback about the game, using an adaptation of the questionnaire presented by (Savi, 2011) .
Thus, this research deploys a hybrid DSG’s evaluation method, applying it experimentally on JASPION. The combination of two instruments intends to achieve results’ triangulation, strengthening the assessment’s study credibility. Triangulation in evaluation is defined by (Laguardia, 2007: p. 3) as “the use of multiple data sources, observers, methods or theories in the investigation of the same phenomenon, supporting a finding with the help of others.”
5. Data Analysis Method
With the regent class teacher’s consent, Computer Science graduation students were invited to participate in the experiment, that didn’t offer any curriculum reward. Due to low adherence of these students (only two), it was opened space for students of other courses and institutions of Computer Science education levels. So, 16 more students applied, most of them members of the authors’ research group.
After this procedure, an invitation by e-mail was sent to each one of the recruited, scheduling individually the experiment date, which was supervised by the authors.
It’s noteworthy that the Computer Networks discipline curriculum of Computer Science course used as basis for JASPION’s phases preparation is similar or had an equivalent discipline also in the other courses that were part of students’ sample. Also, we took care not to recruit graduation participants who had not attended yet the CN discipline or its equivalent, making it possible to count on with a consistent sample. Thus, the experiment had an audience of 18 participants, nine graduation students and nine post-graduation students.
The assessment instruments used in this research are presented below.
We start from the assumption that many DSG evaluation studies’ have focused on games effects in learning, comparing the obtained notes on pre-tests with post-tests, disregarding important aspects like fun, motivation and challenge.
This instrument is an adaptation of the questionnaire shown in the research of (Savi, 2011) . With its three subcomponents (1: Motivation, using ARCS Model (Keller 1983) ; 2: User Experience, using basically EGame Flow (Fu, 2009) ; 3: Learning, using Bloom’s Taxonomy (Bloom, 1956) , this evaluation model has 29 closed questions.
However, to evaluate JASPION were used only the first two subcomponents: Motivation and User Experience, due to learning not being focus on this research, for the reasons outlined in the beginning of third section, involving the difficulty and subjectivity on learning evaluation.
In addition, from the User Experience subcomponent were disregarded three questions that correspond to social interaction evaluation, due to the game being on single-player format and do not directly provide this type of interaction.
Thus, the adjustments made on the instrument of (Savi, 2011) resulted in 23 closed questions, using (as the author) Likert (1932) scale as response parameter, with five alternatives: Strongly Disagree, Disagree, Neutral, Agree and Strongly Agree. It was also included a 24th open question, requesting the student’s opinion about the game and the approach, and a diagnostic question seeking to identify the user’s familiarity level with digital games: “How much is your familiarity with digital games (computer games, mobile phone, video games)?”. It aimed at verifying a possible correlation of this aspect with student’s vision about the approach and had six options of choice, being: a) No familiarity (never played games); b) Little familiarity (less than one time a month; c) Regular familiarity (more than one time a month); d)Good familiarity (more than one time a week); e) High familiarity (more than three times a week); f) Much familiarity (I play almost every day).
Instrument 2―Interaction data
This assessment instrument is an observation and analysis of the user’s route(s) within the VW while playing JASPION, checking data from its interaction with the game and with the overall ubiquitous game approach.
Transparent to student, an audio-visual capture and recording of the computers’ screen used in the experiment is done, using the free application OCam, enabling further and posterior analysis.
To establish the interaction items to be evaluated were taken as reference examples of observation and metrics techniques used in usability testing for HCI, as total and average time to perform tasks, number of hits and misses, and persistence after an error (Preece, 2015) . The items were grouped according to its similarity, namely: general data, pre-game, game and resource view, and can be seen on section 6.4, Tables 2-4.
6. Results’ Analysis and Interpretation
This section presents data obtained through the research’s evaluation, analyzing the DSG approach’s impact within the students and its possible motivational influences. For this, correlations are made between the data found, seeking for patterns.
6.1. Sample Profile
The analysis starts with the sample’s profile, which had 18 students. Five participants were identified with gameplay profile EL 2 and Divergent CS, which was also the cognitive style that received more users. Nevertheless, most students received the difficulty EL 1 (10 players). The CS with less participants was the Holistic (only two). Cognitive styles Reflexive and Serialist counted with four and three participants, respectively. The EL 3 had only one player. Table 1 summarizes the sample profile.
The sample revealed to be partitioned between ones having good or more familiarity (plays more than once a week), accounting for 10 participants (just over half), and ones having regular or little familiarity (plays more or less than once a month), with 8 participants. No participant claimed not to have no familiarity with digital games.
In order to optimize the analysis we established two groups of users: gamers (who play more than once a week) and non-gamers (who play less than once a week). It is true that in this sample of participants there are no individuals who reported having no familiarity with digital games, making it impossible to call them non-gamers. However, this denomination serves to distinguish individuals who are not as assiduous players from the really assiduous ones on the inferences made in the results.
Table 1. Sample profile of JASPION’s participants.
6.2. Experiment Execution
After the last game session (for those who have made more than one attempt), the participant was asked to evaluate the approach submitted by answering the evaluation questionnaire-instrument 1. For this activity, we stimulated the student to reflect about it so it could report, sincerely and spontaneously, its perceptions about the game, whether they were good or bad.
6.3. Results of Instrument 1―Questionnaire
Aiming to illustrate the questionnaire’s consistency, validating it as an evaluative tool, it was calculated its Cronbach’s Alpha, resulting in a value of 0.896. This suggests a good degree of consistency, allowing to consider it a good tool for JASPION analysis, according to Cronbach’s Alpha coefficient rating scale of (George, 2003) .
The overall average score of the questionnaire’s 23 closed items, accounting data of all participants, ranged between 3.44 and 4.66 points per item, i.e. between the concepts “Neutral” and “Strongly Agree” of the evaluative scale. The median value of the questionnaire was 4.73 points, between the concepts “Agree” and “Strongly Agree” (closer to the latter). These results permit infer that JASPION got a good overall rating, complying with the requisites from the two constructs that make up the questionnaire: ARCS Model (Keller, 1983) (motivation) and EGameFlow (Fu, 2009) (user experience).
Analyzing a possible impact of student’s familiarity with digital games in its evaluation, we calculated the average of the two profile groups. The overall differences between gamer and non-gamer users end up not being significant, resulting in 4.29 points to the first and 4.23 points to the latter. Being this difference so small, it can be stated that, in general, all users rated near “Agree” evaluation concept (between 4 and 5 points) allowing affirming that this aspect (game familiarity) did not directly influenced the assessment of JASPION. This finding is important to highlight how the approach was compatible with various types of gamers, be they occasional or frequent players.
For the discursive contributions’ analysis, evaluators’ observations were grouped into two axes, separating the ones related to JASPION as a digital game and as an educational tool, which are discussed in the topics below.
a) JASPION as a digital game:
According to students the game is creative, fun and interesting, the story is consistent with the characters and fit well with the kind of environment. The scenarios are engaging and thought-provoking, highlighting the importance of context to ensure greater immersion, stressing in this regard the introductory video posted in the beginning of the game.
The evaluators found interesting the avatar of Satan Goss, that remember them of Darth Vader from the famous Star Wars movie, and the issues challenging the user towards a goal (to save the planet), together with the presentation of the game itself, affirming that it were motivating factors while using the game.
Among the negative points evaluators stressed that OpenSim is not as usual as other gaming platforms, as Unity 3D, causing a little discomfort.
b) JASPION as an educational tool:
Among the positive recommendations grouped in this topic is that if the game used regularly in the classroom as an auxiliary material would be very effective, as it addresses contents in a different way from usual, but using common materials (such as slides and texts).
Evaluators also said that games are interesting in the educational field and expressed that they felt stirred by the challenge of moving a door, and therefore was not unpleasant go back and read PSM to make it possible. They stressed that, despite not knowing the answer to some questions, the “extra material” helped. No relevant negative comments about JASPION as an educational tool have appeared.
6.4. Results of Instrument 2―Interaction Data
This subsection deals with the analysis of the evaluation instrument 2 (interaction data), which explores a model inspired by techniques of observation and metrics used in HCI usability testing.
After the approach application and the video capture of user interaction with the game, it was made a manual and individual analysis of each of the files, aiming to fill the table with the interaction items (Table 2). Thus, we obtained an overview, allowing the extraction of various data that are presented in this subsection.
For this instrument’s analysis we sought to maintain a standard in order to guarantee the data accuracy. For example, it was only counted as access to supporting material (both PSM and GSM) when the player’s avatar spent more than five seconds accessing such material (standing right in front of it), preventing to include data from users who only quickly opened it to test the feature or unintentionally positioned its avatar in front of the object, without necessarily accessing it.
Table 2 provides the data of the average JASPION user, i.e. the average interaction data of all students who participated in the experiment, as well as the realization of some inferences.
It is important to highlight that item 2 refers to time due to technical difficulties faced on the experiment execution, like slow internet connection and network problems. That was included to separate it from the time of actual gaming. On the other hand, item 12 refers to errors committed by the players, i.e. incorrect answers given, missing a question.
It can be observed with the data in Table 2 the extraction of information that
Table 2. Interaction data and inferences about the average JASPION user.
can influence and spread decision-making by other DSG approaches. From the findings shown in Table 2, it is possible to highlight, for example:
a) Item 7: the average time responding to the questionnaire that defines the gameplay profile, indicating the need for adjustments due to the high time spent by users with this activity.
b) Item 8: the number of played phases, showing that, taking into account that the majority of users received the gameplay profile at EL 1 (basic), most of them managed to advance to EL 2 (intermediate), completing more than 50% of the game (10 of 19 phases), which is a good result and indicates, possibly, a students’ knowledge evolution.
c) Item 9: the number of completed phases (one less than played phases), indicating that students dropped out at the last stage played. Given the user spent on average 51 minutes with the approach, it can be inferred that this long time may have influenced the decision to give up the approach without completing the last phase in progress. It is worth noting that it does not indicate that no student has completed its last phase, as many have concluded. However, the average user did not complete the last phase played.
d) Item 16: the number of PSM accessed (equals to the average number of phases played). That’s an average of one material seen per phase, indicating that the user has made in fact a choice between the materials available for consultation, considering that there were at least two PSM on each phase.
e) Item 18: the average time visualizing each PSM (30 seconds), corroborating with findings of Prado (2015) and Carlson (2005) about the significant change on current generation’s concentration, that is, difficulty maintaining attention on something. This may be an indicative that the material available cannot be long, it is necessary to resize, for example, extensive videos and long texts to reflect this.
Through the data in Table 2 we can infer that JASPION’s three-dimensional virtual scenario enabled interactivity and respected student’s individual cognitive time, important aspects as pointed out by Ávila (2013) . It also permitted to encourage the search for more knowledge, showing that students used additional sources of study to progress in the game.
However, in order to make this analysis more efficient, some proportional data regarding the number of phases played were calculated. For this, users who have played the same number of phases were grouped to calculate the group’s average.
Thus, it was observed that the group who played the lowest number of phases (only three), accessed the higher average number of GSM per phase (1.7) and had the highest average time visualizing it (37 sec). This finding becomes logic when the largest number of GSM is available precisely at the ambiance’s introduction.
Another important finding is that the highest average number of PSM accessed per phase corresponds to the users who played seven phases. But those who played 12 phases spent the most time viewing this feature (1 min and 28 seconds per phase). With this data it can be said that the user who received the EL 1 or 2, and made a level evolution (surpassed the six stages of its initial level), made use of PSM more strongly, allowing to associate this behavior to success in the game and showing its importance.
In the next subsection crossings with interaction data and other aspects of the students themselves that might be relevant in this analysis are made, in order to obtain new information and make new findings.
6.4.1. Comparative between Graduation and Post-Graduation Students
The first subsection of comparatives conducts a parallel between the average interaction data of students at graduation and post-graduation levels of Computer Science who participated in the experiment. Despite the differences between the two groups being, in general, minimum, some exploratory considerations were made.
Table 3 shows some findings, like that post-graduation students were on average more affected by technical problems encountered in the experiment (item 2), which may have influenced their perceptions and opinions about the game.
Table 3. Comparison between graduation versus post-graduation student’s interaction data.
Another inference possible to make in Table 3 is that one of the items with the greatest discrepancy values between the two groups is 17, showing that graduation level users accessed more and spent more time viewing PSM (almost double), which might indicate a good strategy when it comes to this public, revealing they sought for more complementary educational materials and were more time held in this activity compared to users on post-graduation level. Graduation students also concluded more phases.
Thus, an important inference made possible with the data listed in Table 3 is that Group 1 (graduation students) viewed more and for more time PSM items, and won (completed) more phases. With this information, it can be deduced, again, that the use of the additional resources (PSM) may have been a differentiator factor for success in the game.
In general it is possible to infer that graduation students were possibly more instigated by the challenges generated by the game. They did not give up the approach with phases in progress: completed all phases started. In addition, they missed fewer times during the game, allowing to conclude that they possibly took the game approach more incisively than post-graduation students. This finding can be derived from the interests of both, in so far as graduation students might have a greater prospect of applying the knowledge extracted with the game in its academic life.
The above signs show that JASPION could have fitted better for the audience of graduation level students, highlighting the approach to one of its objectives: to motivate students of CN discipline on Computer Science degree.
6.4.2. Comparative between Gamers and Non-Gamers
In order to verify possible differences between gamers (users that play more than once a week) and non-gamers (users that play less than once a week), this subsection provides a comparison among the interaction data of the two groups. The salient characteristics of gamers and non-gamers users’ behavior on JASPION are listed below.
• More time spent on recognizing the game and responding to the initial questionnaire (15 min 6 sec);
• Higher number of phases played and completed (11), and more time spent only with the phases (33 min 23 sec);
• Less time spent per phase (3 min 20 sec) and fewer errors (6);
• Accessed more (7) and for longer (2 min 56 sec) GSM items and had the highest average time viewing each one (27 sec);
• Accessed more (9) and for longer (5 min 7 sec) PSM items, but with the lowest average time per material (31 sec).
• Less time spent on recognizing the game and responding to the initial questionnaire (14 min 50 sec);
• Higher number of attempts (3), but less time spent on each of them (21 min 42 sec);
• Fewer played (9) and completed (8) phases, and less time spent only with the phases (31 min 4 sec);
• Highest time spent per phase (3 min 40 sec), but with more errors (7);
• Accessed fewer (5) and for less time (2 min 3 sec) GSM items and had the lowest average time visualizing each one (25 sec);
• Accessed fewer (7) and for less time (4 min 26 sec) PSM items but with the highest average time viewing each of one (32 sec).
Through the above data it seems clear that individuals with the gamer profile, that is, who have the habit of playing digital games with some frequency, have had apparently increased use of JASPION, by playing it for longer (more played and completed phases) and using its resources more (supporting materials). In addition, gamers’ group spent more time on recognizing the game. These findings indicate, in a way, the gamer profile user’s interest by the approach.
This information shows evidences that the DSG approach may be more appealing to students who are closer to digital entertainment gaming reality, which can be natural considering the student’s profile. However, it is noteworthy that the differences in the values of interaction items between the two groups were very small, which cannot strongly base this aspect.
The next section is dedicated to carry out the comparative interaction data among the four CS covered in this research: Divergent, Reflective, Serialist and Holistic.
6.4.3. Comparative among Cognitive Styles
This subsection aims to verify possibly influences associated with the students’ CS in their interaction data. Taking into account that this feature is related to data perception mode and knowledge construction (Franciscato, 2010) , it also wants to check the relevance of this aspect on player’s behavior. Table 4 shows the average interaction data per CS, where it is define R―Reflexive, D―Diver- gent, S―Serialist, H―Holistic.
The evidences extracted from Table 4 were grouped according to CS, followed by a synthesis of overall finds.
a) Reflective’s CS behavior: this group of students spent more time in the approach (61 min 45 sec), played and completed more phases (13). Franciscato (2010) points out that people of this CS need time to reflect on the information received, which may have influenced this need for more time to understand and use the game. Also, they accessed the higher number of GSM (7). As Bariani (1998) affirms, reflective are considered people whose thoughts are more organized and make prior consideration to an answer, a characteristic that can have influenced their need to see to more materials, to understand the rules and features of the game before action.
b) Divergent’s CS behavior: this group of students view the higher number (10) and for the longest time (6 min 25 sec) PSM items. According to Basílio & Vasconcellos (2011) , people of this CS contemplate the situations from different points of view and prefer to learn by concrete experience, which may have contributed to the fact that they have sought for more PSM in order to experience it
Table 4. Interaction data comparison amongcognitive styles.
in a way that could lead them to the phase’s resolution. Also denotes a point made by Bariani (1998) about the Divergent style: emphasis on quantity and variety of resources.
This group of users also spent less time on each attempt in the game (18 min 58 sec). Basílio & Vasconcellos (2011) states that the Divergent individuals are considered impulsive, which may have influenced them on trying to pass more rapidly through the phases, an aspect that can also be associated with their characteristic of curiosity, where everything is a bit interesting and they want to see all that is available. It can be concluded that Divergent CS tried more to guess or, in a way, rushed the answers because of its eagerness to see what was on the front (impulsivity), making one error about every 2.5 min. In contrast, for example, Reflective CS made about one error every 3.5 min.
These findings lead us to wonder whether DSG/GBL would be good approach to this student profile. Since Divergent individuals are curious and will probably want to do quicker to see what’s next, does the fact of playfulness and dynamism of scenarios and stages of a game can make them want to just play without worrying about learning?
c) Serialist’s CS behavior: this group of students spent more time on each attempt in the game (31 min 7 sec) and viewing GSM items (2 min 56 sec). According to Bariani (1998) , Serialists are people who give greater emphasis to separated topics and logical sequences. This may have influenced the findings mentioned, which shows how much these individuals spent time on actions that could possibly assist them on setting up this linear sequence that they need to understand something. Being an analytical style, they also spent more time responding to the initial questionnaire (14 min 3 sec), which identifies the gameplay profile. In addition, this group also had the lowest number of attempts in the game (1), however, they devoted themselves more to it, spending more time.
It can be noted the emphasis on analytical characteristics of this CS, which involves itself on what is doing, and the influence of its logical-linear approach (Bariani 1998) . Despite not having accessed the highest number of GSM (5), were the ones who most invested time in this activity (2 min 56 sec), corroborating with the statement of Franciscato (2010) , that these individuals starts from reading supporting materials to only then carry out the activity in its entirety.
d) Holistic’s CS behavior: this group of students spent less time on the categories: approach (total) (38 min 35 sec), response to the initial questionnaire (11 min 48 sec), with the phases (total) (17 min 30 sec) and at each phase (2 min 55 sec). As Bariani (1998) points out, holistic individuals give greater emphasis to the global context, preferring to examine a large amount of data, looking for patterns and relationships between them. This feature may have influenced this “fast” behavior in activities of them, to maybe cover a wider list of resources and scenarios, and thus achieve the desired overall context.
In addition, the students of this CS viewed the smaller number (5) and for less time (1 min 52 sec) PSM items. This finding may be linked to the fact they prefer to use more complex hypothesis, combining various data (Bariani, 1998) , making them seek for less didactic materials and not very detain to it, possibly trying to solve the phases grounded on empirical information.
e) Overall Cognitive Style analysis: it can be stated that the CS which took less advantage of JASPION might have been the Holistic, since it remained less time in the game (38 min 35 sec) and viewed fewer PSM items (5). Similarly, it can be concluded that the Reflexive CS had the opposite behavior, staying longer on the approach (61 min 45 sec), spending more time with the phases (40 min 5 sec) and concluding more phases (13). However, only those data do not support these findings, as the concept of advantage can be very relative, according to several aspects, such as social, emotional and cultural influences.
Some CS showed some similarity, as Reflexive and Divergent, who made the same number of attempts (3) total number of errors (7) and average time viewing each PSM (27 and 37 sec). Serialist and Holistic CS tied in the number of played and completed phases (6), number of errors (4) and GSM items accessed (5).
This contradicts researches that claim to exist proximity between Divergent with Holistic, and Serialist with Reflective, as Franciscato (2010) . JASPION’s interaction data shows that the proximity occurred between Divergent and Reflective, and between Serialist and Holistic styles. However, it is pertinent to point out that the research mentioned used the VLE MOODLE as a parameter. Thus, taking into account that JASPION is a game set in a 3D virtual world, it can be inferred that the criteria of CS approximation might not be the same.
It also contradicts Geller (2004) when stating that the Holistic CS has intrinsic motivation. Or shows that, at least in our approach, this aspect has not been fully aroused, leading to a reflection about the needs of this group of users in the use of DSG.
While contemplating the preferred educational materials by CS, the very dynamic visual of JASPION environment, as well as other DSG, might not be consistent in its fullness to the needs of certain CS. Thus, it is elucidated a demand by a detailed study to ascertain and point out guidelines and better ways to adapt a DSG/GBL approaches to the student’s CS, using different parameters and terms referenced in this study.
Next, the subsection analyzing the comparative data interaction among the three levels of expertise on JASPION (basic, intermediate and advanced) is presented.
6.4.4. Comparative among Expertise Levels
The comparison made between the average data interaction grouped by users’ EL aims to verify students’ behavior differences that may be related or linked to its level of knowledge about Computer Networks and identify the consistency of this feature on the game. The data below show the most relevant aspects of each group of EL on JASPION, allowing to make some inferences after them. Aiming more efficient indications of user interaction with the game, some data proportional to the number of played phases is presented afterwards.
• Expertise Level 1?basic:
a) Less hampered by technical problems faced in the experiment (4 min 9 sec);
b) Spent the shortest time responding to the initial questionnaire (12 min 23 sec).
This group, despite having been less affected by problems in the experiment’s execution, showed intermediate levels of average values in the interaction data items, just appearing with the shortest time spent on responding the gameplay profile questionnaire. This finding can be derived from the highest number of users belonging to this EL (10), which may have created a generalization that prevented the extraction of accurate inferences.
• Expertise Level 2―intermediate:
a) Made the highest number of attempts (3), played (11) and completed (10) a greater number of phases and was also responsible for the highest number of errors (8);
b) Invested the shortest time with the approach (49 min 30 sec), on each attempt (16 min 33 sec) and recognizing the game (2 min 14 sec);
c) Spent the shortest time directly with the phases (29 min 16 sec) and obtained the lowest average time per phase (2 min 46 sec).
As shown by the above data, the average EL 2 users were the most ventured in JASPION, playing and completing more phases, trying and making mistakes more often. Nevertheless, they spent less time with those actions, leading to the conclusion that they had a certain “anxiety” to evolve in the game. So it was a group of users who made a good use of the game, but did not get much stuck to the activities, possibly wanting to find solutions faster. This may be linked to the fact they have landed on a higher level than basic, but lower than the advanced, putting them on an emerging condition.
• Expertise Level 3―advanced:
a) Spent the highest time on the approach (1 hour 13 min) and on each attempt (32 min 46 sec), but also faced more time with technical problems in experiment’s execution (7 min 35 sec);
b) Spent the longest time on recognizing the game (4 min) and responding to the initial questionnaire (13 min);
c) Invested the highest time directly with the phases (48 min 25 sec) and had the highest average time on each phase (6 min 55 sec);
d) Spent the highest time viewing PSM items (11 min);
e) Played and completed the shortest number of phases (7), but received the game with reduced number of phases (the last level?only 7).
It can be deduced that the average EL3 user had a good use of JASPION, spending more time with the approach and viewing PSM. This finding shows that, despite having advanced knowledge in the area, this student used the materials available in the environment even more intensely than other users. In this sense it can be inferred that the didactic material was not consulted only by those who “did not know” the content, or instead, it was consistent with difficulty level of the challenge launched.
Despite having played the least number of phases, as has already received the game on the last level of difficulty (only containing the last six and the final phase), the average EL3 user spent the longest time at each phase. Thus, it can be inferenced that it took greater care on game execution, giving more attention to activities and turning noticeable its interest by the approach.
• Proportional inferences by played phases:
The proportional data were calculated by grouping users with the same number of played phases. Initially, it is highlighted the harnessing of each EL, considering the number of phases received, in which stands on the first place EL3 users (100%), followed by EL2 (85%) and EL1 (48%) users.
It is also observed that the EL3 group had the largest number of accesses to PSM items per phase (0.9) and spent the most time viewing GSM (24 sec) and PSM (1 min 34 sec) items per phase, just getting slightly behind EL1 users regarding the number of PSM accessed per phase (1.0). So, again it is emphasized EL3 student (advanced) dedication with the approach.
Users’ interaction data brings no information to show that a particular Expertise Level group behaved exceptionally in comparison to the others in the game, maintaining close values of measures from one to another. The only discrepancy occurred regarding the EL3 group, allowing to state that the game was somewhat more attractive to this audience, possibly for being closer to CN area.
Given the above data it can be said that JASPION was adequate to the three levels of expertise on Computer Networks (basic, intermediate and advanced), making it possible to deduce that the ubiquitous’ aspect regarding the student’s EL was consistent with the game phases.
Next, final considerations about the research results are made, seeking to summarize the conclusions found.
7. Conclusions and Final Considerations
This research aimed at verifying the possibilities of motivational influences derived from the application of a DSG/GBL approach in the context of Computer Networks discipline from Computer Science degree, through the development and implementation of a ubiquitous serious game integrated to a 3D immersive virtual environment built on OpenSim platform.
The approach shows that the environment is able to increase students’ interest for the course content. According to the sample of participants, gaming activities were suited to their cognitive style and level of expertise, complying with the ubiquitous requirement of the game, not letting them get bored and motivating learning. Evaluators said the game as a whole is interesting in the education sector, not only for CN learning, but in other contexts, such as data communication.
Through the data it was possible to observe the impact of the DSG/GBL approach on motivating the students, noting its influence on several factors. Among the aspects that support this statement, some are below elicited, drawn from direct students’ statements and inferences about their behavior on the environment (evaluation instruments 1 and 2):
a) The knowledge acquired was important and useful, enabling “learning through playing”, making “to forget a bit the real and to focus on the virtual”, and therefore a great opportunity to improve the practice of learning, because, although they were studying during the phases “it looked like it was something relaxing”;
b) JASPION “maintains the standard way of learning used in the classroom” (e.g. slides, texts, videos), adding the game interactivity, which awakens creativity and makes learning most attractive and dynamic, helping to “retain knowledge while distracted”, also allowing to “record a graphic memory” according to the game environments and certain content;
c) The game is intuitive, simple and easy to understand and play, making students interested by the approach and to pass the levels, highlighting the range of scenarios and phases, in addition to the “different formats of consultation materials”, which made them “feel they were evolving in the game”. In this context, “the villain taunts (Satan Goss) served as incentive for striving to provide the correct answer”;
d) The evaluators reported they wanted to go to the next steps to “see if they would know the answer”, that didn’t want to “lose lives” in the game, and although not so easy, the questions turned learning interesting, that is “guaranteed even with failure” (learning from mistakes), showing “a good method of knowledge review”;
e) The game complemented and allowed to “fix contents addressed on Computer Networks discipline”, making understandable what “is seen in the classroom”, covering practical and theoretical issues with “application in the work field”, so that students who “thought they knew many things” realize that there was more to learn, recurring thus to the PSM;
In addition, the game allowed the expression of autonomy and self-awareness: students were free to consult the supplementary materials when and which interested them, and individually advance on phases and reach good results in the game, being consulting or not these materials. Brindley (2015) and Barbosa (2007) suggest that these characteristics are important factors to develop on the contemporary student.
Thus, it can be concluded that it is possible to improve motivation and interest of students in a courseware (game) that is tailored to their cognitive style and level of expertise, providing resources according to their reality, as it was possible to verify the positivity that was the students’ interaction with the object of study, especially in a complex and dynamic subject like CN of Computer Science, intensifying interest about the contents addressed and the will to seek for more knowledge.
Although the results were not statistically valid due to the low number of participants in the sample it is observed in this study the extraction of some pertinent information to technology in education area, which can influence and spread decision-making by other DSG/GBL approaches.