Each person has their own learning style, or more commonly known as learning modality. There are three known types of learning modalities: visual, auditory and kinaesthetic . Learning modalities can be roughly understood as the way in which a person perceives information. When put in the educational context, it implies that a learner is able to grasp new skills, concepts or gain understanding better when they are presented in a way that matches with their preferred way of perceiving information that is, learning modality.
In pre-school education, young children are taught the basic skills necessary to form the foundation for the building blocks of their cognitive abilities, one of which is handwriting . However, the process of learning the handwriting skill (handwriting-instruction) is complicated and time consuming. Additionally, pre-school learners are quick to lose focus, enthusiasm as well as motivation. This could very well be due to the possibility that they are not taught in a manner that matches their learning modality.
On the other hand, pre-school instructors face the challenge of satisfying the needs and concerns of each individual learner, all of whom have differing individual needs, with their learning modality being one of them. In the average Malaysian pre-school setting, an instructor has to handle a class of around 10 - 15 pre-schoolers―this can indeed be a daunting task that affects both instructor and learner.
Augmented reality (AR) is a technology that allows for the augmentation of virtual 3D objects superimposed upon real-world objects and landscapes. Using this technology, interaction with real-world objects is enhanced through the overlapping of 3D objects (inclusive of animation capabilities), allowing for a more immersive and engaged experience    .
This paper describes the development of an AR book and mobile application to facilitate the handwriting instruction by using the features and capabilities of AR to address the needs of different learners.
2. Background & Related Work
2.1. Augmented Reality
2.1.1. Definition of Augmented Reality
Augmented Reality (AR) is generally defined as the augmentation of virtual 3D objects or information onto the physical real world. These objects can vary from text, images, video and other objects such as animals, items, and even humanoids . The combination of both the virtual objects and the real environments results in a mixed reality in which information is conveyed meaningfully .
In , there are three identified characteristics of an AR application:
・ Combines both real world and virtual elements
・ Rendered in 3D
Over the past two decades, AR technology has gradually gained attention as a means of training and teaching by allowing users to interact and manipulate objects and information in relation to the current reality. AR technology has been used to create applications for numerous fields such as advertising and marketing, architecture and construction, entertainment, medicine, military as well as travel .
2.1.2. Marker- and Markerless-Based AR
There are two methods in which AR applications can augment objects, which are marker based and markerless-based AR . According to , marker-based applications usually comprise of three basic components which include:
・ A booklet for offering marker information
・ A gripper for getting information from the booklet and converting it to another type of data
・ A cube for augmenting information into 3D rendered information on a screen
Marker-based AR uses images which are already embedded within the system to augment virtual objects onto the real world .
2.2. Augmented Reality in Education
Despite the fact that a lot of the progress done in AR technology and applications are mainly driven by business-needs, researchers and educators have recognized the potential and benefits of using AR in the educational field        . Today, AR technology has been used in varying stages of education, from the primary level to the tertiary level. It is interesting to note that AR has been mostly applied in higher education settings; in fact, 23.64% of applications exist for the Bachelor level; 16.36% for the primary level; 18.18% for the junior school level; and 5% for the early childhood education level .
There are five categories of AR applications for the educational sector :
・ AR book
・ AR gaming
・ Discovery-based learning
・ Objects Modelling
・ Skills Training
The overlaying nature of AR that is able to convey and render objects in the real environment in order to express the relationship and connection of information opens doors to numerous benefits in the educational setting. Based on observatory studies conducted on existing educational AR applications, researchers have identified increased motivation, active engagement, “improved perceived enjoyment”, increased accessibility and variety of education content, complements active learning, “greater opportunities for collaboration”, improved capacity to retain information, facilitates understanding of the subject or topic, rendering concrete manifestations of abstract concepts, “provides powerful contextual, on-site learning” as benefits of using AR       . AR has been implied to be successful especially in engaging learners in learning a particular topic, subject or concept.
Studies have shown that students are able to learn and understand concepts at a greater degree when they are able to not only visualize and see manifestations of the concept in terms of objects and shapes, but also when they are able to virtually manipulate them    .
2.3. Handwriting-Instruction and Learning Modalities
Different types of learners learn differently, even when it comes to learning a basic skill such as handwriting. There are three known types of learning modalities: visual, kinaesthetic and auditory . Not one person is wholly dependent on only one type of modality; each person is said to have a predominant modality, meaning, they tend to or prefer to, or are more able to learn in that style compared to other modalities. According to, 30% of all elementary and middle school children are visual learners, 25% are auditory leaners, 15% are kinaesthetic learners, while the remaining 30% are “mixed modalities”, which means that their learning style adapts to how the material is being presented.
According to , each modality has different ways of learning, for example:
・ Kinaesthetic learnings prefer a classroom atmosphere that allows for movement, are fidgety and hyperactive. They learn by doing.
・ For auditory learners, they will be able to absorb visual demonstrations better when it is accompanied with verbal cues. They become more engaged with sounds and changes in tone of voice. For example, the educator can add tone to their strokes by raising their voice on an upward stroke and lowering their tone on a downward stroke.
・ Visual learners greatly benefit from one-on-one demonstrations and examples. It is even better when the demonstrations are frequent. They also respond well to colour cues. For example, educators can reinforce basic strokes using colours, or associate different strokes with different colours.
3. Development of AR Book and Application Prototype
3.1. Current Handwriting-Instruction
A preliminary study was conducted at a local kindergarten to understand the current handwriting-instruction method, challenges faced by both the learners and instructors and the current solutions to these challenges.
Figure 1 shows the most commonly used method to learn and teach hand-writing is the use of tracing activity sheets. These activity sheets have a visual reference of the completed character in one corner, and a series of boxes with dotted lines that makes up the character. The students are expected to trace over these dotted lines repetitively until they are able to write it fluently. What is meant by fluent in this context is the smoothness of their strokes-from shaky-lines to well-formed lines.
Figure 1. Current handwriting-instruction makes use of tracing activity sheets.
However, the biggest challenge for both learner and instructor is maintaining focus. The students tend to lose focus and enthusiasm midway―and this varies from student to student. Sometimes they would suddenly stop trying to trace at all and scribble instead, some would suddenly go off into a corner and cry and some would completely do something else entirely. It is then the instructor’s job to get them back on track with the activity. To do this, they would try to get the students motivated and enthusiastic again. For example, they would do activities to make them associate the character they are tracing with colour or shapes, and they would cheer them and give verbal encouragements.
This is daunting for the students and instructors. In an average pre-school classroom in Malaysia, one instructor would have to cater for 10 - 15 students in one class. To have to address the needs of each individual, and conversely, for each individual to wait to be addressed by one instructor would affect their learning process and experience.
3.2. The AR Book Prototype
The prototype consists of two parts, which are the mobile AR application and the physical book. The following sections will describe the physical book design and development, and the mobile application development respectively. Figure 2 depicts the mobile AR application (on the mobile device) and physical book respectively.
3.2.1. Physical Book Design and Development
Since the mobile application makes use of marker-based augmented reality, there exists the need to create visual markers. These visual markers were compiled in the form of a book. Each page was designed to meet the standards of a high-quality AR marker. According to , high quality markers are generally:
・ Rich in detail―each page is carefully designed to contain just enough detail and elements to fulfill the requirement for detail, but not too much until it takes the user’s focus away from the main activity
・ Has good contrast―the colours used to colour the illustrations on each page were chosen carefully so that each colour is very vivid and distinguishable, allowing for a high contrast
・ Does not have repetitive patterns―each page has a unique design, albeit with similar elements such as grass, rocks and clouds
Figure 2. The system prototype which consists of the mobile application and book.
The book is designed to have ten numbers (numbers one to ten), with each number having a demonstration page and an activity page. The demonstration page serves as a visual reference, and the activity page is where the tracing activity takes place. Figure 3 shows the demonstration and activity pages for numbers three and four. The use of these pages (demonstration and activity) with the mobile AR application will be discussed in detail in the following section.
The researcher also incorporated a theme throughout the book to ensure a consistent feel and flow from one illustration to another. This was done to give a better user experience. In order to ensure consistency and relevance between what is illustrated in the book and what is augmented, the book’s animal illustrations are designed based on the chosen models.
The selection of the theme (farm animals) was done based on the 3D models that are used in the mobile AR application (this will be discussed in detail in the following section).
3.2.2. Mobile AR Application Development
To address the needs of different learners, the researcher has developed the application to have the following main features:
・ Visual demonstrations
・ Auditory and visual cues
・ Gratification mechanism
・ Upbeat background music
As aforementioned in the previous section, the mobile application is used to together with the book. The book has two pages for each number: a demonstration page and an activity page. Upon detection of the demonstration page, the application augments an animated pencil that shows the movements of strokes to write the number. A rainbow coloured trail is also augmented as the pencil moves along its course. An upbeat background music plays throughout the demonstration. A repeat button is augmented on the top-right corner of the book, readily available to reply the animation. Figure 4 shows screenshots of the stroke demonstration animation in motion for numbers seven, two and three.
All the features and mechanics implemented in the demonstration page are developed to be able to appeal to the different types of learners. The demonstration itself is theorized to address the needs of visual learners. The strokes themselves are reinforced with colour cues, and the image on the book provides a
Figure 3. The demonstration page and tracing page for numbers two and three.
Figure 4. The demonstration animation in motion.
constant visual reference. Auditory learners appreciate music, thus justifying the use of an upbeat background music. The animated pencil is theorized to address the needs of kinaesthetic learners for movement. Additionally, they can point and trace the animated pencil throughout the demonstration.
Upon detection of the activity page, the application augments a 3D pencil and black 3D spheres whose quantity and position correlate to the quantity and position of the black 2D dots that make up each number in the book.
The user interacts with the objects via the tablet’s screen, dragging the pencil using either their index finger or using the tablet’s stylus. However, because its purpose is to teach handwriting, it is recommended to use the stylus.
The user traces the number by using the pencil to hit the spheres. The moment a sphere is hit, its colour turns from black to bright green (visual cue) and the application plays a “ting” sound effect (verbal cue). Thus, as the user traces the number in a consistent, uniform and correct manner, there is a smooth accumulation of green-coloured spheres and a build-up of “ting” sound effects-this indicates progress and implies that the user is tracing it correctly.
Once the user has hit all the spheres (thus completing the tracing of the number), the application plays a “cheering” sound effect and augments a solid outline of the number and animated 3D animals upon a coloured plane, a repeat button, a text to congratulate them, and a text to show the name of the animal. Figure 5 shows screenshots of the activity page, from the beginning to completion, whereby the 3D animals are augmented. on The type and number of animals augmented corresponds to the animal illustrated in the current page and the number being traced. For example, if the user traces number three and the illustrations on the page depicts rabbits, then the application will augment three rabbits upon completion. Similarly, if the user traces number four and the illustrations on the page depicts horses, then the application will augment four horses upon completion.
The reason for using farm animal models is because the models were available in a package. When searching for 3D models to be used for the application, the factors considered were cost and level of detail of the model. If a model is high in detail, it would take longer to render and requires more processing power. Additionally, purchasing models individually would not only be expensive, but would also create a sense of inconsistency. This is because models created by different
Figure 5. Once the user has traced the number, animals will be augmented, along with cheering sound effects.
developers have a different art style as well as different mechanics. The end result would not be uniform.
As such, the researcher had chosen to use a package, which is a collection of models created by the same developer. These packages usually contain 10 - 15 models, and each model would have a standard set of features or animation. Since the target users are children, the farm animal package was deemed suitable.
The package has the right number of models, and the designs are detailed enough to look like it is of high quality, and simple enough that it does not take too much processing power.
For ease of use and delivering a better user experience, a tablet stand is used to support the tablet.
4. Preliminary Survey Results
The researcher brought and demonstrated the prototype to pre-school instructors at kindergartens, and has also exhibited the prototype at various exhibitions. They were given a brief explanation of what it is supposed to achieve, and a demonstration of how to use the book and application. They were then asked for their feedback to gain an insight on their acceptance of the application, their willingness to adopt the technology in their classroom and areas of improvement.
4.1. Feedback from Pre-School Instructors
The prototype was tested with intended users (i.e. pre-school students). However, results of these observatory studies will be reported in a different publication. This section describes the instructor’s feedback towards the application.
An informal interview was carried out with one of the instructors at the kindergarten, as well as the headmistress. The researcher gave a brief explanation of how the application works and what it is supposed to achieve. They were then shown a demonstration of how to use the application, and were given some time to use and explore the prototype.
The instructor responded well to the application, saying that “the kids were visibly enjoying it”.
They were also pleased that the by using the application, the students were not only able to actively learn by themselves, but they were also collaborating with each other to learn. “It’s great that the students are helping each other. We can give a simple demonstration in the beginning, to show how to use the book, then allow the kids to explore it by themselves―of course, with our supervision-but with this application, the strain is less-it’s able to keep the kids happy while they do it.”
Furthermore, they see potential in using AR technology to teach toddlers form and mov1ement―the phase before actually learning to write any character. “It’d be great if you could make an application that is one step back, for before they even begin to write anything―hand-strokes. Before learning to write, three to four year olds would have to familiarize their hands with movement. They need to learn to draw lines, curves, zigzags―they need to do activities to loosen their muscles. We have various worksheets for learning these―hand-strokes alone. So that’s what you could try to do next.”
Additionally, the instructors were excited to know if there are plans for an alphabets version of the application. They asked questions like “would you be making an extension for alphabets soon?”, and for potential new features in upcoming versions.
They also advised for the book and application to be displayed at an upcoming registration day, so as to give exposure to parents about new tools, technology and applications that can be used to complement their children’s learning.
4.2. Feedback from Exhibitions
The prototype was exhibited at various design, information technology and teaching and learning exhibitions. At these exhibitions, the researcher gave an explanation as well as a demonstration to the researchers and other academicians who had happened to inquire about the prototype. This section describes some of the feedback that were given, specifically by a researcher in the pre-school education field, a researcher in the human computer interaction field, as well as lecturer in the information communication technology field.
According to the pre-school education researcher, “this application is good for young learners to get to know the numbers, how to write it, to familiarize themselves with it.”
The human computer interaction researcher (who is also a mother of a special needs child) suggested to upgrade the application so that it is able to address the needs of dyslexic learners. For example, dyslexic learners perceive form and movement very differently; they may start from the bottom, or from the side-anything but how non-dyslexic learners perceive form and movement. Thus, she recommended to add colour cues to indicate start and end points.
The information communication technology lecturer suggested to add a scoring mechanism so the user would know how well they have traced the number-for example, if a stroke was slightly “shaky”, then the overall score would be 90% as opposed to 100%.
5. Conclusions & Recommendations
This paper described the development of a mobile AR application that is used together with a physical book to enhance the handwriting-instruction experience for pre-school learners by appealing to different learning modalities. It included features that would be able to appeal to each type of learner, such as animated and interactive 3D objects, music as well as verbal and visual cues.
A preliminary survey was conducted to gain an insight on the acceptance and potential adoption of the technology in the modern pre-school classroom. Survey results and feedback from other researchers and academicians in relevant fields (such as pre-school education, human computer interaction and information technology) indicate that the application has a high potential to be used in the classroom.
The augmented reality book and application was developed to complement and facilitate the handwriting-instruction, and not to completely replace traditional means. This application can also serve as the foundation for other potential edutainment applications such as alphabets writing as well as for various other languages (for example, Mandarin, Japanese, Korean, Arabic, Tamil) in which the number, order and direction of strokes matters.
The researcher would like to thank Azfar bin Tomi for his advice during the development of the application.