Educational Game for Early Literacy -Sprinkle
#Independent Game Development, #Natural Language Processing
#Speech Recognition, #Python Programming
Duration: 4 weeks, Apr. 15th, 2019 - May.15th, 2019
Designer and Developer: Kexin Yang
Video demo of Sprinkle
Speak Game
When a kid correctly pronounce the word in the bubble, the bubble will disappear and the kid will gain 3 points.
Sort Game
Level 1
Level 1 assesses the simplest sentence structure: Subject+Object+Verb.
When a kid correctly sort a sentence into grammatical order, the kid will gain 5 points.
Level 2
Level 1 assesses more complex sentence structure: Subject+Object+Verb+Prepositional Phrases (PP). When a kid correctly sort a sentence into grammatical order, the kid will gain 10 points.
Development Process
Natural Language Toolkit and Grammar Writing
To add on to the algorithmic complexity, instead of using existing NLTK grammar, I defined my own grammar that can handle the two sentence structures mentioned
(1) Subject+Object+Verb (SVO),
(2) Subject+Object+Verb + Prepositional Phrases(SVO + PP).
I made use of the recursive parser in the NLTK to recursively parse the sentence structure, and return True or False, depending on whether the sentence entered is in accordance with the grammar I defined.
Animation and User-Interface
The interface design was coded using the Python module tkinter, with some external package including PIL to insert background image.
Speech Recognition Function
I made use of the speech_recognition library in Python to hear words spoken and transcribe that into text, then I programmed a function to check if the word heard is the same as the word currently exist.
Limitation
Due to time constraint, there is a lot of room for improvement from perspective of learning sciences and educational game design. (For example, while the disappearance of bubbles serve as correctness feedback, the sorting game still lacks immediate corrective feedback).
Python Libraries Used:
Tkinter (Graphic module) Speech_recognition
Natural Language Toolkit (NLTK)
numpy
PIL(ImageTk, Image)
playsound
Github Repository: [Here]
