Random Problem Generation and the Wolfram Problem Generator

Brad Janes
Junior Developer 
Education Software Technology
Wolfram

Contents

The basic idea

Other problem generators

Wolfram Problem Generator

Future development plans and ideas

Q&A

The basic idea - Why make a problem generator?

What does a problem generator do?

* Generates (random) practice (math) problems based on a topic
* Allows for some form of input/interactivity
* Checks if the user’s answer is correct

For educators

* More educators are moving to online assignments
* Textbooks and homework sheets are finite in their number of problems
* Automatic grading
* Prevents cheating - if all students get different questions, they can’t share their answers

Import["https://lh3.googleusercontent.com/-W-VNWOyfmgw/Upyc7n_UuuI/AAAAAAABGas/rmtpIXKhUQA/w426-h305/324.gif", "Animation"]

For students

* Many subjects, benefit from repetition of problems
* Problem generators have as many problems as you may need
* Decreases the need to carry around heavy textbooks
* Immediate feedback vs waiting for work to be graded
* Ease of use

Issues with other problem generators

Not random

* Pulling from a cache of problems, users will see repeats

Answer checking

* String match your answer with the expected answer.

* Computers vs. Real numbers (precision, floating point, etc)

* What are the "correct" rounding rules?

* What constitutes the simplest closed form of an expression?

* Mixed number vs improper fraction, rationalize radicals in denominator, etc.

* Forced to use the program’s popup buttons for symbols instead of keyboard

* If not using free response input, then what? 
	True/False?
	Multiple choice?

No feedback (or not helpful feedback)

* Answers are marked as correct or incorrect without any feedback
* Emphasis on final answer over method used

Many students do not like these systems

Our Solution - Wolfram Problem Generator

True random problem generation - not pulling from banks of problems

We make use of Mathematica’s Random[] and statistics functions for our various problem types

Answer checking with the parsing power of Wolfram Alpha

We are able to accept a variety of inputs as correct with answer checking catered to each problem type.

Feedback to users

Immediate answer checking
Input interpretation, curated hints, and Wolfram|Alpha’s step-by-step solutions
Problem History

Multiple choice problem sheets

“Distractors” - What makes a good “wrong answer” for a particular problem?

People seem to like it

Most common type of feedback we receive is people asking for new content areas

Try it!

Hyperlink["http://www.wolframalpha.com/problem-generator/"]

Future of WPG

More content

Geometry
Trigonometry
Physics
Chemistry
Word Problems

Visual and interactivity improvements

Teacher/admin tools

User history tracking
Sending problem answers to a 3rd party (teachers / TAs / parents)
Certifications / Gamification / Leveling up

Better hint generation

Possibly use machine learning to better tell exactly where the user went wrong.

Adaptive problem selection

Another opportunity for machine learning

Integration with W|A and other Wolfram products

Interactive problem sheets

API - For educators or software creators

Enable fine tuning the type of problem, difficulty, expected answer form, etc.

Try it!

Hyperlink["https://www.wolframcloud.com/objects/user-d95db9dd-3ed5-4df3-9c3b-fec7c03fe862/WPGform?_key=WTC16"]

Huge potential for growth

We collect user data and feedback to give us ideas of content/features to work towards

Questions?

Please try out Wolfram Problem Generator and send in any feedback that you have!

Thank you!

Sources

http://www.d.umn.edu/math/Technical%20Reports/Technical%20Reports%202007-/TR%202011/TR_2011_2.pdf