({{, Quezzio}})

Instructional Precision Driven by Computation

Danny Finn
Wolfram Technical Consulting

(RowDefault[, (Spacer1), , (Spacer1), ])  #WolframTechConf

ABSTRACT

Quezzio provides dynamic, reliable assessments that work the way you teach.

Instructors author questions in a Wolfram notebook.

Students answer questions in a web interface.

Quezzio grades students’ answers using instructor-defined rules.

I will show how LLMs can simplify question authoring.

What is Quezzio?

Learning and assessment platform

https://www.quezzio.com/

Previously known as Wolfram EdTech Learning Environment (WeLE)

Now appears on the Wolfram Products Page:

Originally designed for colleges and universities, Quezzio equally enhances K-12 instruction.

What is Quezzio?

Quezzio allows you to do three main things:

What is Quezzio?

Quezzio allows you to do three main things:

What is Quezzio?

Quezzio allows you to do three main things:

What is Quezzio?

Quezzio allows you to do three main things:

Presentation focus: Author (and LLMs to speed this process up)

Question Generator Applications

Question Generator: In the simplest form, each student sees the question with different values inserted.

Generate as many solvable versions of a question as you want:

Different questions for each student

Additional practice

PDF problem sheets

Reliable questions for AIs

Solutions and answers generated computationally for each question.

Computational generation removes the possibility of mathematical errors which may occur when questions are generated by LLMs.

Accessing Quezzio Questions

Accessing Quezzio Questions

Accessing Quezzio Questions

Integrate into your pre-existing Learning Management System (LMS)

Accessing Quezzio Questions

Pull questions directly into online systems

Grade student answers

Question Banks

4 approaches:

Wolfram-Made Question Banks

US Common Core State Standards for Mathematics

Aligned with grades 4–12, this bank delivers ready-to-use math questions built from Wolfram’s computational engine. Teachers get a structured library of problems tied to recognized standards, designed for flexible classroom and assessment use.

Sample Questions

Scope & Standards

Covers grades 4–12 across all Common Core math domains

~4,000 algorithmic question generators

10 generators per standard, each with 5 difficulty levels

Content & Pedagogy

Worded and visual problems with typeset math and generated graphics

Written specifically to align with US Common Core standards

Solution steps and automated grading built in

Answer input type: multiple choice

Delivery & Integration

Available via API for seamless integration

API response formats: JSON with HTML (markup) and MathML (math expressions)

PDF worksheets available for offline us

Topic List

Wolfram-Made Question Banks

Wolfram Problem Generator

This bank connects directly to the Wolfram Problem Generator, providing an endless stream of pure math problems with typeset formatting. It spans core areas from arithmetic to statistics, giving teachers a reliable source of fresh practice and assessment material.

Sample Questions

Scope & Standards

~360 algorithmic question generators

Covers arithmetic, number theory, algebra, calculus, linear algebra, and statistics

3 difficulty levels per topic: Beginner, Intermediate, Advanced

Content & Pedagogy

Problems are fully typeset with professional math formatting

Solution steps (via Wolfram|Alpha) and automated grading included

Answer input types: multiple choice and free response

Delivery & Integration

Available via API for seamless integration

API response formats: JPlain text, MathML, LATEX, Wolfram Language

PDF worksheets available for offline use

Topic List

Author your own Questions

AI-Assisted Question Authoring

Question author provides initial problem text or image

LLM generates a corresponding initial template

Random variables, question, solution steps, answers, computations

Author refines the template (Human-in-the-loop)

Questions feeds into Quezzio

Content ingestion Case Study:
	OpenStax Elementary Algebra 2e (HyperlinkTemplate["†", https://github.com/openstax/osbooks-prealgebra-bundle/?tab=CC-BY-4.0-1-ov-file, https://github.com/openstax/osbooks-prealgebra-bundle/?tab=CC-BY-4.0-1-ov-file, Hyperlink, HyperlinkActive, , ])

Template the question and configure the randomization

->

Template the solution steps

Include graphics

Define worded problems

Human-in-the-loop refinement

The Quezzio question template notebook:

Designed for authoring questions from scratch

Wolfram Language makes defining content easy

Ideal for reviewing, refining and testing

Column[
	{WordCloud[
		Map[
			Column[StringSplit[#], Alignment -> Center, Spacings -> 0]&,
			{
				"Question Text",
				"Typeset math",
				"Procedural Correctness",
				"Teaching Workflow",
				"Learning Objectives",
				"Numeric Ranges",
				"Generated Graphics",
				"Grading Rules",
				"Online Rendering",
				"Validity Checks",
				"Solution Steps"
			}
		],
		WordOrientation -> {{-π/3, -π/6, 0, 0, 0, π/6, π/3}},
		WordSpacings -> 6,
		ImageSize -> {600, 300}
	]},
	ItemSize -> Scaled[1],
	Alignment -> Center
]

...and we are on hand to help out also.

Wolfram Notebook Assistant

Find and explain Wolfram functions

Wolfram Notebook Assistant

Insert plots and customize the styling and labels

Wolfram Notebook Assistant

Suggest improvements

Conclusions

Quezzio: Author, Answer, Assess

For instruction of both K-12 and Higher Ed

Two Wolfram-made question banks. More to come!

LLM assistance in question authoring

Quezzio provides a perfect environment for refining LLM-initiated templates

Programmatic workflows for mass ingestion into Quezzio

The consulting team are ready to build custom question banks for you

Useful Links

Related Presentations

[Previous Talk] Question Framework and Applications to Computational Thinking

Christophe Pakleza & Xiaofan Zhang

Wed Nov 5, 9:00 am - 9:30 am (CDT)

What’s New in Chemical Education

Jason Sonnenberg

Thurs Nov 6, 12:30 pm - 1:00 pm (CDT)

Transforming Teaching: Prompting Techniques and Assessment Strategy for Learning with AI

John McNally

Fri Nov 7, 9:30 am - 10:00 am (CDT)

Bonus: AssessmentTree Resource Function

https://resources.wolframcloud.com/FunctionRepository/resources/AssessmentTree/

AssessmentFunction: Comparison Methods

AssessmentFunction[Hold[(x - 1) (x + 1)], "PolynomialResult"]

Column[
	{ResourceFunction["AssessmentFunctionComparisonTable"][
		Hold[(x - 1) (x + 1)],
		{Hold[(x - 1) (x + 1)], Hold[(x + 1) (x - 1)], Hold[((x^2)- 1)], Hold[D[(((x^3)/3) - x), x]]},
		{"HeldExpression", "PolynomialResult", "CalculusResult"}
	]},
	ItemSize -> Scaled[1],
	Alignment -> Center
]

Can we chain multiple of these in an arbitrary way and apply conditional branching?

This would allow us to provide conditional:

Scoring

Explanations

Bonus: AssessmentTree Resource Function

https://resources.wolframcloud.com/FunctionRepository/resources/AssessmentTree/

AssessmentTree

Create a tree of AssessmentFunction expressions to traverse during assessment

at = ResourceFunction["AssessmentTree"][
	(* Nodes *)
	<|
		"Node1" -> AssessmentFunction[
			(x - 1) (x + 1) -> <|
				"Explanation" -> "Well done, you successfully factorized the polynomial."
			|>,
			"PolynomialResult"
		],
		"Node2" -> AssessmentFunction[
			(x - 1) (x + 1) -> <|
				"Score" -> 0,
				"Explanation" -> StringJoin[
					"Your answer is mathematically equivalent to the correct",
					" answer, but has not been provided in the required form."
				]
			|>,
			"CalculusResult"
		]
	|>,
	(* Path *)
	<|"Node1" -> <|False -> "Node2"|>|>
]

Dataset[<|#->at[#][{"Score","Explanation"}]&/@{(x+1) (x-1),(x^2)- 1}|>]

QuestionObject["Factor the polynomial (x^2)- 1.",at]