How Research Shapes Assessment Designs and Placements Tests

A look at the research and approaches behind Möbius MAA Placement Test Suite
and ALEKS placement solution

The Cognitive Foundation: MAA’s Research-Based Approach to Placement

There are several schools of thought when it comes to the knowledge and understanding needed to succeed in algebra, pre-calculus, and calculus courses, which can also inform the design of an assessment or the construction of placement tests. Developed by the Mathematical Association of America (MAA), both its Algebra and Precalculus Concept Readiness (APCR) and the Calculus Concept Readiness (CCR) assessments (which are available through Möbius) were created with close attention to what qualitative educational research has indicated to be critical conceptual understandings, skills, and reasoning abilities needed for success in entry-level undergraduate mathematics. 

A large and growing body of research in mathematics education has demonstrated that students’ underdeveloped conceptions of foundational ideas (e.g., function, variable, equality, rate of change) and their struggles to engage in particular ways of reasoning (e.g., proportional, quantitative, and covariational reasoning) underlie the difficulties they experience in entry-level undergraduate mathematics courses. MAA’s APCR and CCR assessments were developed through an iterative design process to assess these foundational ideas and essential ways of reasoning to better determine a student’s level of preparedness to succeed in conceptually focused, entry-level undergraduate mathematics courses.  

Understanding the Limitations of Procedural Proficiency Models

ALEKS, in contrast, treats mathematics as a collection of topics at which students have achieved various levels of proficiency. ALEKS, a web-based learning and assessment system, uses adaptive questioning to quantify a student’s knowledge of various mathematical topics, as well as instruct students on the skills they have not yet demonstrated proficiency. 

Distinguishing Between Knowledge States and Conceptual Mastery

What makes ALEKS unique is its use of AI to continuously assess individual students and to engage them in tasks that are appropriate for their level of development. Through the application of adaptive questioning, ALEKS identifies a student’s “knowledge state” (i.e., their mastery of the declarative knowledge and procedural skills associated with a set of mathematical topics). The adaptive questioning component of the ALEKS assessment is grounded in research that focuses on mathematical psychology, psychometrics, and artificial intelligence. It is in this sense that the ALEKS assessment is “research based.” However, there are many genres of research that can inform the design of an assessment. Additionally, mathematical tasks within ALEKS are not meaningfully informed by the research literature characterizing non-normative conceptions that underlie students’ difficulties with learning essential concepts in entry-level, undergraduate mathematics courses. 

The “Learning Space” vs. Qualitative Reasoning

Learning within the ALEKS system is conceptualized as a student’s trajectory through a complex organization of independent knowledge, which ALEKS calls a “learning space.” This restricted view of mathematical knowledge and simplified conception of learning is necessary for the adaptive questioning component to function. Each knowledge state within a learning space reflects a combination of behavioral learning outcomes (e.g., solving proportions, graphing linear inequalities, factoring quadratic expressions), and are easy to specify. The adaptive questioning component of ALEKS engages students in a succession of procedural tasks to continuously compute their precise location in a vast network of trillions of possible knowledge states. This achievement would not be possible if knowledge states were instead defined in terms of diverse ways of conceptualizing mathematical ideas or the ways of reasoning that facilitate their effective application in novel contexts. 

Why MAA Prioritizes Meaning Over Mechanics for Student Success

While the adaptive approach makes ALEKS an effective tool for determining a student’s declarative knowledge and level of procedural fluency, it is not appropriate for assessing mathematical meanings or the student’s ability to engage in productive ways of reasoning. Procedural fluency is an important component of mathematical competence, but it is not the central impediment to students’ success in undergraduate mathematics. The mathematics education research literature has characterized the many non-normative conceptions that underlie students’ difficulties with learning essential concepts, and the MAA applied this research to develop the instruments found in the Möbius MAA Placement Test Suite. These placement tests better identify whether students are truly prepared for their entry-level mathematics courses.

About the Author:

Michael Tallman is an Associate Professor of Mathematics at  Oklahoma State University. He received his B.Sc. and M.A. in mathematics from the University of Northern Colorado and his Ph.D. in mathematics education from Arizona State University. Dr. Tallman has taught mathematics at secondary and post-secondary levels. His primary research is in the area of mathematical knowledge for teaching secondary and post-secondary mathematics. Dr. Tallman’s work informs the design of teacher preparation programs and professional development initiatives through his empirical research into the factors that affect the nature and quality of the mathematical knowledge teachers leverage in the context of teaching. In particular, his research examines how various factors like curricula, emotional regulation, identity, and teachers’ construction and appraisal of instructional constraints mediate the enactment of their subject matter knowledge.

To learn more about the differences between Möbius MAA Placement Test Suite and ALEKS, read our blog: Möbius MAA PTS vs. ALEKS: Which Placement Testing Solution Is Right for You?