Unlocking the Potential of Assessments

Join host John Kleeman in an insightful conversation with Carla Aerts, a trailblazer in the reinvention of education. Carla, with an unconventional journey from the performing arts to education technology, shares her accidental foray into the sector and delves into the importance of making impactful contributions to teaching and learning.

Carla explains what "perfect EdTech" looks like to her and critiques traditional exam-style assessments, advocating for a shift towards formative assessments and diverse models that prioritize skills, collaboration, and social engagement.

The discussion extends to the role of AI in education and Carla's predictions of how it will impact the future. The episode concludes with three key takeaways: the significance of social learning, the need for skills development in reflection and metacognitive thinking, and the promotion of learner agency for effective lifelong learning.

Tune in for a thought-provoking exploration of education, technology, and assessments, guided by Carla Aerts' forward-thinking perspectives.

John Kleeman is joined by assessment luminary, Norihisa Wada, executive adviser of EduLab in Japan, and director of the Japan Institute for Educational Measurement. Nori has experience with AI-driven voice recognition and has worked with Carnegie Mellon University and Kyoto University. He is also involved in EdTech investment. Delve into his insights on assessment and his background in the industry. Nori shares how he transitioned from his work on game development for consumer educational games and fitness games for Nintendo to the assessment industry, the future of assessments and gives advice on how assessment organizations can be more inclusive.

John Kleeman is joined by assessment luminary, Dr. Gary A. Gates, the Managing Director of Pearson VUE. Originally a nuclear physicist, Dr. Gates details his journey into the world of assessments. He discusses his key priorities going into his second year leading Pearson, the future of test centers during the rise of online proctoring and whether new AI technology is harmful or helpful to proctoring and assessments.

John Kleeman is joined by assessment luminary, Mike Olsen, the CEO and Founder of Proctorio. Mike discusses what drove him to create a company focused on proctoring, how it has expanded beyond the education sector, and how the tool can be used to address test fraud. 

John Kleeman is joined by assessment luminary, Professor Phillip Dawson, the Associate Director of the Centre for Research in Assessment and Digital Learning (CRADLE) at Australia's Deakin University. With an academic background in education, artificial intelligence and cybersecurity, he leads CRADLE's work on cheating, academic integrity and assessment security. This work spans hacking and cheating on online exams, training academics to detect contract cheating, student use of study drugs, the effectiveness of legislation in stopping cheating and the evaluation of new assessment security technologies. Phillip shares his thoughts on constructing an assessment that gives feedback, reducing test cheating and the privacy concerns associated with remote proctoring.

John Kleeman is joined by assessment luminary, Thomas Goettsche, SAP Certification Principal Production and Psychometrics Consultant. A trained psychologist, he has designed key performance indicators, e-learning programs, best practice standards and much more over the course of his career. Thomas identifies the intrinsic value of certifications, his contributions as a member of SAP's development team and explains the ins and outs of standard setting.

John Kleeman is joined by assessment luminary, Dylan Wiliam, Emeritus Professor of Educational Assessment at University College London. Dylan's academic work is focused on the use of assessment to support learning. A leader in formative assessment, he explains that testing can go beyond just measuring a student's learning as it can also be used to improve it. Dylan also shares his thoughts on test validity, online learning and the college admissions testing controversy.

John Kleeman is joined by assessment luminary, Isabel Nisbet, Affiliated Lecturer at the University of Cambridge. Isabel discusses the two fundamentals of fairness, the importance of public confidence in educational assessment and shares her thoughts on the steady push for test-optional college admissions in the US.

John Kleeman is joined by assessment luminary, Will Thalheimer, Principal at TiER1. Will details his 30-year journey to becoming a learning expert, which started as an instructional designer, trainer, and simulation architect. He explains which key findings from learning science we should put into practice, the four pillars of learning effectiveness, and the best way to evaluate how effective learning is.

John Kleeman is joined by assessment luminary Siddhartha Gupta, CEO of Mercer | Mettl. Over the course of his career, Sid held key leadership positions at some of the most prestigious global IT companies. It was during that time that he discovered the niche of assessments and eventually moved to Mercer | Mettl. Sid shares his opinion on technology and bias, assessment innovation, and Mercer | Mettl's partnership with Questionmark for the upcoming Bring Your Own Proctor feature.

John Kleeman is joined by assessment luminary Suzanne Heiob, the Global Certification Product Owner of SAP, who began her journey with the company as a student. Among her responsibilities, she oversees the product roadmap and keeps an eye on market trends to ensure that they are adopted into the program. Suzanne shares her thoughts on the best practices and future of certification programs, along with discussing international challenges, the computer translation capability co-innovation with Questionmark, and the ITCC award it recently won. 

John Kleeman is joined by assessment luminary, Andre Allen, Managing Member and Executive Vice President of FifthTheory, who has over 23 years of continuous experience in the test publishing industry. Andre has pioneered many of the technologies that are the basis for many of the assessment delivery and scoring options for today. Andre shares how he got into assessment and discusses how you can assess leadership, admissions testing and how we can do better around diversity in testing. 

John Kleeman is joined by assessment luminary, Linda Anguish, SPHR, Director of Accreditation Services at the Institute for Credentialing Excellence (ICE). She is responsible for the accreditation programs at ICE including the NCCA, ACAP and ICE’s newest accreditation program for ISO 17024 compliance. Linda shares how she got into assessment and credentialing and discusses the advantages of credentialing, the ICE certification program, diversity and the future of certification.

John Kleeman is joined by assessment luminary, Stephen Sireci, Professor in the Psychometrics Program and Director, Center for Educational Assessment at University of Massachusetts Amherst. His work is well known in the world of assessment. Steve shares how he got into assessment and discusses his research around validity, the five sources for validity evidence and his good practice advice.

John Kleeman is joined by assessment luminary, Jarrod Morgan, Founder of ProctorU (now known as Meazure Learning) and Chief Strategy Officer. Jarrod has been a leader in the educational technology field for over 16 years. Jarrod shares how he got started in assessment and discusses all things proctoring, including where he sees proctoring going, good practices in remote proctoring and the human first approach.

John Kleeman is joined by assessment luminary, Marten Roorda, Chief Measurement and Learning Officer at AI and education company, Riiid Labs. Marten is a pioneer in educational technology, a leading innovator in computer-based testing, e-learning and adaptive learning and has an extensive background, which includes ACT, CITO and more. Marten shares how he got started in assessment and discusses how to use AI in assessment, adaptive learning and his best practice advice.

John Kleeman is joined by assessment luminary, Alina von Davier, Ph.D, Chief of Assessment at Duolingo. At Duolingo, Alina and her team operate at the forefront of computational psychometrics. Alina shares how she got into assessment and discusses how to make a high stakes test equitable with multiple instances, her experience as a female leader in the assessment space and her advice to people who want to progress in assessment.

John Kleeman is joined by assessment luminary and "Learning Enthusiast," Riz Khan of SAP. Riz is Global COO and Vice President of Training and Adoption at SAP and frequently partners with CxOs, executives and leadership groups to help them understand the importance of corporate education and the power of user adoption. Riz shares his background in assessment and learning and discusses the value of investing in learning and training and how assessments fit in, his best practice advice and the future of learning and adoption.

John Kleeman is joined by assessment luminary, Gavin Cooney, CEO and co-Founder of Learnosity, the leading provider of assessment infrastructure for the learning industry. As you may have heard Learnosity recently acquired Questionmark. Gavin discusses how he got into assessment and dives into various topics, including the importance of accessibility and inclusivity in education, the trends he's noticed and how Questionmark and Leanosity fit together.

John Kleeman is joined by assessment luminaries, Sharon Shrock and Bill Coscarelli, co-authors of award winning book "Criterion-Referenced Test Development." Sharon and Bill discuss how they got into assessment and tell the story of their book. The pair also shares patterns in testing they've seen working with various organizations and their advice.

John Kleeman is joined by assessment luminary, Dr. Leanne Furby, Director of the Assessment Services Division at National League for Nursing. Leanne discusses how she got into assessments, talks about the assessment program at NLN and remote proctoring during COVID-19 and explains spaced learning and how it helps.

John Kleeman is joined by assessment luminary, Akil Bello, who is an expert in assessment reform. He currently serves as the Senior Director of Advocacy and Advancement at FairTest, the National Center for Fair and Open Testing. Akil discusses how he got interested in assessments and talks about his work with admissions tests, whether recruitment tests are fair and his thoughts on what the testing industry should be doing better.

John Kleeman is joined by assessment luminary, Helen Baron, who is an expert in psychometric measurement and assessment. Helen works with test publishers and employers on developing tests and assessment processes. Helen discusses how she got into assessment, and dives into her time at SHL, the different types of assessments used in recruiting and where personality tests come in and her work on assessment standards.

John Kleeman is joined by assessment luminary, Wendy Kirkpatrick, President and Founder of Kirkpatrick Partners. As a global driving force of the use and implementation of the Kirkpatrick Model, Wendy leads companies to measurable success through training and evaluation. Wendy discusses how she got into training and assessments, and dives into the four levels of evaluation, why level three is the most important and how to measure behavior and her tips for evaluating training while remote working.

John Kleeman is joined by assessment luminary, Chris Norton, Director at Mentor Group, a global sales transformation company based in UK. Chris shares his background and experience with assessments and discusses the value and best practice advice on using assessments in the sales space, the new Questionmark B2B Sales Qualification by Mentor Group and the future of digital selling.

John Kleeman is joined by assessment luminary, Bernt Nilsen, founder of Datakoret (ECDL Norway) and Norsk Test, the leading test developer and supplier of national certification programs in Norway. Bernt discusses how he got into the assessment world and shares what's important when running a testing service business, his best practice advice for creating high-stakes assessments and the value in anonymous testing.

John Kleeman is joined by renowned attorney, Alan Thiemann of Hans Santos, PLLC and General Counsel to the Assocation of Test Publishers (ATP). Alan discusses how he got into testing and his role at ATP and dives into topics such as privacy and remote proctoring, how to avoid being taken to court for a bad test and testing controversies in the US.

John Kleeman is joined by assessment luminary, Liberty Munson, who is the Senior Psychometrician for Microsoft. Liberty discussess how she got into assessments, why people stay in jobs, the value of certifications and the future assessments, among other topics. Liberty also shares her aha! moment - listen to hear more!

John Kleeman is joined by assessment luminary, Jim Holm, CEO of Examity, the leading solution for learning validation and test security. Jim discusses how he got into his current role and the challenges he's faced with COVID-19, and dives into remote proctoring, including the value, his best practice advice, the future of proctoring and his views on artificial intelligence and proctoring.

John Kleeman is joined by assessment luminary, Paul Crump of Cambridge Assessment, the non-teaching department of the University of Cambridge that develops and delivers examinations and tests around the world through their three exam boards.  Crump discusses his background in assessments and dives into the Thinking Skills testing initiated by Cambridge Assessment and his best practice advice for those creating questions and assessment.

Host, John Kleeman is joined by assessment luminary, Sue Martin, a pioneer in the assessment industry responsible for implementing the first test center network in Europe. Sue shares her story and advice on transforming certification programs. Discover the five most important considerations when setting up your certification program.

John Kleeman is joined by assessment luminary, William G Harris, Chief Executive Officer of the Association of Test Publishers (ATP), the leading international organization for publishers of assessment programs. G Harris discusses his background in assessments and dives into his advice for those starting out in assessments, the importance of validity in assessments and the different sets of standards for testing, among other topics.

John Kleeman is joined by assessment luminary, Dave Foster, CEO of Caveon, the first test security company. Dave is a psychologist, psychometrician and recognized luminary in the measurement industry. In this episode, Dave discusses his background in assessments and dives into the main risks and threats of testing, his recommendations on the key measures you should take with test security and how statistical analysis fits into this, among other topics.

John Kleeman is joined by assessment luminary, Steve Dept, CEO and founding partner of cApStAn, a high-profile language service provider. Steve received his education in English, Dutch, French and German but he is essentially an autodidact and a field practitioner. In this episode, Steve discusses his background in assessments and dives into good practice advice for translating tests, including the translation process, translation technology and other translation resources.

John Kleeman is joined by one of Questionmark's product managers, Steve Lay. Steve is an expert on scalable, computerized assessment and integration between systems, and has ensured that the million of assessments delivered by Questionmark are robust and scalable. He is also active in the IMS and other e-learning standards groups. In this episode, John dives into many topics with Steve including the impact of technology on the assessment process, QTI and LTI and proctoring.

Host, John Kleeman is joined by assessment luminary, Jim Parry, who has over 40 years of experience as a course designer, developer and instructor. Jim discusses his 12-year tenure at the Coast Guard as a civilian employee as the Test Development and e-testing Manager, during which he guided the move from paper to online testing for the entire Coast Guard.