Quest for Success / SmarterMeasure Webinar


Mac Adkins: I’m looking forward to sharing
this information with you, I’m Mac Adkins with Smarter Services. I’ll be joined today by Christy Davis from
Chabot College, as well as Logan Murray and Bob Nash, both with California Community Colleges. And so, we’re glad that you’re here today,
just a few bits of housekeeping. Since this is a webinar, your lines are automatically
muted, but we would like to hear from you. So near the bottom of this interface, you
should see a chat icon, click that. We will be monitoring the chat so if you have
a question, feel free to type your question there. We may respond to that question during the
presentation. Chances are, we’ll be responding to the question
at the end of the presentation. We are planning on leaving time for questions. While now, you are seeing the video, for some
of us, we’ll be turning the video off during the webinar, just in case there are some users who are
on lower bandwidth connections, that way, we can have the best audio possible. I would like to inform you that today’s webinar
is being recorded and a link to the recording can be provided after the webinar if you are
interested. Also… today’s webinar is being captioned
for accessibility. So, thank you for joining us today. We’ll go ahead and get started and Logan,
you can advance us to the next slide and we’ll go from there. Briefly, the agenda for today, we’ll start
out with Dr. Bob Nash sharing some information with us really about the purpose of Quest,
and also of SmarterMeasure, working in tandem with it. Dr. Logan Murray will share some information
with us specifically about the Quest for Success program. I’ll give an overview of SmarterMeasurer,
then Christy Davis from Chabot will be sharing with us information that she’s currently
experiencing as they’re planning to do a pilot of both Quest and SmarterMeasure. And then I have more in-depth information
about SmarterMeasure to share with you, not only information about the assessment but
also some research that’s related to student success that I think you’ll find relevant. Okay, and you can go ahead and advance from there, Logan. So, we would like to know our audience today
so we have a question for you. And Logan will be launching this in the form
of a poll with two questions. First of all, is about your level of experience
or familiarity with the Quest for Success program. And then the second question has the same
scale and it’s about your level of experience and familiarity with the SmarterMeasure Learning
Readiness Indicator. And as Logan provides that poll to you, just,
the first option is that you don’t have any prior knowledge or experience and that’s okay,
so we’re glad you’re here today to learn. Second is that you’ve heard about these tools,
but you really have limited knowledge. The third level was that your school uses
Quest or SmarterMeasure, but… at a low level and you could improve that. And the fourth option would be that you’re
currently using Quest and/or SmarterMeasure at a high level and if so, we’re glad that
you’re here to contribute, perhaps, some of your experiences today as well. Bob Nash: There it is. Mac Adkins: So, there comes the poll and you
can provide your feedback to us there. Logan Murray: And, Mac, would you like us
to keep the poll open throughout? Mac Adkins: Let’s give the attendees … If
you’re seeing the results … if you can kind of share those results with us orally then
keep it open in case there are some attendees that join late. Logan Murray: Sure. Actually, they’re coming in pretty rapidly. We have 55%, 60% response rate so far. Mac Adkins: We’ll give it just a moment for
the participants to respond and then if you want to orally report on those results that
would be useful. Logan Murray: Would you like some of the results
now, Mac? Mac Adkins: Please. Logan Murray: Okay, so we’re at about 66%. In terms of question number one, relating
to experience and familiarity with the Quest for Success program, about 21% have said they have no prior knowledge. 45% have a little bit of knowledge, limited
knowledge. 30% have reported Quest use at a low level
and just 5% have reported Quest at a high level. With respect to the SmarterMeasure Learning
Readiness Indicator, now with 71% of the polling in, 35% have reported no prior knowledge or
experience. Another 35% have reported limited knowledge. 20% have reported use at a low level at their
institution and just 10% have reported use at a high level at their institution. Mac Adkins: Okay, excellent, thank you for
sharing those results with us. And that helps us understand you and your
level of experience and knowledge and we’ll be able to tailor our comments today reflecting
that. Okay… you can advance from there, Logan. And, so, Dr. Nash, you can share with us the information
you have. Bob Nash: Thank you, Mac, for promoting me. I wish I held a Ph.D. or EDD, but I do not. Sadly, that’s on my bucket list but it’ll
be for a future day, but I do hold a master’s degree, so maybe you can call me master. [laughter] I wanted to mention, just quickly, the reason
for Quest and SmarterMeasure. We’re all very familiar with the growth in
online education relative to on-site education. The on-site has been flat for years now and
online continues to grow. So we are experiencing more and more students
taking online courses for the first time, or taking more online courses or going fully
online where they had some mix before and that creates the conditions for which Quest
and SmarterMeasure can help us. In particular, I’d like to point out that
the CVC-OEI has been promoting fully online ADTs and now fully online certificates through
our website at cvc.edu. And, we feel that too, will bring students
who have decided not to get their education because they can’t fit it in their busy lives. I think we’ll see more and more students who
are getting back into the realm of that pursuit of their college degree. Probably older, certainly with jobs and children,
and to help set their expectations and improve their readiness for this type of learning
modality will be ever more important, increasingly important. And this is where SmarterMeasure and Quest
can help a great deal and it is free. This is a free resource for all the Consortium
colleges and the other California community colleges, unlike some of the other ecosystem
resources we offer. So… with this, I’ll hand it off to Logan
who will tell us more about the Quest product, itself. Logan Murray: Thanks, Bob. So… a little more about Quest for Success
and sort of the composition for the program. So… first of all, you know… Quest for Success is unique in that it, it’s
an online learner preparedness program that is comprised of both the SmarterMeasure Learning
Readiness Indicator, and so that’s a diagnostic designed to measure the readiness for students
in terms of an online education experience. That’s the diagnostic component to it. And then the other big components are the
homegrown OSRT, those are the online student readiness tutorials. And those are really designed to remediate
or address factors related to online student success, right? So, it’s designed to get students… get students
prepared for an online learning experience. And the way that the OSRT and Quest is set
up, is it’s designed for both novice beginning, students … beginning students that are new
to online learning, I should say. But it’s also … there’s also a different
pathway that’s designed to address the needs of more experienced online learners as well,
right? So we’re trying to hit a broad population
of students, if you will. The OSRT … the entire Quest program, including
the learning readiness indicator and the online student readiness tutorials, the interactive
multimedia tutorials that we’ve built are all designed to operate within the Canvas
learning management system. And then there are some different ways that,
that Quest can be, you know… can be deployed. There are aspects of the program that are
certainly customizable on both the learning readiness indicator side and on the student readiness
tutorial side. And that’s something, maybe somewhat out of
scope, for our conversation today, but something that would be, if you’re interested, in… in
deploying, the Quest for Success program, I’d certainly be … would want to have a
conversation with you about you know… some of those features that, and some of the options
in terms of… in terms of launching this at your campus. So I think that’s …I think that’s about
it on my end. Mac Adkins: Okay… and so… bear with us
just for a moment as we transition here, as Logan gives me screen presenter controls. So I don’t have to …awkwardly ask him to
forward. Let’s see… Logan Murray: Yep… So I’ve made you host, but … if that doesn’t work,
then I can just advance the slides. I’d be happy to do that. Mac Adkins: Let’s see… Logan Murray: Okay. Mac Adkins: How does that look? Are you seeing my screen on the next slide
now? Bob: Nash: SmarterMeasure, yes. Mac Adkins: Okay… very good. So thank you, Bob and Logan, for sharing that
information with us about Quest. And… so… I want to reiterate Bob’s comments there that
you can relax, no one is going to try to sell you anything today because it’s already provided
for. Sometimes we may think of something that’s
free is not valuable or cheap, but I can guarantee you that many hours of research and planning
and development have gone into both Quest and SmarterMeasure. So these are resources that are truly valuable,
but they have been funded for your institution. So I’m glad that the poll shared with us a
large percentage of the attendees today have limited experience or knowledge with these
resources, so we’re glad that you’re here today to learn about them. I want you to know that after this conversation
today if any of us who are panelists can help you implement these things at your institution,
that’s exactly what we’re here for, because… we’re all here to help students succeed. So let me begin with a little more information
about the SmarterMeasure Learning Readiness Indicator. As I was just saying, the reason we all come
to work every day is because we want students to improve their situation in life. And Bill and Melinda Gates, through their
foundation, they feel the same way. They understand that the best way to improve
a person’s standing in life, especially economically, is through education. And so they are pained by the fact that so
many persons feel that desire to continue their education, but then they drop out because
retention rates are not what any of us would want to be at our institutions. So just a few years ago, the Gates Foundation
did some research where they interviewed thousands of people who had started college, but then
dropped out and they asked them. They said why, why did you drop out? And so you
see the reasons here that people gave. Just conflicts with work schedule, they couldn’t
afford it, support from family, all of these reasons are what I would call noncognitive
reasons. None of the reasons were, you know… college
was too hard for me, it was just that my situation in life was not appropriate. There was more recent research done last year
by the North Carolina State University where they surveyed 6,000 two-year college students,
some of which were even at your institutions at the California community colleges. And asked these
community college students, what are your main challenges to success? When you’re enrolled, what do you struggle
with? And you see the results there. It’s just trying to juggle it with work, finances,
and family, and friends and… everything down to all that, all that you see there. And the only cognitive reason on the list
is the next to the bottom, of doing college-level work. So most of these reasons were noncognitive
factors. So what are noncognitive skills? These, you may call them soft skills, things
like motivation, procrastination, willingness to ask for help, locus of control, things
that kind of define the learner’s learning personality. Which, when you think about those things,
it begs the question of well… can noncognitive skills be taught? Or can they be improved? I really like the mental model that one of
our clients shared. He said “you know what? You may not be able to change a tiger’s stripes,
but you can teach a tiger to hunt in a different environment.” And that’s the spirit in which we offer the
SmarterMeasure Learning Readiness Indicator and the spirit in which Quest is offered,
as well. It is that what we’re trying to do here is
not necessarily change a person to be who they’re not, but to hold up a mirror to that
person, to let them see what they, as a tiger, look like. What do their stripes look like? And how can a tiger that looks like them,
that tiger may be someone who is prone to procrastinate or someone who has low technology
skills, or someone who has a very social learning style. Whatever their stripes look like, how can
a tiger that looks like them, be successful in this environment that we call Higher Ed? And that’s what this combination of SmarterMeasure
and Quest is really trying to do. So, the SmarterMeasure Learning Readiness
Indicator is an assessment that quantifies primarily these noncognitive habits and traits
and competencies and skills that your students have. It’s been around for 17 years now, since back
in 2002. During that period of time, it’s been taken
by over 4.8 million students, there are hundreds of schools that use it. It really is the most common learning readiness
tool that’s based on noncognitive factors. So what does it measure? We start out looking at the student internally,
we call those individual attributes. Those are things like the motivation and procrastination,
and willingness to ask for help. Also while we’re looking at them internally,
we identify their dominant learning style. That’s based on the multiple intelligences
model which is whether they’re social, solitary, visual, verbal, those sorts of things. After we’ve looked at the learner internally,
then we look at them externally, we call those life factors. Those are the things going on in the student’s
life. How many hours a week are they working? Do they have support from family that could
include babysitting? Do they have support from their employer? Really an environmental scan of what’s going
on in their life … health, finances, those sorts of factors. So after we’ve looked at the learner internally
and externally, then we look at their skill set. We look at their technology skills, their
keyboarding skills, their onscreen reading skills. So the full assessment is a combination of
a skills test, a personality inventory, and situation in life and assessment. So as you can see, it’s a rather exhaustive
assessment that measures quite a bit. I’ll come back in a moment and share with
you a little more about it, but we’ve asked Christy Davis from Chabot College to join
us today because she is in a position not far from where many of you are. Chabot was not currently using SmarterMeasure
within Quest. They’re in the process now of putting that in
place which is why we asked her to join us today. Because we know where she is, those of you
who are interested in using these tools will soon be in that same situation. So, Christy, thank you for joining us today. Christy Davis: Thank you, Mac, thanks for
having me. Can you hear me okay? Mac Adkins: I can. Christy Davis: Okay, excellent. So… as Mac mentioned, I am from Chabot College
and as some of you may or may not know, Chabot is an urban community college. And, like many of you, we have students who
face a lot of obstacles to their desire to further their education. They are first-generation college students,
students who face socioeconomic disadvantages. They’re students who must work to support
their families. They may come from cultures that do not place
a high value on continued education. We have students who struggled in high school,
both academically and socially. And then we have a certain number of students
who just don’t know how to ‘do’ college. And I know that this narrative is certainly
not unique to Chabot, but here at Chabot, we’ve been really looking at this idea of
persistence and how do we help our students succeed and continue to succeed. So we’ve been looking at the data and anecdotal
stories that have been told to us by our students, to find ways that we, as educators, can both
lower the hurdles at our own institution and also help students overcome the obstacles,
the personal obstacles that are blocking their path. Kind of like the noncognitive things that
Mac was talking about earlier. So… persistence, it really boils down to
why do students stay and why do they leave? We have an amazing institutional research
team here and they’ve been conducting a variety of surveys and reports over the last ten years
and they have kind of come up with some interesting things. First, why do students stay? They found that students tend to persist and
stay at Chabot to completion when they have a specific goal in mind; they have a pathway,
they know what they want to do. They tend to stay when they interact with
support services in a positive and meaningful way. Those support services being things like counseling,
they interact and engage with tutors. Perhaps they take study skills classes, all
of these things that help them feel engaged on campus and part of the campus community. The next thing that our institutional research
team looked at was, of course, why do students leave. Again, a lot of these things are similar to
all of our narratives, all of our student’s narratives. They have a lot of overall time pressures
and outside responsibilities with work and family commitments. They have difficulty studying because maybe
they were never taught how to study properly. They don’t generally understand how the college
system works, meaning that they’re unsure of what requirements they need to take, what
types of prerequisites they need to complete. And the last one, that really kind of surprised
me, was that many students felt it was too difficult to get student services. And so our research is really showing us that
students who have a path and receive personal attention, who are meaningfully engaged, who
receive assistance as they advance through their journey are more likely to be successful
and to persist. So what we wanted to do was to really kind
of look at, and what we’ve been trying to look at, is how do we help students effectively
utilize the services that we already offer? One of those old Proverbs comes to my head. You can lead a horse to water, but you cannot
make him drink, right? We have all of this water, how do we encourage
our students to drink? So Chabot faculty and staff have embraced
the idea of meeting students where they are, by creating and supporting services for students. So… again, this narrative, I realize, is not
unique to Chabot. Many of our schools struggle with this. Many of our students struggle with this. So… some of the services that we have are
things like the traditional academic and personal counseling, tutoring, and STEM and writing
workshops. We also have a communication lab for oral
communication for presentations. We have some really amazing learning-based
groups and programs such as our Pace Program, our Puente TRIO, and our Veterans services
programs. Again, very standard. A lot of you have these programs as well,
but again, how do we get students there? How do we get them to use those services? And then we’ve also been working on some programs
that are a little more out of the box. Things like our food pantry to help students
persist who are having food insecurities. We have a laptop lending program out of our
library so that students who can’t afford to have a laptop or mobile device are able
to complete their homework and to do their studying. We also have things like emergency hardships
scholarships, so if a student has something that comes up mid-semester, a bill, maybe a medical
bill or their car breaks down or something happens that’s going to get in their way,
they may be able to apply for and receive an emergency hardship scholarship to sort
of fill that gap. So those are just a few things we’ve been
trying to do as far as student services are concerned. And with so many amazing resources for our
students, it was really surprising at first to me, anyway, to see that students regularly
reported not knowing how to get these services or that it was too difficult to utilize the
services at Chabot. So… I’ve been an adjunct here at Chabot in the
Communications Department for over ten years. So I’m not new to Chabot, but I am new to my
position in distance education. I just started that new position in January
and that’s when I ran across the Quest and SmarterMeasure programs. And to me, it really seemed like kind of a
no-brainer. I immediately saw Quest as an opportunity
to help faculty get the word out about our student services because… as a faculty member
myself, as an adjunct, myself, I know a lot of times, I’m the first line of contact for
students. So, I saw Quest as a really great way to help
our faculty get the word out to students about our services. And, SmarterMeasure seemed like a really amazing
way to help students recognize some of those unique challenges that they’re facing. Because sometimes you don’t know you’re in
it, you’re just in it. And so… to have someone… something that
helps you recognize what’s going on, is going to help you succeed and get around or over
that hurdle. So… by linking the SmarterMeasure student
self-assessment to direct intervention from a counselor or faculty member, we hope to
use the overall interaction to build some of that engagement that students repeatedly
cite as part of their path to success. That kind of took me to, alright, how are
we going to make this happen and to looking at how do we create a pilot. So… to see if others really felt that there
would be value to this the way I did, I made an appointment with a counselor. And I know that hopefully the irony of that
is not lost on those of you who also teach in the classroom because we’re always trying
to get students to talk to counselors. But when we were converting from Blackboard to
Canvas back in 2017- 2018, I worked extensively with a handful of counselors to help get their
courses moved to Canvas. And so I contacted one of those counselors and ran
a demo of the Quest for Success and the SmarterMeasure program. The counselor, she was very impressed and
excited about what she saw and suggested a meeting with our Dean of Academic Pathways
and Student Success. So I completed another demo and the dean gave
me the go-ahead to start recruiting from her team of First Year Experience instructors
and counselors that teach in our psychology counseling academic programs to work with
me to create a pilot for our Quest and SmarterMeasure. And so, we actually have a number of First Year Experience
programs that will be participating in the pilot for fall in our STEM areas, our social
sciences, our business, our health and wellness, and then also our Change It Now program which
is a program focused on social justice. So… from speaking and working with the faculty,
as we’re working to build this pilot, really overall, they’re most excited about how easy
the Quest modules are to modify and to integrate our campus materials and their specific program
details into Canvas. So that’s one of the things that I really appreciate
now that I’m in the Distance Education Department, is that it’s going to be easy for them to
adopt. It’s going to be easy for them to customize
to make it their own and to be directly applicable to their students, so their students are getting
the most out of it. It really allows them to highlight also, their
specific program resources, such as subject-aligned tutoring centers, the learning-specific student
clubs that we have and also, the particular academic counselor that’s assigned to that area. So really, again, customizing the modules
and really making that information directly applicable to the students and giving them
the opportunity to… to really engage with it. Faculty that are participating in the pilot
also plan to use the SmarterMeasure self-assessment material to help their students recognize
their strengths as well as their challenges. Another outcome of using this SmarterMeasure
self-assessment that we’re working towards is as a catalyst for dialogue between faculty
and students. And again… what we’ve seen… all of us have
seen in the research is talked about as a faculty member, we’re really sort of the frontline to connect with our students. Whether it’s in a face-to-face classroom,
a hybrid classroom, or an online classroom, students look to us for that feeling of welcoming,
that sense of engagement with their studies and the community campus as a whole. So… we’ve been working with Mac’s team to
have the results of the self-assessment sent directly to the faculty members and/or the
student’s counselor. Again… with the idea of… to create that
catalyst for engagement. The student will receive a message that is
tailored so that they understand that their faculty or their counselor, whoever’s in charge
of their program, will be contacting them directly to discuss the assessment. So again,… sort of that idea of directed
interaction with the student and the faculty member or the counselor. And that will allow the faculty member or
counselor to use the results from the SmarterMeasure assessment to really, in that follow-up meeting,
help them increase recognition of the different services, as well as hopefully increase the
interaction with the various services that can help students overcome the hurdles that
they’ve self-identified using the SmarterMeasure assessment. So really, what we’re hoping is that this…
that this kind of all comes together to help us deal with some of those reasons that students
have said in our surveys that they’re not persisting. So overall, our hope is that the combination
of the Quest for Success and SmarterMeasure will help us help our students to persist
and thrive at Chabot College. We believe this will help us address some
of the narrative shared by our students over the years about why they choose to leave school. And as we know, persistence can change failure
into an amazing triumph and so we are hoping at the end of our pilot to have a lot of triumphs. Thank you, Mac. Mac Adkins: Alright, thank you, Christy. I really appreciate you sharing that. And I just want to remind our attendees that, if
you have questions or comments or observations you would like to share with Christy or any
of us, please type those in the chat dialogue box here in Zoom and we will be responding
to those at the end of the webinar. So… thank you, Christy, for sharing that
information about the personal obstacles that your students face and the way that you guys
at Chabot are responding to that. You know… to continue that metaphor of,
you can lead a horse to water, but not make it drink, imagine a horse that’s dehydrated
and does not even know it. There are some horses out there that don’t
know they need water and then sometimes there are horses that know they need water but they
don’t know where to find it. That’s exactly what we’re trying to do through
these combined resources of SmarterMeasure and Quest is, first of all, help that horse
realize that they’re thirsty. And so, all of the factors there that Christy mentioned
about reasons for dropping out, about not having a strong reason for going to college,
not knowing what student service is available, or the reasons for persistence, about managing
time, experiences and knowing where to find services. All of those things are variables that are
measured by SmarterMeasure. So, imagine having a quantitative measure of these
factors that really influence a student’s decision to persist or not. That really helps you help that horse know
whether or not they need to get water. And then as far as telling that horse where
the water is, as you’ll see in a moment, in the school report, there is a section at the
bottom of each scale that we measure, where you can inform the student of resources for
support. It’s editable so if there are certain resources
that your school has, just like the ones that Christy was describing there about some of
the special things they do with providing food and providing laptops. You can use SmarterMeasure and the school
report to inform students about that. So… thank you for that information, Christy. It really helps set the context. Let me share with you a little bit about SmarterMeasure. There are different ways that schools use
it. There’s no right and wrong way to use it,
you decide what’s the best way for you. You heard a very good use case and rationale
from Christy at Chabot. But what it is is really an optic. I mentioned a mirror earlier that you hold
up to the tiger. But it’s a lens that helps that student see themselves
and understand their strengths and their opportunities for improvement by doing so. It’s a great student service, but it also
allows them to be informed of the student services that you provide. There are some schools that use it for placement
to help determine which students may need some developmental or remedial course placement. One of the best uses of it is for a talking
point for your academic advisors or success coaches – whatever you call them. In fact, one of the documents we provide is
called the educator report. And we offer a key to that report which literally
puts words in the mouth of what your advisor can say back to a student based on their SmarterMeasure
score. Obviously, the SmarterMeasure scores are a
great early alert because they’re indicating which students are at a higher likelihood
not to persist. And when you do that, as you’ll see in a moment
from some of the research data, it also becomes a very predictive analytic as far as retention
and students who need support. So… how do schools use it? Obviously what we’re talking about here is
using it in the student support modules through Quest. There are some other ways that schools use
it. They put it in orientation courses. Obviously, they equip their advisors with
it, use it in First Year Experience programs or quality enhancement programs. Some schools put it in their onboarding, their
enrollment services or even in recruitment. So there’s no right and wrong way to use it,
there’s just the way that best fits your school. So let me share with you a little bit of research
about it. This is really what I’m passionate about as
an academic. I’ve made my life’s work about measuring noncognitive
readiness and helping students with that. And so… here’s some information that was
actually collected a couple years ago from the California community colleges. One of your schools there, I’m not even sure
how to correctly pronounce that as you can tell by the way I’m speaking, I’m not from
California, but you guys recognize that school [Cosumnes College]. And so… what they did is they integrated
these tools into their orientation or they asked a question, rather, in their final exam,
where they asked the students, “Okay, now that you’re at the end of this experience,
what would you recommend to other students? Do you think they should have a required class? Do you think there should be an optional class?” So these students at the end of their program
looked back on it and they said, “You know what, I was really a thirsty horse and I didn’t know where to find
the water. But for the horses coming along after me, we need to help them.” So even students agreed that it’s better if
there’s a required class out there to help these students. The RP group, there within the California
community colleges, also did some surveying of students that had done Quest and SmarterMeasure
and asking them to identify and to rank the services that they found useful in that experience. Some are listed there for novice students
and you see it, that a high percentage of them said SmarterMeasure was very useful. Other modules like Introduction to Online
Learning and Getting Tech-Ready were very useful for new students. And then even for experienced students, many
of the modules in Quest are really helpful to help those experienced students as well. Some of the other research that we’ve done
in other schools, here’s Middlesex Community College. They were having a problem with failure rates
in their online courses. You can see back in the blue bottoms there,
in the first two years, like 1/3 of their students were failing in their online courses. And so the green line indicates where they
started using SmarterMeasure and having those conversations with students about their SmarterMeasure
scores. And you’ll see that just over a couple years
time, they reduced those failure rates by almost 10%. So that was substantial. Ashford University, headquartered in California
looked at the additional value, that adding this noncognitive data gave to their predictive
analytic about which students would drop out. And as you can see there, adding the noncognitive
data onto the demographic data improved their predictive algorithm by about 4%, which is
substantial. Career Education Corporation did an analysis
of statistically significant relationships between those key student performance indicators
across the top. And then the scales that we’re measuring,
you can see that most of them were statistically significant in that relationship. Another community college, J. Sargeant Reynolds,
they got to noticing that one of the things you’ll see in a moment that we do with SmarterMeasure
is we classify the students as being at a high, medium, or low level of readiness. That’s a classification that the student doesn’t
see, but only the faculty member or the administrator sees that classification. But of their students who are classified as
being at a low level of readiness, less than 10% of those students went on to experience
academic success. You’re probably familiar with Noel-Levitz. They provide lots of great resources to the
Higher Ed community. They have a survey that schools often give
at the end of an online program, it’s called a priority survey for online learners and
it measures satisfaction. So as a student is completing your program, they
ask how satisfied were you with several elements of the program? So we were wondering, what’s the correlation
between incoming student readiness and outgoing student satisfaction? As you can see, there were several statistically
significant relationships between those two families of variables. This is just a lot of numbers here, but just
to communicate to you that we do collect lots of data, there are lots of ways that we can
analyze that data. As Christy was saying, many of you guys have
great IR Departments at your school that can take a deep dive into the data. But here was just looking at the distribution
from a little over 7,000 students, from California community colleges over the past few months,
who have taken SmarterMeasure, so we compute all of the measures of central tendencies
so you can compare your student’s performance to the performance at your peer institutions. Just a couple slides here to kind of give
you a visual representation of the distribution of the scores from the scales that we measure
and those of you that enjoy stats classes like I do, you’ll see these are normally distributed. It looks like that bell curve there, which
is what you want to see with a group of data this large. So… I want to show you just a few slides here,
we’ve got about five or six more minutes of this part of the webinar, so that you can experience what it looks like
to the student. SmarterMeasure is embedded into Quest, it
also can be provided in a custom user interface that’s branded with your school’s identity. Just a few screens about what it looks like
as the student is going through and taking the assessment. I do want to point out here that obviously
with this being around for 19 years and hundreds of schools using it, we’ve gone through extensive
accessibility testing. A few of these things are evident on this
screen. You’ll see the little As there that are larger
so if a student needs to enlarge the text, they can do that. They’re always one click away from our 24/7
help desk. They can start and stop the assessment and
return back to it at any time. Just another screen or two. A couple of screens here are just showing
from our school report, how the scores are presented both textually and graphically to
help the student understand their scores. For each of the scales that we’re measuring,
that are listed on the left, there’s going to be a full page of information, that not
only presents their quantitative score, but also provides qualitative feedback that is
score specific. At the bottom of that page is where those
resources for support are presented. Just a few shots from the administrative panel,
showing you that there are lots of ways you can search and export the data. A few of the views here of administrative
reporting. The image there on the left, you’ll see the
green, blue and red icons, which are indicating the high, medium, and low levels of readiness. So imagine you’re an academic advisor, and
that list of students there on the left is part of your caseload. You can tell quickly which students you may
need to reach out to and which ones will be most at risk. The image on the right is also data provided
to advisors that shows them not only for the scales but also for each subscale, what the
student score is. So the advisor can quickly tell what they
need to be in communication with the student about. You can adjust the cut points that make that
determination of high, medium, and low and we actually even work with schools to look
at the distribution of your data to determine what is the best placement of those cut points
for your students. A few shots here that just show you some of
the dashboarding of your data that we provide. If you’re a data junkie like I am, lots of
ways that you can visually see the data. And it’s really interesting to look at these
trends about your students. For example… let’s say after you administered
this assessment to a few hundred of your students and you look at trend data, such as this…
we’ve had schools to say… “you know… this data is showing me that many of our students
are really struggling with finding an appropriate place to study.” And so… maybe they then add a unit to their
orientation course or whatever their approach is to, to improve awareness among their students
about finding an appropriate place. And just more views of the administrative
reporting in the admin panel and… a fresh new look is coming to the admin panel soon. For those of you who are already using SmarterMeasure,
you’ll be glad to know that you’ll see even more data at a glance in these reports coming
soon. Put in a few slides here, because… you know…
to kind of you know… I get into data, right? But data is aggregate and represents a lot
of students, but really, the reason we do this is to help that one individual student. So I wanted to share with you just a couple
slides of feedback, this is from a school, Auburn University in Montgomery that piloted
last fall. And to kind of get a feel from their students about how valuable they perceived this to
be, they asked the students to provide some feedback. And so, you’ll see some of the things that the students
said there. ‘You know… to help overcome these obstacles
I started putting myself out there more, meaning I take responsibility for what happens in
my life. Over the term, I had to take a step back and
realize things happen to a person for a reason and I need to own up to things.’ That’s what warms our hearts as academic advisors,
right? Informing that student and getting them to
take responsibility. It’s just more feedback from students and
we’re going to share this slide deck with you so you can read these more slowly. But… I would encourage you, as Christy was doing
at Chabot, to collect this qualitative feedback from your students so they can … you can
understand the way they perceived it. In case you’re not familiar with us as a company,
SmarterServices, as I said, has been around since 2002. We serve hundreds of clients, eight different
statewide consortia, including you guys there in California. And many of you have seen us out on the road,
we go to about 20 conferences per year. Some of the things you may not know about
us, is with our other products, we have partnerships with ProctorU, Examity, Google, BlackBoard,
and Canvas. We do lots in the Higher Ed community and
for that, we’ve even recently been honored by the governor of our state. As well as … I received an award recently from
the school where I graduated, Bob. And that was a personal achievement and professional achievement
that I was proud of. But… we do provide three services SmarterMeasure, that we’ve talked about
here, SmarterID, there’s so much excitement about this. It uses facial recognition during the course,
not just during a final exam, to make sure the student that’s doing the work in the online
course is the person that should be doing the work and not an overeager parent or a
spouse or a tutor or someone they’ve paid or an assistant coach or someone else. And then we also provide SmarterProctoring
which is a multimodal proctoring process management system. So you can learn more about us there at SmarterServices
and there is my contact information if you’ve got any questions. I will pause there, it’s exactly 2:45 and
so we’ll now look at our questions and Logan I’ll let you kind of move that part of our
conversation. Logan Murray: Yeah, thank you…thanks, Mac. So we had a few questions in the chat. I’ll just relay them… sort of as they came
in. The first question, let’s see… ‘SmarterMeasure results can be forwarded to
counselors or faculty. Would this be set up within the LTI?’ So, can you describe how those SmarterMeasure
results can be communicated to certain people on campus? Mac Adkins: Sure… several different options
there. One option that most schools use is that immediately
when the student finishes taking the assessment, they see their score report. Copies of that
score report can be sent to other persons, such as faculty members or administrators,
if you so choose. But as Christy was saying at Chabot, they
want to intercede at that moment. And so, instead of the student immediately seeing
the school report, the student will just see a message that says ‘Thank you for taking
this assessment, your academic advisor will be reaching out to you to talk with you about
the results.’ And so, in that way, you know… they’re going to
make sure that the appropriate person follows up with the student. So there are several different workflows and
we can accommodate those workflows through our administrative services on the back end. Logan Murray: Thanks for that. And then a related question here, and this
is having to do with the admin panel the and reporting. So… the person asks… ‘we started using
Quest and SmarterMeasure and notice that the results are not associated with student ID
numbers.’ So I think they’re referring to the admin
panel. ‘We can see their first name and last name. Is there an additional field that can be added
to allow students to add their school ID information?’ Or perhaps maybe an automated workflow for
that? Mac Adkins: Yeah, both. If you want to add custom questions such as
having the student report a student ID or maybe there’s just some other that data you’d like
to collect from the student, you can add custom questions. And/or… through the integration, obviously
we’re integrating into your learning management system, and that’s where our system knows
that student ID and perhaps, what you’re seeing is on the data export when it exports it into
Excel, that it’s not exporting that ID. Whoever that is, if you’ll get with me, individually,
then… I can reach out to our developers to make
sure we add that to your report. But obviously, we know who which student is which one, so…
that ID should be able to be displayed. Logan Murray: Okay… thanks for that, Mac. That’s all the questions that I found on the
chat but… we do have, it looks like, another, you know… ten, eleven minutes to open it
up to the audience here to ask their questions. Bob Nash: And, Logan, let me grab two of those
minutes to describe another deployment method I’ve seen at several colleges. They take our Quest generic course in the
IMSCC file format, upload it into their college instance and they can have students self-enroll
or they enroll all new students to the Quest course. And it is a Canvas course. The first module in that course is the SmarterMeasure
instrument that is integrated through LTI through key in secret that… Mac arranges a unique key in secret for each
college. You’ll see students and their student ID numbers
in Canvas in the Quest course, and each module of that includes the video tutorial, followed
by a quiz. The question of whether or not you can associate
the student with the SID number in SmarterMeasure is what we’re talking about, that was the
prior question. Logan Murray: Yeah, thanks for that clarification,
Bob. Mac Adkins: While you guys may be typing other
questions, one thought I had as the information was being shared is, I just want to tell you
guys what a great resource and Christy was commenting on this, especially with a student
online readiness tutorials. Those things were created under a creative
commons license, so they’re open source. And so obviously, as a company, we work with schools
outside of California as well and several times I’ve shown schools those as an example,
either that they can, you know… point to, because they’re open source obviously,
they’re branded from the OEI or just to see an example of a very well done tutorial. So if you haven’t taken time to look at those
student online readiness tutorials, do so. They’re really well done. They’re really student-friendly in that they
would, they would not be perceived by the student as something that’s you know…
demeaning or cheesy, but really useful. So… I really do highly recommend those. Logan Murray: We have another question from
D Kemp. And hopefully, I’m characterizing the question
appropriately here. They write, ‘I’m curious as to who you started
this with, Christy? Did you reach out to counseling first? I have it all set up in Canvas, but now I
need to get this to live with someone other than our DE Office of one.’ Christy Davis: So, Logan, I can go ahead and take that
one. Logan Murray: Sure. Christy Davis: Just so that you know, D Kemp,
as you know I’m also a DE Office of one, so I feel your pain. The first thing I did was…actually I went to one of
the counselors that I had worked with, helping her convert her counseling courses from Blackboard
to Canvas because I was originally brought on to help as a faculty mentor and trainer
and ran workshops and that sort of thing during the conversion. So I had worked quite a bit with her to move
her courses from Blackboard to Canvas. Because our counselors here also are responsible
since they are considered faculty, they’re also responsible for teaching a variety of
courses over the span of the year. I started with her to see if this was something
that she felt would add value to her courses. She liked what she saw, was interested, that’s
when I went to the Dean of sort of student success and kind of got permission to start
talking with other counselors. So… really, what I’ll be doing for this
pilot is, over the summer, working with the different counselors who are going to be implementing
this as well as the different faculty who are in charge of the First Year Experience
programs and pathways. I’m going to be working with them to help
them customize the Quest and the SmarterMeasure for use in their courses. I also have a couple of instructors who would
like to take modules, specific modules and embed them in their current courses, instead
of doing it the other way around. But, as far as where it lives, it still sort
of lives with me in the Distance Education Department of one. I hope that answers the question. Logan Murray: Thank you, another question
came in the chat, this is this from Nicole. ‘How would you recommend adding students through
the SIS? I’m wondering if the enrollment would be overwhelming
for students in a single course. I’m interested in getting students in the
Quest course, but it’s not mandatory on our campus at this point.’ Bob, can you take that one? Bob Nash: I’ll do my best. I had a prior position as DE Coordinator at
Coastline Community College and we imported the Quest course and manually enrolled through
CSV files, each term, all the students from a download that we saw enrolled in an online
course. Since I left, I’m told they have a more elegant
system where they auto-enroll these folks, all students who are new to distance learning
and excluding those who are not. And… if you want, I can connect you with
the Coast District folks to do a little consulting there. Also, we might have power users of Quest in the
room. I saw a few in the participants’ list. So… if anyone listening has another solution
for enrolling students in a Quest course to reduce the workload, feel free to share. Logan Murray: Thanks, Bob. Hopefully, it’s not too premature to mention
here too, that… cross my fingers, that Bob and I, at a certain point, will probably be reaching
out to members of the community to get a work group together to create recommendations and
to have a discussion about sort of, improved … making improvements to Quest and you know…
creating models for deployment and recommendations for deployment and so forth. So… a call for participation will be going
out at some point. Anything else you want to say about that,
Bob? Bob Nash: I was on mute there, yeah… we
have had this version of Quest now for a couple years so it’s time to do a needs assessment
and… look at all the aspects of it, including deployment. As we normally do, we’ll be conducting that
with probably a work group of eight or so colleges now using Quest to gather their input and come out
with version 2.0 in about a year. Logan Murray: Right, thanks. It looks like Michelle would like to share
a comment verbally. Is there any way that she can be unmuted? Michelle Pacansky-Brock: I’m unmuted, I’m
good. Logan Murray: All right… good, gotcha. Michelle Pacansky-Brock: Thanks, I’ll make
this quick. I work on the CVCOEI team in professional
development. And I’m a faculty mentor and I’ve taught online
for many, many years. I think all of these strategies to best support
our students to success is… they’re all so important. I just want to be sure that when we’re looking
at data bar graphs, about our students in a course… that we be very mindful about
recognizing the numerous factors that we are not aware of that our students are going through
and… remember that being a human, interacting with our students and being sure that our
students know that there’s someone on the other side of that computer screen that cares
about them is a really … data shows, that’s really the driver. And… we’ve recently found out that you know…
through the recent report from the Hope Center, more than 50% of the students in our system
experience threats of food insecurity and homelessness. And… those kinds of things just really have
to be remembered so… I just want to put that in as an important
part of the context. It’s important to just really remember the
importance of teaching, being present as an instructor and… caring about our students,
digging beyond the graphs and the bars that you know, the data that we collect and reaching
out and… understanding what’s really happening on their end before we make judgments about
what some of those things mean. Bob Nash: Thank you, Michelle. Also… I’d like to add there, are certain interventions
available in the marketplace, if you will, that can automate early alerts to students
and this is up to each college to decide. But I much prefer given what Michelle has
shared, that a human being reaches out to a student who finds themselves behind in a Canvas
course or whatever. I think that’ll be much more effective, even
though it does take some personpower, the result will be worth the effort. Logan Murray: Yeah… thanks for that. Looks like we’ve got just a couple minutes
left. I don’t see any other questions coming in. Any final comments that you’d like to share
Bob or Mac, perhaps? Mac Adkins: Yeah…I just want to echo what
Michelle was sharing there. She’s done a great job in emphasizing, you
know… humanizing not only our courses but also our student services, and that’s exactly
why we all come to work every day, is to help students, right? And so… those of us who have served as panelists
today, we’re all passionate about providing these resources, Quest and SmarterMeasure
to your students with the end result of helping that individual student. So… thank you, by joining us today and illustrating
your commitment to that as well through your attendance today in the webinar. We’ll be following up with an email afterwards
through which we can give you a link to the recording if you need that. Feel free to reach out to any of us by email
if we can assist you in some way. So… thank you for your time today. Logan Murray: Thank you very much, Mac. Bob Nash: Let me just finish by adding that…
reminding the Quest course modules are public domain, open for your editing. And… use as you see fit on your campus. The SmarterMeasure tool is paid for but the
CVC-OEI pays for the license and all California community colleges can take advantage of that,
either separately or combined with Quest. You can use Quest, you can use SmarterMeasure,
or we would like to think you would use both in an integrated fashion as we described. Logan Murray: Yeah… if I can just add a
final final thought. If you’re interested, if you haven’t yet deployed
Quest on your campus and you have an interest in doing so, please feel free to contact me
directly and I can get you started on that journey. And… you know… we can …if you have,
you know… specific considerations in terms of customizing Quest… you know, for your
college’s needs, I can get you started down that road as well. So I would welcome those, those inquiries
and that conversation. I’ll just include my email in the chat, one
final time. Bob Nash: All right… thank you, all, so
much. The recording will be later available, we
sure appreciate your attendance. Have a great afternoon. Logan Murray: Thanks, everyone.

Leave a Reply

Your email address will not be published. Required fields are marked *