LMS RFPs and Blaming the System

This great post by Michael in the Chronicle highlights, again, the challenges with how particular technology decisions get made at universities. If you’ve not had the chance to read it, go do so now. The article highlights a number of items related to purchasing online learning software that we all love to gripe about. I agree with Michael that this process can be (and generally needs to be) vastly improved, and I’d like to offer a few additional thoughts to this discussion.

Question historical practices and review your actual procurement rules

Working for a state government, people often joke with me that government can’t get anything right or done. I usually acknowledge the comment with a brief chuckle, but yes there are things that government accomplishes well, though getting things done can be downright frustrating when the procurement of goods and services is involved. I’ve been challenged several times over the past several years to identify the root causes/reasons why we do things a certain way. For what it’s worth, the reason more often than not is not always the bureaucracy of complex regulation (as is the case for procurement), but rather just a bad precedent set long ago that now continues unchallenged.

In the case of educational technology procurement, however, I found something interesting in my research. In Maryland code §11–203(e), the university system in select cases has been exempted from following the full extent of state procurement policies. University system policies further refine this to include a very specific exemption such that “University Policies do not apply to: Contracts for the purchase, use, or development of curricular materials.” My university places the specific authority for judgment calls related to this procurement area to the “Associate Vice President for Procurement and Business Affairs” and “Assistant Vice President of Strategic Contracting” specifying that decisions “shall be handled in a business like manner as best serves the interest of the University.” While the system policy was originally written in 1999, the Board of Regents in January of this year received a recommendation to further clarify this exemption specifically “to include Learning Management Systems for curricular purposes, library materials and the like.” I suspect that this is because these technologies have become so pervasive in the delivery of curricular materials today. 

If you think about this, and I am not a lawyer, this almost looks like decisions related to purchases impacting certain aspects of teaching and learning don’t really have to go through anywhere near as much red tape and bureaucracy as your typical government procurement process.  So before you go through all the hassle, search for the list of exceptions to your university’s procurement rules. By Google, I’ve actually found many examples of schools that include language to this effect. Maybe we’re unnecessarily perpetuating practices that are more difficult than they need to be. This was an eye-opening discovery for me, but like an LMS minivan, your mileage may vary when you attempt to pursue this path.  I’ve also never quite seen a requirement that a 30-person committee be used to make a software purchasing decision either. Maybe that’s one of the made up rules. Let’s be honest, though. Schools, one way or another, are going to find a way to get what they want anyhow.

Addition: someone tipped me off to the fact that some university systems and state governments have pre-negotiated contracts and pricing with multiple LMS vendors, so be sure to check this option out, too.

Consider what differentiates your institution and how you actually use your LMS

I love it to my core that universities are huge proponents of diversity and inclusion of people and opinions. However, I think the thing that universities struggle with is drawing that line between hearing about one opinion from one stakeholder about a Second Life integration with their LMS and actually mandating that it be part of the solution for everyone.  One professor’s opinion on one tool, as passionate and valid as it may be, really should not drive the decision for everyone. Rather, schools would do well to put more focus on determining the type of experiences they want to provide their students and then which strategies, practices, systems, and features support that goal rather than jumping right into a jumbled list of bullet points. Schools are not software product shops, but in my opinion many do need to get better at defining their product (ie: what an education from my university should be, the online experience included).

At the same time, schools need to be pragmatic about their choice of LMS. Though teaching and learning might be complex – its unknowns still being revealed by learning science – people just aren’t doing complex things with LMSs. I would argue that there is probably more to be gained from having a well-designed course than using a spiffy new feature in your LMS. Want the facts? There is surprisingly little public information on this specific topic, so you’ll have to go query your own LMS’s usage statistics to see data on which features are used the most. Take a hard look at that data and tell me that the top 3 or 4 used features aren’t:

  • Uploading/viewing PowerPoints, PDFs, links, or other content
  • Checking grades
  • Viewing news/announcements
  • Maybe even submitting assignments is in the top half, too

Then there’s a long tail of lesser-used features (and barely used features). I would bet quizzing and discussion forums don’t even make it into the upper half, and that the most common use of content adaptivity is to automatically open the folder to the next week’s learning materials without human intervention. Even Canvas (the “new hotness” in LMSs) released some really great data recently on how many LTI tools are being used by their customers. LTI is probably the easiest way to integrate additional, powerful learning tools into any LMS – and the average per institution was 8. Eight. And the top use of LTI was merely to link to content. Sadly, this does not show evidence of a Cambrian explosion of learning tools that I expected when LTI became tangibly real over 7 years ago. (Note: Instructure has since sent me a note that this number only reflects the tools installed at the root level for all instructors to use and not the total of all used in individual classrooms, so maybe all hope is not lost.)

My point is, let’s not kid ourselves. There is much more value to designing a well-thought out and pedagogically-sound assignment than supporting some new way of submitting an assignment via an Internet of Things brain wave device which has a known issue of not working quite right for users with thick skulls. There are always great demos and great stories about how individual instructors are using LMSs. They are great, truly exemplary (and probably took an awful lot of work and patience to pull off). However, this is just not generally the case. In fact, there are still large demographics of faculty who don’t really use LMSs much – in particular those in math and engineering for whom it’s frankly still easier to do things by hand. (I don’t exactly count My Math Lab as an LMS, though some do.) If any of this isn’t the case for your institution, I’ll happily buy you a beverage of your choosing the next time we meet.

Believe me, you can live without that one professor’s pet feature, and to ease their pain just remind them that it would probably break in the next upgrade anyhow. The bottom line is that most LMSs are used in generally the same ways to do the same basic things. If you want to make your university stand out, the LMS will likely play a role, but the responsibility is still on you to wow your students with great teaching and course design despite the LMS chosen.

Ground yourself in the realities of LMSs

The number of dominant LMSs is shrinking as fast as the number of Republican presidential candidates – even faster now that Pearson is removing itself from the running. Though Pearson has not yet endorsed any one of the others, if you are going to play it safe there are really only a handful to choose from these days: Blackboard Learn (in its current not-so-Ultra form), D2L Brightspace, Instructure Canvas, and Moodle. Maybe Sakai, too, but that’s really it. If you really want to be crazy or innovative you can be bold and choose a non-LMS-based approach, or bet big on a new entrant, or let us know how Blackboard Ultra works out for you.

For now though, I just don’t see any one LMS out there as much different than any other. Don’t get me wrong, there are definitely reasons to choose one over another. There are new, modern hosting models, improvements in reliability and maintainability, improvements to usability, and some nice new features here and there that begin to scratch the surface of advancements to teaching and learning practices. But there is nothing too wildly different from 10 or even 15 years ago. There have been no great breakthroughs and only incremental improvements.

Maybe this is because the LMSs of today are generally meeting our current needs, or mostly meeting most people’s needs. Or maybe we’re setting the bar too low or not dreaming big enough. While LMSs are certainly bloated, many users still use them as nothing more than a web-based shared drive with a gradebook and a discussion forum. Again, yes there are exceptions and great case studies of individuals doing great things that we should strive to emulate. But I’d still argue that this is just not the norm. And I don’t know of anyone who is using that Second Life integration in real life though it did demo well, once.

If you’re in the market for an LMS – checklists will probably always be a convenient way to compare apples to apples, but I’d argue that you need to focus more on questions such as:

  • At a basic level, will the LMS vendor keep my system up and running with minimal defects?
  • Will the vendor respond to my support tickets and enhancement requests in a timely manner and actually resolve them?
  • Is a one-size-LMS-fits-all approach still the right approach for your institution?
  • How easy is it to get your job done using this LMS?
  • Will your students enjoy using this LMS?

For these last questions, you need to actually spend time using the product, not just compare spreadsheets and demo-ware. Think about it – while (I hope) you would use a checklist as one data point when buying a new car, dare I say an LMS minivan, would you buy it without test driving it? And when comparing vehicles, I will tell you this – at least for minivans certain auto manufacturers have paid attention to detail in the layout of the vehicles and features such as automatic trunks and doors that make your “workflows” as a parent much, much easier.  Following the test drive analogy, institutions should try before you buy, even if this means investing some budget in a pilot or sandbox. And you should make your decision based on an experience, not a checklist.

Advice to LMS vendors

In higher ed, reputation is important, and perception is reality. Relationships, transparency, openness to debate, and sharing of information are all critically important criteria on which a buy decision will get made. These are things that you are judged on, not just product characteristics. You don’t always lose because of the product.

In sales situations, you will want to get to know us, our technology systems, our peer institutions, and how we work together. Also, don’t be afraid to push back and advise or guide us when you know that we can use the help.  Just do it gently and professionally and not in an arrogant or condescending way – and not as a secret sales tactic for professional consulting services unless we ask for that.

Understand that no faculty member ever builds their course from scratch every semester with new content and to take advantage of all of your product’s new capabilities. They already have enough to do. Speaking on their behalf, most seem to enjoy the delivery of the course material more than its construction – as is evidenced by the persistence of the textbook as a convenient (though loathed) jump-starter to this process.

Some of the major reasons that you might not win include – someone’s been burned by something that your software did, either 10 years ago or yesterday, or offended by something someone somewhere in your company said once at a conference in the bathroom. It may also be because you didn’t get some paperwork in by some arbitrary deadline. Again, the reason may not even be product related. Even if you don’t win, though, please don’t let the relationship sour. You never know when a new LMS implementation will fail, and you will be in a position to save the day.

Finally, ask yourself the question – does Apple win or lose RFPs because it has a Second Life integration? No. That would be ridiculous. The company that provides a reliable, polished LMS with a solid and flexible core feature set and supports their customers with genuine empathy and great people will win. Bonus points for a strong culture and good vision.

And finally

We as institutions need to think much harder about what we want our student learning experiences to be, not only in real life but online.  This takes conscious thought and effort. In my mind, the most powerful and meaningful work to be done is not driven by the LMS technology chosen but by the design of the course, the nuance of each discipline, and the interactions among students in their course with their instructors. Those features on that checklist probably are not going to amount to much, if anything. Instead, focus on what really matters, understanding that there are newer, better approaches for teaching and learning and that an LMS might not always be the only or best solution.

Still, one thing is clear in my mind. Making an LMS selection does not have to be as difficult or convoluted as many make it. But before being too quick to blame the system, we must also acknowledge that we are the system. While this is certainly a tricky, multifaceted problem, only we can make the process better though effort, hard work, gumption, and persistence. And yes, I assure you, it is possible to make this happen.

-George Kroner (personal thoughts do not represent my employer’s)