Aziza Ellozy and Caroline Mitry One of the biggest challenges that faculty developers face in the US (Johnson, Adams Becker, Estrada & Freeman, 2014) and elsewhere is to enhance the digital literacy of faculty, and more often than not the benchmark by which faculty measure themselves is by the use of a Learning Management System (LMS)1.
Last year at the request of AUC’s CTO, Ms. Nagwa Nicola, and of University Academic Computing Technologies (UACT), the CLT conducted an analysis on the use of Blackboard (Bb) by our faculty and students. Because of the very small number of faculty using Moodle, we did not include it in our study. All three parties were interested in evaluating the extent to which Blackboard is used (are we using university resources effectively?) and to see what can be improved.
In addition, CLT was interested in other questions that researchers in the field had been posing about the LMS, namely:
- Does the LMS enable learning or does it simply enable the administration of learning?
- How effective is the LMS in the service of learning and teaching?
- To what extent are its more advanced features used by instructors?
- Could we get information/insights regarding the digital know-how of our faculty by following their digital footprints as measured by data analytics from Blackboard?
- How do we compare with US faculty as reflected by recent studies?
In this newsletter we summarize how we tracked and analyzed Blackboard data usage of both faculty and students over a period of two semesters (F2014 and S2015) and how we attempted to answer some of the questions posed above.
What Does Research at US Universities Tell Us?
We chose to concentrate on the comprehensive research undertaken by the Educause Center of Analysis and Research (ECAR) in 2014 (Dahlstrom, Brooks & Bischel, 2014) which surveyed 17,000 faculty in 151 institutions in the US and more than 75,000 students from 213 institutions. They found that while 85% of faculty use an LMS, only 41% report using it “to promote interaction outside the classroom”. They also found that:
- Faculty and students value the LMS as an enhancement to their teaching and learning expe riences, but relatively few use the advanced features and even fewer use these systems to their fullest capacity;
- User satisfaction is highest for basic LMS features and lowest for features designed to foster collaboration and engagement;
- Faculty say they could be more effective instructors–and students say they could be better students–if they were more skilled at using the LMS; and
- Students and faculty want the LMS to have enhanced features and operational functions; be personalized; and use analytics to enhance learning outcomes.
Our Approach and Methodology
We collected quantitative data from Bb analytics but found that we needed to use more qualitative methods in order to answer some of our questions. So in addition to the Bb data analytics
- We conducted two focus groups with a total of twelve faculty members from different schools: six sciences/engineering faculty and six non-science faculty. Seven of our participants were female and five were male. Our sampling was purposive and we chose the participants because we knew that
- 1) they were interested in pedagogical innovations,
- 2) they integrated technology in their classrooms, and
- 3) they were using or had used Bb
- We also conducted a survey of faculty during a specific workshop (“Technology to support good practice in undergraduate education “) held eight times in the spring of 2016. In these sessions, the percentage of adjuncts was higher than usual. Participants responded to the open-ended question “When, why and how do you use technology?” using a web-based Personal Response System, “Nearpod”, through their mobile devices or computers. We collected the responses of 182 faculty. These responses were coded and divided into categories by the authors and patterns/trends were identified.
Results and Discussion
- Although Bb provides data analytics within a course, AUC does not have the institutional Bb data analytics tool (according to UACT, it is prohibitively expensive). We therefore had to resort to accessing and analyzing data collected by UACT locally for F2014-S2015. Organizing, analyzing, and interpreting this data proved to be a very tedious, slow and cumbersome manual process. It required the effort of three CLT staff members2 who devoted many days doing this (and that was just for two semesters!)
- In a day and age where Big Data in education is becoming an important tool for strategic decision making, not being able to efficiently access information about institutional use of the LMS is a disadvantage.
Conclusions
Blackboard
- Institutional data is not readily available and accessing, analyzing, visualizing and interpreting it is a slow and cumbersome manual process.
- Bb is underused by our faculty: advanced features are not used mostly because faculty do not know they exist or because they find them cumbersome
- Faculty who are more digitally savvy are satisfied to use the basic features of Bb but go to other tools for student collaboration and engagement.
- LMS analytics are not an indication of digital know-how of faculty.
- Although not totally satisfied with the LMS, AUC faculty are reluctant to change unless a new LMS offers significant advantages.
Use of Technology
Results of open-ended questions and Blackboard data indicate that a low percentage of AUC faculty use technology to promote student collaboration and engagement.
References
- Johnson, L., Adams Becker, S., Estrada, V., Freeman, A. (2014). NMC- Horizon Report: 2014 Higher Education Edition. Austin, Texas: The New Media Consortium.
- Eden Dahlstrom, D. Christopher Brooks, and Jacqueline Bichsel. The Current Ecosystem of Learning Management Systems in Higher Education: Student, Faculty, and IT Perspectives. Research report. Louisville, CO: ECAR, September 2014. Retrieved from http://www.educause.edu/ecar.
