Assessment of the Implementation of the Substitution Augmentation Modification Redefinition Model in Online Courses during the Prep Phase at University of Global Health

Purpose: Instructional technology integration at the institutional level can be fraught with a lack of preparedness (25.6%) and understanding (39.1%) (Kihoza et al., 2016; Lacruz, 2018). To mediate these issues a sound instructional technology integration approach is critical to the success of educational programs. The Substitution, Augmentation, Modification, Redefinition (SAMR) model (Lacruz, 2018) was designed to aid educator’s measuring the level of technology integration in their class and to offer suggestions for the lack thereof. For technology integration to be successful overtime, faculty and student satisfaction must remain high. The purpose of this study is to implement an instructional technology approach based on the SAMR model and assess faculty and student satisfaction with this model. Methodology: This study was a mixed methods cross-sectional study that evaluated Prep Phase courses for the level of SAMR model use. Findings: Results showed that courses with a high level of technology integration at the Modification and Redefinition levels also had high levels of student satisfaction. What is distinctive in this study is that the implementation was tracked from beginning to end, with student and faculty satisfaction being assessed. Unique Contribution to Theory, Practice and Policy: The study recommends that a looser version of the SAMR Model may be used so that faculty training focuses more on engaging technologies and less on a broad model. Faculty training time can be limited so finding ways to maximize that time would certainly be beneficial.


INTRODUCTION Overview of the SAMR Model
The SAMR model is a framework first introduced by Dr. Ruben Puentedura to categorize four levels of technology integration in the classroom (Terada, 2020).The SAMR model is designed to aid educators measuring the level at which they are using technology in classroom activities by using Bloom's taxonomy as a guide (Lacruz, 2018).The model encourages educators to create curriculum that allows students to learn the technological skills needed in the 21st century, and hence it can be evaluated at both the educators and student's levels (Lacruz, 2018).
At the Substitution level of the model, technology acts as a direct substitute, with no functional changes; for example, providing lecture notes online instead of printing and handing them out in class, in this case, no changes to the content are made.At the Augmentation level, the technology acts as a direct substitute but with functional improvement; for example, students and educators use google docs to provide feedback, comment and to share hyperlinks or multimedia.At the Modification level, technology allows for significant task redesign including the use of learning management systems (LMSs) like Canvas which allow students to create collaborative documents and multiple instructors to provide feedback.Lastly, at the Redefinition level, technology allows for the creation of new tasks that were previously inconceivable without it, and those include the use of platforms that allow students to create materials to share with the public (Jude et al., 2014;Lacruz, 2018).An example of this technology used at the Redefinition level could be allowing students to edit and add to webpages with a LMS.
These four levels of the model are further categorized as Enhancement (substitution and augmentation), and Transformation (modification and redefinition).The Enhancement category refers to the use of technology in the classroom with little to no changes to the tasks, whereas the Transformation, the adoption of technology modifies the learning activities (Jude et al., 2014).

Implementation and Evaluation of the SAMR Model
Usually, when switching from face-to-face to online classroom, educators focus more on the first two levels (Substitution and Augmentation) which involve replacing traditional materials with digital ones, including converting lessons and worksheets into PDF files, and uploading them online, or recording lectures on video and making them available for asynchronous learning (Terrada, 2020).This step is helpful for institutions that are adopting an online format for the first time.However, for institutions that have reached a proficiency level of using technology in the classroom, the remaining two levels are used in their approach (Terrada, 2020).Research has shown that student satisfaction with technology is highest when technology promotes interaction and engagement, something the modification and redefinition levels of the SAMR Model address (Rios et al., 2018).
A literature review conducted to review the SAMR model and its use has revealed that the SAMR Model is a tool to describe and categorize the level of technology use in classrooms (Hamilton et al., 2016), and it encourages educators to improve from lower level of Substitution to the higher level of Redefinition, which according to the model leads to a higher level of teaching and learning.A similar study done by (Kihoza et al., 2016) used the SAMR model to evaluate educators on which technological tools are used in their classrooms and other technologies they think students should be using in the classrooms.This study gave a picture of how technology is being integrated, and the knowledge and willingness of lecturers to learn and use technology in Tanzania.Lack of preparedness (25.6%) and understanding (39.1%) on how to integrate technology in classrooms were the main findings (Kihoza et al., 2016;Lacruz, 2018).

Educational Approach and Overview of the Preparatory Phase
As of 2020-2021, there were limited instructional technology models being used at the University of Global Health Equity (UGHE).UGHE is a new university located in rural Rwanda.The University has developed a six and half year unique program that provides its graduates with a bachelor's level medical degree.The UGHE's MBBS curriculum is made of four phases (Preparatory phase, Biosocial foundations of medicine phase, Clinical and health delivery phase, and Exploration and development phase).The Preparatory phase (Prep Phase) is the initial phase of the MBBS program that lasts six months, and it is an intense training period designed to help students transition into a rigorous medical curriculum.
The lack of preparedness (25.6%) and understanding (39.1%) regarding how to integrate technology in classrooms were the main findings (Kihoza et al., 2016;Lacruz, 2018) of previous studies.To mitigate these issues, this study will explore an instructional technology model (SAMR) implementation approach that examines faculty and student perceptions of the implementation approach and the SAMR model.
The precise issue that the research will address is faculty and student perceptions of SAMR model implementations within three Prep Phase courses.These perceptions will be addressed by the Technology Assessment Model (TAM).TAM looks at the perceived utility of a technology and its ease of use, two critical factors when examining the potential success of the introduction of a recent technology into an organization.
Studies (Kihoza et al., 2016;Lacruz, 2018) have determined that technological integration in classrooms can be difficult to manage.Many online learning implementations, including those at UGHE, lack proactive instructional technology.By examining faculty perceptions of the implementation of the SAMR model, the utility and usefulness of this instructional technology approach can be determined.In addition, student perceptions can be examined to determine the effectiveness of SAMR for improving student engagement.Though there has been previous research focused on the implementation of the SAMR model in courses, there appears to be a limited number done examining faculty and student perceptions of its implementation using Canvas learning management system (Blundell et al., 2022).This project seeks to examine the following: 1. Faculty perceptions the implementation and utility of the approach 2. Student perceptions of the SAMR implementation within Canvas courses This project is a pilot of the SAMR model at UGHE.It is hoped that this study will allow the e-learning team and faculty of UGHE to critically evaluate the impact of the model's implementation process and faculty perceptions on teaching and learning with it.The focus of the SAMR Model training will be on five faculty working on two Prep Phase courses: Gender and Critical Thinking.

METHODOLOGY Research Design
This study was a mixed methods cross-sectional study that evaluated Prep Phase courses for the level of SAMR model use.In addition to this, three identical surveys were distributed to faculty involved in the Prep Phase from December 2021 until May 2022 to determine their knowledge and perceptions towards the SAMR model and to measure change in knowledge and perceptions towards the model.A survey was also conducted amongst students undertaking the prep phase from December 2021 until May 2022 to determine their satisfaction with the SAMR model elements used in their courses.Finally, focus groups and in-depth interviews were conducted with both faculty and students to discuss their perceptions and experiences with the SAMR model.

Implementation of the SAMR Model at UGHE
The UGHE e-Learning department introduced the SAMR model to five faculty members and assisted them in creating SAMR model implementations at the Modification and Redefinition levels.The Modification and Redefinition levels of the SAMR model are the most complex to design and require some creativity on the part of instructors.The initial goal was to implement between five and six SAMR model implementations at the Modification and Redefinition.For example, traditionally students would individually write research papers.
However, at the modification level, collaborative tools in Canvas (such as Google Docs integration) can be used for group research paper writing.The benefits are that students collaborate in real-time, offering varied perspectives and learning how to co-author documents digitally.
The UGHE e-Learning department developed and implemented SAMR model training and assisted faculty in determining where SAMR can be implemented in courses.While faculty were responsible for attending training sessions, one-on-one sessions, and completing a Canvas SAMR short course.

Data Collection Methods and Tools
The SAMR Module Evaluation Tool was used to measure the overall SAMR implementation level of each Prep Phase course used in this study.The tool evaluated two aspects of each Canvas course: 1.The number of Canvas technologies implemented in each course.2. The SAMR level of that implementation.
Every potential use of Canvas technology was tracked.There are twenty-three technologies to use in Canvas.Examples of these technologies include the discussion board, chat, pages, and the use of Zoom videoconferencing as a part of courses.For the course evaluation piece, evaluators noted if a particular technology had been used or not and at which level the technology was used in SAMR.For example, as shown in Table 1, uploading PowerPoint slides into Canvas is determined to be part of the Substitution level as they are simply used as a substitute for handing out lecture notes.Allowing students to provide feedback is evaluated at the Modification level because the technologies used allow for significant modification of a task.

Student and Faculty Surveys and Focus Groups
Student satisfaction was surveyed after each course.And faculty were surveyed before, during and after the SAMR Model training and implementation.Two focus groups discussions (FGDs) will be conducted, one with faculty and one with students after the implementation of the SAMR model and once the prep phase has concluded.For students, this is standard evaluation practice at UGHE and the FGDs will explore feedback on online learning experience.The faculty FGD will explore key themes arising from the surveys and investigate faculty perceptions of the implementation of such a model across UGHE.

Courses
First, a review of the SAMR level of six Prep Phase courses was completed.This review showed that most technology implementations in the six Prep Phase courses fell within the Augmentation category (see Table 2).Half of the courses used at least some Redefinition.The history course used an Augmentation approach 81% of the time.The high use of Augmentation technology can be attributed to the extensive use of Zoom technologies in all the online courses and the classification of Zoom use at the Augmentation level because Zoom technology acts as a substitute for in class lectures with some use of recent technology.

Student Surveys
As shown in Tables 3 and 4, students in the two courses where the instructors had been trained stated that they were engaged with the material between 38 and 44 percent of the time, engaged with other students between 27 and 38 percent and with their instructors between 27 and 33 percent.In the two courses with the highest level of Modification and Redefinition, IT and Health and Communications, students stated they were engaged with the material between 46 and 66 percent of the time, with fellow students 25 percent, and with their instructors between 25 and 50 percent of the time (see Tables 5 and 6).

Focus Groups
When asked how they felt about online courses students responded that the courses went well and that it was a particularly pleasant experience.When asked if there was enough interaction between fellow students during the online, students responded that it depended on the course, some are very engaged and well-organized, while others are challenging.The interaction therefore depends on the course's structure and delivery method to the students.
When asked about some of the elements of Canvas that encouraged student interaction, most of them answered that the chat box was the most helpful, as opposed to other sections where they could post greetings or arguments.They thought the chat box was an excellent place for the discussion.Several of the students said that there needed to be more innovation and creativity in the way things were done, such as making comments anonymous so that it was impossible to tell who was making them, which would let them freely give feedback.Some students also said that the calendar section needed to be improved and introduced to the students so that they would understand what the canvas calendar part was and what it was for.
When asked about how their online course went some faculty stated that it went well that the SAMR model was used in a way that can engage students and that the model is good as a planning tool.Some of the complaints by faculty were that they needed more practice but that they have been using the SAMR model trying to revise the program in a way that allows student engagement.

Discussion
A review of the data showed the two courses with the most Modification and Redefinition, IT and Health and Communications, were courses where the instructors did not receive SAMR training.In addition to this, the original plan to implement between five and six Modification and Redefinition instances did not happen; only between three and four were added to the Gender and Critical Thinking courses.As one can see from the faculty comments, there were time constraints on the training; faculty perceived the utility of the SAMR Model but believed more time was required for its proper implementation.
IT and Health and Communications had the highest Modification and Redefinition levels, with IT and Health having 24% and Communications having 48% at the modification level.60% of the students in IT and Health also said they were engaged with the material.And 46% in Communications also said they were engaged with the material.In the History course, which had the least amount of Modification and Redefinition, only 27% of students said they felt very engaged.Students appeared to be more engaged when Modification and Redefinition were implemented at higher levels.
The Communications course was an interesting development because the instructor had received no SAMR training but had the most implementations at the Modification level.For example, the instructor used FlipGrid technology as an assessment tool.Research has shown that student satisfaction with technology is highest when technology promotes interaction and engagement, something the modification and redefinition levels of the SAMR Model address (Rios et al., 2018).And which were implemented in the Communications course.

Table 6 : Communications Not Engaged Somewhat Engaged Mostly Engaged
The SAMR Model gives good direction to the instructor. Adds more work on the instructor, given existing time constraints in meeting priorities. Continuous training on the model should be provided; individual coaching can continue along with follow-up and supervision to guide instructors who do not understand the model. It is time taking and given urgency of tasks at UGHE, more planning and design time is required. More supervision and support in the design of courses  Provision of relevant feedback to course designers