Designing Online Courses: An Iterative Approach to Planning and Developing Highly Effective Online Courses

 

Terrence R. Redding, Cynthia Blodgett-McDeavitt & Katherine Moser

 

This is a practical “how to” paper which describes an iterative methodology used to create highly effective online courses.  This iterative process has been used to plan and then develop pre-licensing courses for the insurance industry, and it has been applied to basic adult education GED preparation courses.  Until this methodology was developed (Redding and Rotzien, 2000) it was generally believed that there was no significant difference between the educational outcomes of online courses and in the classroom courses (Russell, 1999).

When educational content is moved from the classroom to the Internet, access is increased by making the educational content available to the students when they choose to study, and where ever they have access to the Internet.  However, easing educational content delivery also introduces problems associated with instructional design because, typically, instructional design is not optimized for learning online.

One criticism of Internet based courses is that they are poorly constructed, being little more than speaker notes and reference materials published on the Internet. One report that reviewed hundreds of these various writings published over the last decade, finds that the overall quality of the research is questionable and thereby renders many of the findings inconclusive (The Institute for Higher Education Policy, 1999). The report cautions that policymakers and education leaders have “a lot to learn” about how distance education can enhance learning.

 

Purpose

 

The purpose of this paper is to describe an iterative approach to planning and developing highly effective online courses. This approach to instructional design of online courses was developed by OnLine Training, Inc. (OLT).

A wide array of instructional methodologies, delivery modes, and strategies have been evaluated. Online asynchronous courses are very similar to self-paced courses.  Self-paced courses typically perform poorly when compared to other delivery methodologies.  Herbert Walberg and his colleagues reviewed over 3,000 studies and summarized the effect of these methods on student performance (Borg & Meredith, 1989). A subset of their findings, those which may be associated with the instructional design used at OLT, are presented in Table XX.1.

The first column reports the interventions/strategies that were tested. The second column is the effect size (ES). The ES statistic is a quantitative way to describe how the average student who received a strategy performed as compared to an average student who did not receive the strategy. Researchers consider effect sizes larger than .33 to have practical significance. A negative effect size means the students receiving the strategy did not perform as well as the average student who did not receive the strategy (Borg & Meredith, 1989). An effect size of zero would mean that there were no significant differences between the performance of the average student receiving a strategy and students who did not receive the strategy. The third column presents percentile equivalents to aid in interpreting the ES. For example, the percentile of 88 for reinforcement, means that the average student after receiving the intervention of reinforcement, did as well as on an achievement measure as a non-reinforced student who scored at the 88th percentile.

OLT uses effect size to compare the online students’ performance to the other non-online student groups’ performance.

 

 

 

Table XX.1. Instructional Strategy Effects on Student Learning Outcomes

 

 

Effect

 

 

Method

Size

Percentile

1.

Reinforcement

1.17

88

2.

Cues and feedback

.97

84

3.

Graded homework

.79

79

4.

Cooperative learning

.76

78

5.

Class morale

.60

73

6.

Personalized instruction

.57

72

7.

Home interventions

.50

69

8.

Adaptive instruction

.45

67

9.

Tutoring

.40

66

10.

Instructional time

.38

65

11.

Home environment

.37

64

12.

Higher-order questions

.34

63

13.

Individualized instruction

.32

63

14.

Individualized mathematics

.32

63

15.

Teacher expectations

.28

61

16.

Assigned homework

.28

61

17.

Computer-assisted instruction

.24

59

18.

Peer group

.24

59

19.

Sequenced lessons

.24

59

20.

Advanced organizers

.23

59

21.

Homogeneous groups

.10

54

22.

Class size

.09

54

23.

Programmed instruction

-.03

49

Source: Data from Herbert Walberg, “Improving the Productivity of America’s Schools,” Educational Leadership, 41, no. 8 (1984): 24. (Borg & Meredith, 1989)

 

 

The online course—self–directed  learning design

 

OnLine Training bases its instructional design for online delivery on notions associated with self-directed learning, adult learning and motivation theory, and Bloom’s Taxonomy (Bloom & Krathwohl, 1956). The instructional design focuses on knowledge, comprehension, adult motivation as described by Redding (1995), (Redding, Rugolo, & Eisenman, 1999), (Redding, Caudell & Lucius, 1999) and Spear and Mocker’s notions associated with organizing circumstance (1984). Recognizing self-directed learners may desire to control when and where they study, the online course is designed as an asynchronous guided learning experience delivered in a virtual classroom which includes automated e-mail to the instructor, and online conferencing between the instructor and students.

      Instructional design for self-directed study requires considerable forethought. In the self-directed study virtual environment, a teacher is not immediately available for student assistance, as in a face-to-face classroom. The curriculum must be not only a presentation of content, but also serve as a sort of virtual teacher with further explanation as well as encouragement and emotional support. Curriculum developers who fully appreciate the delicate dance between teaching content and supporting learning, must anticipate student questions with difficult concepts and provide additional explanation in language that can be understood by learners of different skill levels. In addition to the content, bytes of information must be presented at strategic points, presented in appropriate affective language, to motivate students who are both intrinsically and extrinsically motivated.

      As often as practical each of the delivery strategies listed in the Table XX.1, above are considered during the instructional design process.  Combining the various strategies overcomes the negative aspects of self-paced instruction and is easily tied to the process of removing learning barriers.  As mention above, OLT uses an iterative process to cycle through the development sequence, improving each step of the instructional designed based on the experience associated with subsequent steps.  Naturally, subsequent courses developed after the initial successful course benefit from the increased experience associated with the iterative process.

The first course was designed to prepare individuals who were seeking to become professional insurance agents in the area of life, health, and annuities. A comparative study of the course presented in the classroom (three sites) and online (through a community college) was conducted (Redding & Rotzien, 2000). The study site was the state of Florida, which tightly regulates insurance agent preparation and licensing. Individuals seeking to become life, health, and annuities agents must complete a mandatory 40 hour pre-licensing course. The course must be approved by the state, be taught by an approved instructor, be offered by an approved educational provider, and be taught from a state mandated study manual. Before an individual is permitted to take the state insurance agent examination, they must first complete the mandatory course, and pass an end of course examination.

 

The courses that were compared had the following attributes:

 

       Same author.

       Same instructor.

       Same course outline.

       Same printed study manual.

       Same end of course examination.

       End of course examination administered under the same test conditions.

 

The courses being compared had the following differences.

 

       Different instructional design. Classroom—lecture vs. SDL online.

       Different delivery mode. Classroom vs. Internet.

 

 

Over coming the negative features of online courses.

 

The online instructional design was crafted by OnLine Training Institute as specified above. The course was limited by design to place emphasis on knowledge acquisition and comprehension in cognizance with Bloom’s Taxonomy (Bloom and Krathwohl, 1956). Special emphasis was placed on reinforcement, clues, feedback, and reducing specific identifiable barriers to learning. Of interest was the programmed instruction aspect of the online course. Table XX.1 indicates the ES can be negative for programmed instruction. This negative aspect was offset by the other elements of the instructional design, ready access to the instructor through automated e-mail, scheduled online chat, FAX, and 1-800 phone access as required. The tone of the course was written in an active voice with the intended audience envisioned as an adult who, through current circumstances, must learn the material in order to gain access to a new career field.

 

An iterative process

 

Preparing online courses for other populations and subjects is a challenge.  OLT’s instructional design staff uses an iterative approach to intentionally redesign, and refine online content to ensure it is as effective as possible.

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 


Figure XX.1 Shows the iterative process in terms of Planning, Production, and Instructional design – as integrated functions of the same iterative process.

 

            An iterative process consist of a reoccurring sequence of steps with previously completed steps being intentionally reselected and repeated as subsequent steps inform the practitioners within the iterative process.  See Figure XX.1 above.

This iterative process was originally developed to optimize asynchronous courses designed to teach a finite body of knowledge associated with acquiring an insurance agent’s license. During the iterative process a sequence of steps associated with placing the course online and testing the title, and then testing the course through a series of gated pilot studies, student satisfaction and success is gradually improved.  This approach is now being used to revise courses in the basic adult education area, which are designed as guided learning experiences.  These courses were originally designed and taught by a single instructor.  The revised courses are being taught online asynchronously by a variety of instructors in varying educational settings. The student is guided through a structured body of knowledge, interacting with the content, being evaluated through short quizzes and exercises under the tutelage of the instructor.  Each module of the basic adult education courses can be viewed as similar to a section of the insurance course in terms of instructional design.

As a result of the successful redesign, these courses have been  revised so that they can be taught by a wider variety of instructors in various educational settings to students who are actively pursuing a GED.  Our iterative approach calls for placing the content in a particular educational setting and running the courses as pilot studies while the courses are actively being revised using the iterative model.

 

 

The iterative model – a sequence of steps

 

            First, a course is proposed.  The proposal first addresses technological capabilities, our approach to course development and instructional design, and project specifications.  Limitations, to include budgeting considerations and the allocation of manpower are considered.  OnLine Training provides a complete range of services in the area of training development, instructional design for online delivery, and course production to include text, HTML, audio, images, graphics, and video. OLT specializes in training and training development designed to be delivered over the Internet or an intranet. However, that should not be construed to imply there are unlimited resources available to be focused on the task.  Nor should the end user of the course be considered as typically having the latest computer and high end connectivity.  In fact the opposite position is taken during the initial stages of the process.  Our design motto states we will use the least amount of technology to deliver the highest level of content to the largest possible number potential students.  This motto focuses our efforts on the low-end user who typically lacks high speed connectivity, may be using an older computer, and legacy software.

OLT conducts training needs assessments during the pilot course phase of the process in order to be advised by subject matter experts to develop, content that effectively addresses learning objectives, materials to support training, and the course content presented in our internet or intranet based virtual classrooms. The instructors external to OLT, in the partner institutions, decide how much assistance they wish to  receive from OLT and how much feedback they wish to provide to our internal instructors and instructional design team.  Most of the data is gathered during weekly faculty meeting conferences conducted using a conference bridge.

The use of a virtual classroom permits the instructor to participate in the development of classes online from any point at which he or she can access the "net." The virtual classroom can host self-paced training; instructor led training, or varying combinations of the two.  These parameters help set the standards for the basic education courses as they progress through the iterative process. 

The focus of our instructional design is providing learning experiences for students which foster excellent learning outcomes.  The student gains access to the training by entering the virtual classroom from any point at which the student can access the "net."

 

Training in the OLT virtual classroom has the following advantages:

 

 

 

 

 

 

 

 

 

Developing an OnLine Course.

 

Developing an online course, from a production control perspective, involves planning, prototyping a title, developing content, authoring the title, testing the title, and preparing the title for distribution. The instructional design steps include establishing learning measures, identifying definitional content, foundational content, contextual content, and linking content.  The instructional design steps occur within the iterative framework of the production control process. The following considerations must be addressed in order to develop an online course:

 

 

The specifications for the project must be formally written out.  For the purpose of beginning a project a few basic assumptions based on the best information available can be made.   Our basic adult education courses are designed to be stand-alone (instructor assisted) and self-paced.  The total number of individuals who can take and complete the course online is limited to approximately 200 students per instructor.  The objective of the course is to prepare an individual to gain mastery over the objectives of the course and to prepare him or her to pass a particular knowledge component of the GED examination.

 

 

The course will be HTML based, delivered within the framework of a virtual classroom, with a design that will provide quick, responsive interaction at 28.8kb modem speeds.  Options may include student selection of online PowerPoint introduction and summary with audio, which may provide student orientation and closure at the end of each course.

Finally, each course will provide a self-assessment as to learning outcomes and a way to provide the student, instructor, and course management feedback, which will address student learning objectives and satisfaction with the course.

 

 

Content will be based on the publicly available GED descriptors.  The content will be restructured and designed for online delivery by OLT’s instructional design team.

 

 

The design goal includes the integration of instructional content and multimedia content, with optional downloadable components into a completely coherent online instructional package.  Initially a look, feel, and interactive set of templates with navigational buttons are designed for a course.  The same templates can be used with subsequent courses, saving development cost.

 

 

An initial pilot study for each course is conducted online to evaluate the course as “a proof of technical performance” for various hardware/computers configurations within Internet environments.  Feedback forms, which work with perl/cgi scripted bins, are used to facilitate communications between participating institutions, instructors, students, and the instructional design team.  This is an internal test and evaluation that is open to the participating institutions prior to the course being placed online.

 

 

The final step is to place the title on the website or other media (CD ROM) for distribution.

 

 

Project Specifications

 

Project specifications address the basic concept, practical considerations and technical factors.  The authors have developed a generic set of project specifications below.  Other courses/titles may dictate other specifications.  These are used to guide the initial instructional design and production efforts.

 

 

Each course developed should have a concept to identify the audience, purpose of the course, and the course content.  For the purpose of these courses the audience is assumed to be adults, ages 18 through gray.  The purpose of each course is to provide an individual with instruction which will lead to their ability to successfully pass the GED examination.  A secondary objective of the course is that once an individual has completed the course, they will have thoroughly learned the subject material and will have satisfied the course task objectives.

The instructional design team will work with publicly available content.

 

 

 

Practical considerations include budget, schedule, and resources.  Based on the formal acceptance of the concept, the design team should be admonished to adhere to the internal budget, production schedule, and allocate the resources to complete the project.  If proper management procedures are not established, over runs and misleading schedules will simply contribute to the frustration associated with placing a course online.  The instructional design team should make a commitment to high quality work, completed on time, and under budget.  Savings should be passed on to other projects which may be subject to over runs.  

 

 

Two primary technical considerations affect the online course design process.  They are equipment and distribution.  The equipment issue refers to the specifications of the equipment used by the “end user” in other words, the student.  A wide variation in resources exists on the Internet, worldwide.  Here in the United States we may wrongly assume that access to the Internet is equal.  It is not.  The reality is, access to the Internet is non-existent in most places.  Where access  is available, bandwidth and cost vary greatly.

The proposed online training courses may be designed to be delivered to multimedia capable computers with a minimum of 28.8kb bandwidth access to the Internet via a dialup connection.  Thus distribution of these courses will be handled via the Internet.  But it should be recognized that not all who have access to the Internet will be able to accommodate these specification.

 

Discussion

 

Distance education via the Internet is becoming increasingly common. A wide array of instructional formats are used online. Many approaches to course design are criticized as ineffective.  Often the most positive comment about Internet based courses is that there is no significant difference between them and classroom based courses.  This instructional design approach seeks to capitalize on the outcomes associated with a highly effective instructional design approach specifically developed by OnLine Training Institute (OLT) based on notions associated with self-directed learning, adult learning theory, and adult motivation. This instructional design model is highly adaptive and, by design, insures a course will continue to improve until it is highly effective.

 

Student Comments

Students queried concerning why they choose to take courses online uniformly reported it satisfied their desire to control when and where they accessed the course. Some indicated it addressed their need to be home with children (single parents), others indicated it addressed their concern about physical access due to disability, while still others expressed concern about physical safety citing that the online course meant that they did not have to venture out at night nor walk late at night across an empty parking lot to reach their car.

 

Instructor Comments

Most instructors who use the online GED courses are primarily concerned with providing greater access to students, and gaining and interactive advantage through the automated grading of student quizzes in the virtual classroom.  Instructors find the virtual classroom ideally suited to students with limited access to classroom based courses.  The interactive component of the virtual classroom based course permits the student to work at their own pace, while receiving feedback from the course in terms of how well they are doing.  The instructor is able to asynchronously track student progress and success through the automated interactive learning modules. 

      The instructors also believe online students are more confident in their ability to learn the material required on their own. Given the nature of the online course vs. the in-the-classroom experience the instructor’s came to believe online courses are more precise, time efficient, and effective than the traditional in-the-classroom experience.

      The instructors also felt that online instruction is not for everyone. Many, who lack the self-discipline, or confidence simply would not be able to learn online and should be in a classroom.

      The greatest advantage of the online course is that the student controls the pace of the course.  They can learn at their own pace.  Slower students can take all the time they need online to master a particular topic, while faster students can progress at their own pace.

 

OnLine Training Institute Comments

Through online course management tools, online instructional designers and program management personnel are able to measure the number of times and amount of time a students spends online using the course–or dwells on a single question or page. Because of the interactive nature of the online course, the instructional group can know what was  typed, clicked, or selected each time a student makes a choice and moves through the course. Online we can know more about the student choices and cognitive thought processes than the instructor in the classroom can know. This information is reviewed in an effort to improve the instructional design, as part of the ongoing iterative process, during the initial development phase.  Once a course is effective, student performance is reviewed keystroke by keystroke, only when a student fails.

      Two cases have arisen recently in online courses with community college students.  In the first case the student failed a state licensing exam after scoring a 98% on the post-test of an online pre-licensing course.  The community college wanted to know how this was possible.  A review of the course log revealed the student had accessed 99 pages of a 550 page course.  A detailed review revealed that the student had progressed no farther than chapter four of the 29 chapter course.  The virtual classroom log allows OLT to reach an educated conclusion concerning why the student failed the state exam.  But we can’t explain how the student achieved a 98% on a proctored post-test (end of course exam). 

      In the second case the student scored a 54% on a course post-test, complaining the course was ineffective. The student said he had been through the complete course. The mean GPA for that particular post-test is 92.37%.  A detailed review of the student’s actions in the course confirmed he had indeed been to every page in the course.  However, we were also able to see how much time he spent on each page.  The average was 15 seconds.  The average should have been above 120 seconds for even the brightest student.  In effect, this student had simply clicked through every page of content in the course, similar to flipping the pages in a book, and thus had not actually interacted with the course content.

The instructor in the classroom can be sure of little more than the student is present–unless the instructor actively interacts with each student. With the exception of quizzes, when the instructor verbally poses a question–typically only one student can respond. However, when the instructional design team poses a question online–we know that each student will respond before they move forward through the course. Properly designed, using a student centered approach, online training can be an effective way of ensuring cognitive learning occurs.  When a learning outcome is disputed, online we can review every action the student took, and how long it took them to take the action, as we review a student’s performance in a course.

      The preliminary results of the redesigned GED courses are coming in even as this paper is being prepared for press.  An early indication of success was reported by the GED faculty.  One student, who is a professional soccer player currently on a team in England, had been struggling to complete high school.  This student is now enrolled in the online GED program and just completed the Writing Skills course.  The student’s pre-test was a 60 percent.  The post-test score is 98 percent.  It is hoped other students will do as well as the iterative design model is applied to all five of the GED courses.

 

Discussion—Distributive Level of Effort Model

Our first online course took literally years to design, revise and release. But, it produces consistent higher cognitive learning outcomes, well above the 12 year old in-the-classroom model, and we anticipate being able to achieve similar results with the GED basic adult education series as they mature.

Mature in-the-classroom instruction (courses that have been fully developed and offered again and again over an extended period of time) should be near perfection, after several years of being taught by a true sage on the stage.  Online instructors, who also teach classroom-based courses, report that their in-the-classroom course benefited greatly from the detailed effort they made to construct the online course. Each word, phrase, and fact was carefully considered as it was selected for inclusion in the online course. The sequenced elements were taught, laying a firm foundation of knowledge, and then comprehension was considered and then tested and then revised as the design for the course was finalized.

A similar effort for classroom instructional design might well have a similar impact on the over all effectiveness of instruction offered in the classroom. The difference in cognitive outcomes may well be explained as much by the level of effort and collective attention of the instructional design team as it is by the self-selected nature of the students.

From a distributed level of effort model, shown below, it can be argued that a sage on the stage, working with 10 students, must distribute his effort across 10 students. Thus the instructor to student ratio is 1:10. The student can be thought of as receiving 10 percent of the instructor’s time.

Application of curriculum design strategies is modeled in the development of Online Training courses. A five person design team, some of whom are teachers, focuses curriculum design effort on the single student who sits at the computer working through the online course. The team attempts to considers the student’s every potential choice and need. Experienced teachers know the difference between teaching content and supporting learning, and this team is in a good position to create the necessary blend within the curriculum to best meet the varied needs of the learner. (The majority of the design team/instructors are women, as are most online students, so the team is also sensitive to unique gender-related concerns of adult learning.)

Thus the instructor-to-student ratio is 5:1. In the online model the student gets a comparable 500% of an instructor’s time. The instructor’s time and responses are also structured, formal, and never ad-hoc. From an instructional design point of view one could say that the online instructional design has an advantage over the classroom instruction with an aspect ratio of 50:1. This is part of the reason the online course is so successful.

 

 

 

 

AppleMark
 

 

 

 

 

 


                                         

 

 

 

 

 

 

 

Figure XX.2. The distributive level of effort model shows the relationship between the traditional instructional design effort of one instructor to many students, and the online model of a design team to one student.

 

 

 

 

 

Conclusion

 

The online course is an effective way of conducting educational activities when properly designed. The implication of the original research is that the instructional design employed by OLT is effective for designing Internet based courses delivered to adult students seeking to acquire the knowledge and comprehension associated with licensure. The higher GPA of the online insurance students indicates online instruction was more effective than classroom instruction. It is hoped that over time similar results will be achieved with the GED basic adult education students.  However, given the self-selected nature of the online students, more research is needed before definitive conclusions can be made as to whether it is better instructional design, or just better students that account for the success of the online courses. The original study of the online insurance students had several limitations such as: the small sample size for the online group, and an inability to control for reading level, work/home environment, ethnicity, age, or educational background of any of the participants.  It is hoped that working with a basic adult education population we will be able to over come some of these limitations and allow for a more robust study of online learning outcomes.

The results of the previous study did provide support for the conclusion that online instruction for individuals entering the insurance field can be effective, and can be more effective then traditional classroom delivered instruction for those students confident in their ability to learn online.  It remains to be seen whether success with that group can be transferred to basic adult education students as a result of applying the iterative instructional design model.

 

 

REFERENCES

 

Bloom, B.S., & Krathwohl, D.R. (1956). Taxonomy of educational objectives: The classification of educational goals, by a committee of college and university examiners. Handbook I: Cognitive Domain. New York, Longmans, Green, 1956.

Borg, W.R., & Gall, M.D. (1989). Educational research: An introduction. Longman, New York and London.

Redding, T.R. (1995). Reordering Maslow’s needs hierarchy based on self-directed learning considerations. In H.B. Long & Associates, New dimensions in self-directed learning (pp.165-180). Public Managers Center (Educational Leadership and Policy Studies Department): College of Education, University of Oklahoma.

Redding, T.R., Caudell, & Lucius, S. (1999). Comparing self-directed learning to an empowerment organizational motivation model: Self-directed learning as a way to Zapp! Employees. In H.B. Long & Associates, Contemporary ideas and practices in self-directed learning (pp.225-238). Public Managers Center (Educational Leadership and Policy Studies Department): College of Education, University of Oklahoma.

Redding, T.R., Eisenman, G., & Rugolo, J.J. (1999). Training in technology for late adopters: Learning in retirement, Computers for Seniors. In H.B. Long & Associates, Contemporary ideas and practices in self-directed learning (pp.239-251). Public Managers Center (Educational Leadership and Policy Studies Department): College of Education, University of Oklahoma.

Redding, T.R. & Rotzien, J. (2000). A comparative analysis of pre–licensing insurance online         learning with traditional classroom learning.  In H.B. long & Associates, Self-Directed Learning and the Information Age.  (ch. 11).  Motorola University Press, Boyton Beach, Florida.

Russell, T.L. (1999), The no significant difference phenomenon. Instructional Telecommunications, North Carolina State University.

Spear, G.E. (1988). Beyond the organizing circumstances: A search for methodology for the study of self-directed learning. In H.B. Long & Associates (Ed.), Self-directed learning: Application and theory (pp.199-222). Athens, GA: University of Georgia.

The Institute for Higher Education Policy (1999). What’s the difference? A review of contemporary research on the effectiveness of distance learning in higher education. The Institute for Higher Education Policy, Washington, DC.