by David J. Marcey, Fletcher Jones Professor of Developmental Biology, Biology Department, California Lutheran University
marcey@clunet.edu
Keywords: flipped classes; flipped classroom; active learning; online lectures; cinematic lectures; blended learning; blended teaching; flipped pedagogy; hybrid learning; hybrid courses
ABSTRACT
Two sections of an undergraduate introductory biology lecture course were run in parallel as a pedagogical experiment. One section (32 students) was taught in a long-established, traditional manner, with lectures delivered during class, readings assigned in a textbook, and access to lecture graphics/slides provided via the online syllabus. The other, “flipped” section (16 students) lacked both required reading assignments and in-class lectures. Instead, students were assigned online cinematic lectures (cinelectures) for viewing outside of class. These cinelectures, delivered via YouTube, incorporate multimedia elements. In class, students were broken into small groups and engaged in active learning assignments. Accounting for all sources of content, the subject material covered was the same for both sections and assessments of learning were identical quizzes and examinations. Statistically significant differences in learning were observed during the first third of the semester, with the flipped-class students performing better on all tests and quizzes. These differences disappeared during the second two thirds of the semester, coincident with a large increase in the number of views of cinelectures recorded on the course YouTube channel. Survey of the traditional class revealed that approximately 3/4 of the students had learned of the cinelectures at this time and had added viewing of these to their study, providing an internal, if initially unintended, control sample to the experiment. These results, along with other, subsequent applications of the flipped model I term CLICing, provide evidence that supports the conversion of traditional biology lecture classes to an inverted format.
INTRODUCTION
This report describes an experiment in the implementation of a particular model for flipping an introductory biology lecture course at a small, comprehensive university. Because flipping is becoming increasingly popular, and because assessment of the effects of this type of reform at the post-secondary level is sorely needed, this study bears import for a wide range of biology educators.
The Need for Active Learning in Undergraduate STEM Education
“A lecture is a process by which the notes of the professor become the notes of the student without passing through the minds of either.” –R.K. Rathbun (cited in Alberts, et al. 2007)
Limitations of traditional lectures as a primary pedagogical tool in teaching science have been widely recognized over the years, satirically in some cases. Unfortunately, this class format prevails as the most common approach to science, technology, engineering, and mathematics (STEM) education at the post-secondary level. STEM courses should afford students the opportunity to practice critical thinking and high-order cognition, help them integrate knowledge into conceptual frameworks, guide them in linking prior learning to new knowledge, and aid them in developing problem solving skills that allow the application of concepts to situations that are not explicitly memorized. However, these goals are rarely realized for the majority of students in the traditional lecture model (e.g., National Research Council (NRC) 2000; McCray et al. 2003; Honan 2002; Knight 2004; Freeman et al. 2011). For most students, lecturing promotes memorization of facts rather than fostering deep understanding (e.g., Wright et al. 1998; Loverude et al. 2002), and even high academic achievers sometimes gain little understanding of basic science concepts through traditional didactic lectures (Sundberg 2002).
To remedy this situation, numerous clarion calls for reform of standard lecture delivery by incorporating active learning[1](AL) in the classroom have been forwarded by august science education advisory bodies and science education researchers (e.g., Froyd 2007 (Project Kaleidoscope);Davis 1997 (National Research Council – NRC); Nielsen 2011 (NRC); NRC 2000, 2003; National Science Foundation 1996;The Boyer Commission 1998;Bonwell and Eisen 1991 (The Educational Resources Information Center); Cerbin 2012 (The Carnegie Foundation for the Advancement of Teaching);Vision and Change in Undergraduate Biology Education 2011;Handelsman et al. 2004).
Indeed, there is a broad empirical fundament that supports the use of AL in science classrooms (e.g., reviewed by Handelsman et al. 2007; Prince 2004; Knight,2004; Allen and Tanner 2005). Specific examples of the benefits of AL in undergraduate biology education include: 1) the observation that substituting daily and weekly practice in problem-solving, data analysis, and other higher-order cognitive skills for lecture-intensive course design improved the performance of all students and reduced the achievement gap between disadvantaged and non-disadvantaged students (Haak et al. 2011); 2) the finding that an intense, inquiry-based, learner-centered experience was associated with long-term improvements in academic performance (Derting and Ebert-May 2010); 3) the measurement of significantly higher learning gains and better conceptual understanding when student participation and cooperative problem solving during class time was substituted for traditional lecturing (Knight and Wood 2005); 4) evidence that teaching biology in an AL environment is more effective than traditional instruction in promoting academic achievement, increasing conceptual understanding, developing higher level thinking skills, and enhancing students’ interest in biology (Burrowes, 2003); 5) the observation of pronounced differences between students taught biology traditionally and those taught with a series of active, inquiry-based learning modules (Udovic et al. 2000), termed “Workshop Biology.”
The Flipped (CLICed) Class as “Disruptive Innovation”
“I think [the traditional lecture] is going to be kind of blown away in favor of a model where every student is working at their own pace and the teacher now has a much higher-value role…”
– Khan, S. (2012)
“The key to the flipped class is actually not the videos, it is the freedom those videos give the teacher to have engaging class activities and interaction with their students.”
– Bergmann, J. (2011)
A method of introducing AL pedagogy in the undergraduate lecture hall that overcomes some of the barriers to its implementation is the approach known as class inversion or “flipping,” i.e. assigning material as homework that is usually covered in lecture and engaging students in AL during class.
This approach to classroom inversion is not new and there are many different models of class flipping (e.g. Baker 2000; Lage and Platt 2000). One model that has recently gained momentum has been popularized by Salman Khan (www.khanacademy.org) (Bergmann 2011; Sams 2011). This approach, which I term CLIC Cinematic Lectures and Inverted Classes) in order to distinguish it from other flipping formats, relies on the delivery of course content through engaging, cinematic lectures (see Methods) outside of class, which enables the transformation of classrooms into arenas of inquiry and active learning (Figure 1).
Figure 1. The CLIC model of course flipping.
CLIC reform acknowledges the primacy of AL in reforming the traditional lecture class, but also recognizes the important role that faculty provide in creatively contextualizing course concepts within a large base of knowledge. This idea has captured the attention of the educational and popular press (e.g., Berrett 2012; Ojalvo and Doyne 2011) has been sweeping K-12 and educational communities, and is gaining popularity in post-secondary education.
In terms of online learning, CLIC can be viewed as a subset of those formats termed blended or hybrid, involving both online and face-to-face learning (e.g., Garrison and Kanuka 2004; Reasons et al. 2005; Alberts et al. 2007). There is substantial research in cognitive information processing (e.g., reviewed in Walker et al. 2011)and in evaluation of blended learning that supports the use of online multimedia in learning activities (e.g., McFarlin 2008; Lentz and Cifuentes 2009; Kay and Kletskin 2012). Significant gains in learning outcomes for introductory biology students using multimedia vodcasts compared to those viewing class lecture-capture videos have been reported (Walker et al. 2011), and online prelectures followed by analytical problem solving in the classroom increased student understanding in introductory physics classes (Sadaghiani 2012).
The theoretical advantages of CLIC for post-secondary education are pronounced. CLIC enables professors to deliver course content asynchronously to students who can engage the online cinematic lectures or cinelectures on their own terms, viewing them on computers or mobile devices at their own convenience and pace and, importantly, reviewing portions that they wish to understand more thoroughly at will. This, in turn, makes possible the transformation of lecture classes into venues where AL can be implemented. These two critical elements together can provide a more effective learning environment than the traditional lecture model. This game changing approach is a canonical example of “disruptive innovation” (e.g., Christensen et al. 2010) that is being actively promoted by online teaching networks (e.g., www.flippedlearning.com; www.vodcasting.ning.com; www.flipteaching.com). CLIC has garnered some criticism. Most objections, however, concentrate on the inability of online video to engage students in inquiry-based activities (e.g., Noschese 2012), thereby ignoring the raison d’être of the CLIC approach.
Recognizing both the potential value of CLIC and the need for assessment of this type of reform at the undergraduate level, the author ran a pilot experiment in CLIC in the Fall 2011 semester and, based on these early results, has continued to improve implementation of the model in subsequent years. CLIC is still in very early phases of adoption at the university level, and other than the study of the effects of this model that we report here, the author is not aware of an assessment of this precise form of class flipping in an undergraduate, introductory biology class.
METHODS and IMPLEMENTATION
In 2011, two sections of an introductory biology lecture course (Introduction to Metabolism, Genes and Development[2]), taught by the author, were run in parallel. One section (32 students) was taught in a long-established, traditional manner (TR), with lectures delivered during class, readings assigned in a textbook, and access to lecture graphics/slides provided via the online syllabus. The other, CLIC (CL) section (16 students) lacked both required reading assignments and in-class lectures. Instead, students were assigned online cinematic lectures (cinelectures) for viewing outside of class, but were encouraged to purchase an optional textbook for reference. The cinelectures, created with Camtasia software (Techsmith), were delivered via links to YouTube from the course online syllabus. Despite a lack of individual student monitoring capability, YouTube was chosen over a traditional learning management system because of student familiarity with and access to this venue and because there is a need for engaging introductory biology content that can reach a diverse audience (at the time of this writing there have been over 180,000 views of Biology 122 cinelectures and many students from U.S. and international universities have expressed gratitude for these resources). The cinelectures incorporate multiple presentation media (e.g., see Figure 2). In developing cinelectures, critical design elements derived from empirical studies of online, multimedia educational projects were taken into account (e.g., Clark and Mayer 2011). The use of screencasting software permits the incorporation of genuine cinematic components into lectures. Examples include author generated molecular movies, avatars that interact with the lecturer, video clips that illustrate key points, and short video clips of other lecturers.
Figure 2. Screen captures of three cinelecture frames showing a molecular model of ATP Synthase (left), screen work (middle), and a movie of lamellipodial extensions (right).
In class, students in the CL section were broken into small groups and conducted AL activities that varied widely (e.g. building physical molecular models, constructing concept maps, designing plays to illustrate cellular mechanisms, formative assessments using clickers). Often groups of students or even individuals were responsible for presenting material to the class as a whole. AL modules were intended to provide opportunities for students to practice critical thinking about course content and to investigate and develop their understanding of key concepts. AL included the modalities of process-oriented, guided inquiry learning (POGIL– see http://www.pogil.org/post-secondary), peer-led team learning (PLTL – see http://www.pltl.org/) and problem-based learning (PBL), together referred to as PXnL(e.g., Eberlein, et al., 2008).
In subsequent years, the CLIC model was employed exclusively for this same course, covering the same topics, and comparison to earlier learning outcomes were conducted. Analysis of learning outcomes in all cases was based on identical or lightly modified multiple choice tests and quizzes that included a range of questions spanning Bloom’s taxonomy scale. Comparisons between course sections utilized standard Student t-tests (e.g., see http://en.wikipedia.org/wiki/Welch’s_t_test) for learning outcomes, correlation analysis of science GPA and learning outcomes, and Likert scale course evaluations for student perceptions of their course experiences. One-way ANOVA comparisons and post-hoc Tukey HSD tests (e.g., see http://www-users.cs.umn.edu/~ludford/Stat_Guide/One_way_ANOVA.htm) were used to compare learning outcomes over several semesters. Institutional Review Board approval for the use of student data was obtained (IRB number 2011028).
PROJECT RESULTS
Parallel sections of Biology 122 (2011) – Traditional (TR) and CLICed (CL)
There was no significant difference between the TR and CL sections in either overall or science GPAs (t-test, p>0.4, p>0.6, respectively), thus eliminating a potential confounding source of inter-section, academic aptitude variability. Accounting for all sources of content, the subject material covered was the same for both the TR and CL sections and assessments of learning were identical quizzes and examinations. Statistically significant differences in learning (quiz and test scores) were observed during the approximate first third of the semester (unpaired student t-tests, t=2.64, df=46, p<0.01), with the CLIC-class students performing better on all tests and quizzes in the period defined by Quiz 1–Exam 1 (Figure 3).
Figure 3. Mean % of possible points obtained in the Traditional (shaded) and CLICed (unshaded) sections of Biology 122 plotted for three periods defined by Quiz 1 – Exam 1 (q1-e1), Quiz 5 – Exam 2 (q5-e2), and Quiz 7 – Exam 3 (q7-e3), and for the final exam and course totals. A significant difference between the sections was observed for the q1-e1 period. This difference disappeared for the remaining two periods, reappeared on the final exam, and, although not statistically significant, a pronounced difference was observed for the course totals. Error bars = 95% confidence intervals.
These differences disappeared in the remaining two periods of the course defined by Quiz 5-Exam 2 and Quiz 7-Exam 3 (Figure 3), and reappeared on the final exam. As expected based on previous data, a pronounced but non-significant difference in total course scores was observed (Figure 3) – see Discussion.
To investigate a possible reason for the lack of significant differences in learning outcomes in the last two periods of the semester, the number of course-generated cinelecture views was examined. These results are displayed in Figure 4. The loss of significant inter-section learning outcome differences in periods 2 and 3 (Figure 3) was observed to be coincident with a large increase in the number of university-originated views of cinelectures. An anonymous, late mid-semester survey revealed that many of the students in the traditional class were using the cinelectures at this time, having added viewing of these to their study. This defect in the experiment provides an internal, if initially unintended, control (i.e., there was no apparent difference in comprehension ability between the sections). Although the “bleed through” of part of the experimental treatment (cinelecture viewing) to the control group (Traditional section) somewhat compromises the interpretation of the data so far presented, subsequent use of the CLIC approach in following years has yielded results that suggest a clearly positive effect of this pedagogy (see results below and Discussion).
Figure 4. Numbers of course-originated cinelecture views for the three periods of the course defined as in Figure 3. The stars (H) represent short periods in which students in the Traditional section were asked to view cinelectures in lieu of standard, in-class lectures due to professional travel of the professor. It is clear that this resulted in a prolonged use of the cinelectures by students in the Traditional section as revealed by an anonymous survey administered shortly after Exam 2 (s).
The course that is the focus of this pedagogical experiment is notoriously difficult for students, especially the classical genetics section that spans periods two and three as defined in Figure 3. Many of our students give voice to frustration that, although they do well in other science classes, they seem to “underperform” Biology 122. In order to ascertain if CLICing might affect student performance in ways not immediately discernable by direct comparison of test scores, correlations between student science GPA and course learning outcomes were analyzed for the three course periods defined in Figure 3 and for the final exam and course totals. As can be seen in Figure 5, except for the final exam, the learning outcomes for students in the CLICed section are more closely correlated with their overall science GPA than those for the Traditional section, even for course periods in which learning outcomes did not differ significantly (Figure 3).
Figure 5. For consenting students in the Traditional (¾) and CLICed (- – -) sections of Biology 122, science GPA (x axis) is plotted vs the percent of possible points obtained (y axis) for the three course periods defined by Quiz 1 – Exam 1 (q1-e1), Quiz 5 – Exam 2 (q5-e2), and Quiz 7 – Exam 3 (q7-e3), and for the final exam and course totals. Correlation coefficients are presented in the top left of each panel.
In the interest of gauging student perception of their learning experiences in both the Traditional and CLICed sections of Biology 122, CLU-standard anonymous university course evaluations were administered prior to the course final examination. Perceptions of some key learning indicators in the two sections differed dramatically (Figure 6). Students in the CLICed section rated their course higher in the following categories: use of effective teaching methods (t=2.76, df = 32, p<0.01), usefulness of homework assignments (t=3.01, df=24, p<0.01), appropriateness of in-class activities for achieving learning objectives (t=3.27, df=29, p <0.01), explanation of course material (t=2.10, df=32, p< 0.05), and course fostering of active participation (t=5.15, df=32, p <0.001).
Figure 6. Student perceptions of learning experiences in the Traditional (shaded) and CLICed (unshaded) sections of Biology 122 in 2011. Error bars = 95% confidence intervals.
Results with CLICed (CL) Biology 122 sections (2012 and 2013)
Since 2011, Biology 122 has been taught two full semesters and is currently being taught at the time of this writing (mid-semester). Although the results of the 2011 experiment were confounded by experimental “bleed-through” into the control (Traditional) section (see above), the results of exclusively employing the CLIC approach in subsequent semesters can be interpreted as having positive effects on student learning outcomes in this introductory course. Figure 7 compares the mean percentages of total course points for both 2011 sections and for CLICed sections taught in 2012 and 2013. A marked trend in higher scores is observed. A one way ANOVA comparison (p<.01) and post-hoc Tukey HSD test shows a significant difference between the 2011 Traditional and 2013 CLICed cohorts of students (p<.01).
Figure 7. Mean percent of total course points shown for the two Biology 122 2011 sections (TR and CL) and CLICed sections taught in 2012 and 2013. Error bars are 95% confidence limits.
DISCUSSION, REFLECTIONS, and FUTURE PLANS
“Q: How do you get to Carnegie Hall? A: Practice, Practice, Practice” –Anonymous.
There must be compelling and lofty motivations for undergraduate STEM faculty to institute significant pedagogical reforms in traditionally lecture-formatted courses, for such “disruptive innovation” (e.g., Christensen et al. 2010) is both challenging and time consuming. For example, in the present case, a 15 minute cinematic lecture might typically consume several hours of production and post-production (editing) work, and the development of quality active learning materials is not trivial. However, despite the formidible time investments involved, the extensive empirical literature that documents both the ineffectiveness of the traditional lecture and the positive effects on student learning that active learning engenders provides these motivations for reform-minded faculty (see Introduction). Active learning in the lecture hall makes tangible the “Carnegie Hall Hypothesis,” using valuable, in-class “seat time” to promote high-order cognition and metacognition through the practice of critical thinking. This practice promises to increase student learning, enhance retention of students from underrepresented groups, and help dampen the attrition of STEM majors, a disturbing national trend that is projected to result in a 1,000,000 shortfall in STEM-educated graduates over the next decade (PCAST 2012).
Despite its documented value, there remain formidable barriers to widespread incorporation of active learning in the classroom, and traditional lectures remain the norm for most introductory science classes (e.g., Stokstad 2001; Wood and Gentile 2003; Allen and Tanner 2005). These barriers include academic cultures that sometimes undervalue teaching innovations, professorial habits revolving around lecturer-centered education, perceptions of professional identity that conflict with pedagogical reform, the necessity of delivering copious amounts of content in a single quarter or semester, the perceived efficiency of lecturing to students in notoriously large introductory classes, lack of time and institutional incentives, and even student skepticism regarding the value of and participation in active learning (e.g., Hanford 2012; Brownell and Tanner 2012). One other major barrier is the understandable desire of faculty to provide individualized frameworks for course topics that go beyond standard textbook explanations, frameworks that have traditionally been delivered exclusively by in-class lectures.
The model of classroom flipping that is the subject of this report can be used to overcome this last, but important, barrier. The use of cinematic lectures (cinelectures) allows professors to construct their own, individualized conceptual framework for course topics that can be assigned as out-of-class work (even replacing textbook readings) and opens the door for the extensive implementation of active learning modules in the lecture hall.
Several advantages of the cinelecture component of CLIC are noted:
- The four P’s of student accessibility 3 are met: PLACE (students have the flexibility to learn in a location of their own choosing); PACE (students can learn at the speed suited to them); PEACE (students can determine the time in which they learn); PROCESS (students may skip topics they are confident with and concentrate on unfamiliar topics, reviewing them at will);
- Video clips of external lecturers may be captured and inserted into a given cinelecture (e.g., www.genome.gov/Glossary). This allows the use of public domain lecture clips of women scientists and of those from under-represented demographic groups. This “virtual diversity” in role model lecturers may serve as an important contribution in reaching students from underrepresented groups;
- Videos, animations, and simulations that are difficult for students to absorb when receiving them as a “one-off” experience in a traditional lecture are much more comprehensible when available online;
- Students are more likely to engage in the requisite 2 hours of homework for every 1 hour of class time as suggested by the Carnegie Foundation and the Department of Education when their out-of-class work can be monitored and involves viewing engaging videos;
- Unlike textbook reading followed by traditional lecture, the importance of cinelectures in the enhanced learning environment of a CLICed classroom incentivizes students to view them prior to coming to class, due largely to peer pressure from small group members with whom they interact in most active learning modules;
- Many introductory STEM courses are delivered by adjunct or part time professors who lack time and resources to undertake their own cinematic productions, but the creation of a wide variety of cinelectures by teams of faculty, coupled with associated active learning suggestions, could provide adjuncts the means and resources to introduce active learning in their classrooms.
The results of the 2011 pedagogical experiment in which sections of a Traditional and CLICed introductory biology course were run in parallel were confounded by experimental bleed-through, in the form of students in the Traditional class engaging in cinelecture viewing during the last two thirds of the semester. However, significant differences between sections were observed in the first period of the semester, when cinelecture views primarily derived from CLICed section students. Although this latter result is positive, it is nevertheless disturbing because the learning outcome differences obtained could be interpreted as arising from cinelectures alone, irrespective of the active learning modules that were implemented in the CLICed section, given the lack of significant differences in the latter two-thirds of the semester. Nevertheless, the stronger correlation between science GPA and learning outcomes observed for students in the CLICed format throughout most of the semester suggests some positive impact of active learning.
I attribute the significant increased learning outcomes in Biology 122 sections taught since 2011 to be the result of augmentation of the active learning modules that students now engage in as result of my continually increasing experience in developing high order cognitive questions, activities that promote metacognition, and group activities that are more engaging for students. For one example, employing techniques of scientific teaching (e.g. Handelsman 2007), students use clickers to answer a high order cognitive problem and look at the results of the e-poll together. Then they talk in small groups and vote again, which often generates peer-to-peer conversations that alert both students and the author to prominent misconceptions that can be addressed on the spot. A second example employs student directed theatre, in which students act out a particular process, say the regulation of the lactose operon (gene regulation), and then adapt their “plays” to different molecular scenarios. This requires a thorough understanding of the underlying biology and motivates students to apply existing knowledge to new situations. Increasingly, as a result of improved active learning opportunities, I observe students thinking about what they do and do not understand and expressing formulated questions based on this increased metacognition. This is a phenomenon I have rarely, if ever, observed in traditional, introductory, lecture-based classes.
Based on results of course evaluations presented above, and subsequent course evaluations, I can confidently state that most students take well to CLICed pedagogy. Indeed, it is clear from the heavy use of cinelectures by students in the 2011 traditional course section that asynchronous, video-based lecture material is preferred over textbook reading by many students. In past years, although some of my most academically prepared students were comfortable synthesizing information from lectures and readings and applying it critically as demonstrated by test results, I was always disappointed in the lack of deep learning for a significant plurality of my students in traditionally taught lectures. CLICing is serving to remedy this and I am committed to continuing the development of both high quality cinelectures and active learning modules.
There is no question that whole scale conversion of a traditional lecture class to a CLICed format is an ambitious undertaking, especially if a faculty member launches such a project independently, as was done in the present case. The production of high quality cinematic lectures along with the development of effective active learning exercises should be undertaken as a long-term project with the understanding that a thorough transformation of a traditional lecture hall into an arena of inquiry is an ongoing, iterative process. For readers who are contemplating instituting a CLIC model of classroom flipping, several suggestions are in order: 1) if your institution has a teaching and learning center with staff who are familiar with screencasting methods, seek their help in producing cinematic lectures; 2) spend some time researching active learning resources that may be freely available for your discipline; 3) start small – try to create one or two cinelectures and associated active learning modules in order to flip a small section of an existing course, and expand this approach incrementally.
Distributed under a Creative Commons Attribution-ShareAlike 3.0 Unported License.
NOTES
-
The term active learning (AL) may acquire different, contextual meanings, but it is generally understood as pedagogy in which students are encouraged to develop their learning autonomously. AL requires students to ask questions of themselves and their peers in solving problems, in contrast to passively receiving information. Process-oriented, guided inquiry learning (POGIL), peer-led team learning (PLTL), problem-based learning (PBL), and Investigative Case Based Learning (ICBL) are often components of an AL environment (e.g. see reference 26).
-
Introduction to Metabolism, Genes and Development, “The Flow of Life.” This course introduces energy flow within cells, mechanisms of heredity, the expression of genetic information, and the means by which genes encode developmental programs. Emphasis is placed on the concept that genetic mechanisms and developmental controls reveal a fundamental kinship of all life. The tools of genetics, including mutational analysis of model organisms, recombinant DNA, and biotechnology are introduced.
REFERENCES CITED
Alberts, P.P., Murray, L.A., Griffin, D.K., and J. Stephenson (2007). In Joseph Fong, Fu Lee Wang (eds): Blended Learning, pp. 53-65, Pearson.
Allen, D., and Tanner, K. (2005). Infusing active learning into the large-enrollment biology class: seven strategies, from the simple to complex. Cell Biol. Educ. 4, 262–268.
Anderson, L.W., Krathwohl, D.R., Airasian, P.W., Cruikshank, K., Mayer, R.E., Pintrich, P.R., Raths, J., and Wittrock, M.C. (2000). A Taxonomy for Learning, Teaching, and Assessing: A Revision of Bloom’s Taxonomy of Educational Objectives, Abridged Edition. Allyn & Bacon.
Association of American Colleges and Universities (2009). The Biochemistry and Molecular Biology Major and Liberal Education. Washington, DC.
American Society for Cell Biology (2012). Instructions for Authors. CBE Life Sciences Education. Accessed March 10, 2012 from [www.lifescied.org/misc/ifora.dtl].
Andrews, T.M., Leonard, M.J., Colgrove, C.A., Kalinowski S.T. (2011). Active learning not associated with student learning in a random sample of college biology courses. CBE Life Sciences Education, 10(4): 394-405.
Baker, J.W. (2000). The “classroom flip”: using web course management tools to become the guide on the side. The 11th International Conference on College Teaching and Learning, Jacksonville, Florida.
Beichner, R. J., Saul, J. M., Abbott, D. S., Morse, J. J., Deardorff, D. L., Allain, R. J., Bonham, J. W., Dancy, M. H., and Risley, J. S. (2007). The Student-Centered Activities for Large Enrollment Undergraduate Programs (SCALE-UP) Project. Accessed May 2, 2012 from http://www.compadre.org/Repository/document/ServeFile.cfm?ID=4517&DocID=183.
Bergmann, J. (2011). The Flipped Classroom Revisited. Ed Reach, November 28, 2011. [http://edreach.us/2011/11/28/the-flipped-classroom-revisited/].
Berrett, D. (2012). How ‘Flipping’ the Classroom Can Improve the Traditional Lecture. The Chronicle of Higher Education, February 19.
Bonwell, C.C., and Eison, J.A. (1991). Active Learning: Creating Excitement in the Classroom. Educational Research Information Center and The National Teaching and Learning Forum. Accessed April 29, 2012 from [http://www.ntlf.com/html/lib/bib/91-9dig.htm].
Boyer Commission on Educating Undergraduates in the Research University (1998). Reinventing Undergraduate Education: A Blueprint for America’s Research Universities. Accessed February 16, 2012 from [http://www.eric.ed.gov/ERICWebPortal/contentdelivery/servlet/ERICServlet?accno=ED424840].
Brownell, S. and Tanner, K. (2012). Barriers to Faculty Pedagogical Change” Lack of Training, Time, Incentives, and…Tensions with Professional Identity? CBE-Life Sciences Education 11: 339-346.
Burrowes, P.A. (2003). A Student-Centered Approach to Teaching General Biology That Really Works: Lord’s Constructivist Model Put to a Test. The American Biology Teacher, 65: 491– 502.
Cerbin, B. (2012). Assessing How Students Learn. Carnegie Foundation for the Advancement of Teaching. [http://www.carnegiefoundation.org/perspectives/assessing-how-students-learn].
Christensen, C., Johnson, C.W., and Horn, M.B. (2010). Disrupting Class, Expanded Edition: How Disruptive Innovation Will Change the Way the World Learns. McGraw Hill, NY.
Clark, R.C., and Mayer, R.E. (2011). e-Learning and the Science of Instruction: Proven Guidelines for Consumers and Designers of Multimedia Learning. Pfeiffer (Wiley), San Francisco.
Crouch, C.H., and Mazur, E. (2001). Peer Instruction: Ten years of experience and results. American Journal of Physics, 69: 970-977.
Davis, B. (1997). National Research Council (NRC). Science Teaching Reconsidered: A Handbook. Washington, DC: National Academies Press; 1997. [http://www.nap.edu/catalog/5287.html]
Denton, D.D. (1995). On Being a Change Agent in SME&T Education. Project Kaleidoscope F21 National Assembly, “Building a Network of the Next Century of Leaders.” Accessed, February 15, 2012 from [www.pkal.org/documents/denton.pdf].
Derting, T.L., Ebert-May, D. (2010). Learner-Centered Inquiry in Undergraduate Biology: Positive Relationships with Long-Term Student Achievement. CBE Life Sciences Education, 9: 462–472.
Deslauriers, L., Schelew, E., and Wieman, C. (2011). Improved Learning in a Large-Enrollment Physics Class. Science, 332: 862-864.
Donovan, M. S., and Bransford, J. D. (eds.) (2005). How Students Learn: Science in the Classroom. National Academies Press, Washington DC.
Eberlein, T., Kampmeier, J., Minderhout, V., Moog, R.S., Platt, T., Varma-Nelson, P., and White, H.B. (2008). Pedagogies of Engagement in Science. Biochemistry and Molecular Biology Education, 36: 262–273.
Freeman, S., Haak, D., and Wenderoth, M.P. (2011). Increased Course Structure Improves Performance in Introductory Biology. CBE Life Sciences Education, 10: 175-186.
Froyd, D. (2007). Evidence for the Efficacy of Student-active Learning Pedagogies. Project Kaleidoscope. [http://www.pkal.org/keywords/Pedagogies.cfm].
Garrison, R.D., and Kanuka, H. (2004). Blended learning: Uncovering its transformative potential in higher education. Internet and Higher Education: 7: 95–105.
Goldstein, R.E., Nelson, P.C., and Powers, T.R. (2005). Teaching Biological Physics. Physics Today (March): 46-51. Accessed April 25, 2012 from [www.physics.upenn.edu/~pcn/Mss/TeachingBP.pdf].
Haak, D.C., HilleRisLambers, J., Pitre, E., and Freeman, S. (2011). Increased Structure and Active Learning Reduce the Achievement Gap in Introductory Biology. Science, 332: 1213-1216.
Hake, R. R. (1998). Interactive-Engagement vs. Traditional Methods: A Six-Thousand-Student Survey of Mechanics Test Data for Introductory Physics Courses. American Journal of Physics, 66: 64–74.
Handelsman, J., D., Ebert-May, R., Beichner, P., Bruns, A., Chang, R., DeHaan, J., Gentile, S., Lauffer, J., Stewart, J., Tilghman, S.M., and Wood, W.B. (2004). Scientific Teaching. Science, 304: 521–522.
Handelsman, J, S. Miller, and C. Pfund (2007). Scientific Teaching. W. H. Freeman (New York).
Hanford, E. (2012). Don’t Lecture Me. American Radio Works. Accessed May 2, 2012 from http://americanradioworks.publicradio.org/features/tomorrows-college/lectures/.
Herron, J. (2012). Population Genetics Software for Evolutionary Analysis. Accessed April 15, 2012 from [http://faculty.washington.edu/herronjc/SoftwareFolder/AlleleA1.html].
Hestenes, D., Wells, M., and Swackhamer, G. (1992). Force Concept Inventory. The Physics Teacher, 30: 141-158
Honan, W.H. (2002, August 14). The college lecture, long derided, may be fading. The New York Times, Section B, p. 7.
Jacob, F.; Monod, J. (1964). Biochemical and Genetic Mechanisms of Regulation in the Bacterial Cell. Bulletin de la Societe de chimie biologique, 46: 1499–1532.
Kang, J., and Purnell, C.B. (2011). Implications for Undergraduate Education of Two Interdisciplinary Biological Sciences: Biochemistry and Biophysics. CBE Life Sciences Education, 10: 111–112.
Kay, R., and Kletskin, I. (2012). Evaluating the use of problem-based video podcasts to teach mathematics in higher education. Computers & Education, 59: 619-627.
Kennedy, D. (2009). In Integrated Science: A Virtual Roundtable Discussion, Brint, M.E., Marcey, D.J., and Shaw, M.C. (eds), pp. 83-89. Springer, New York, 2009.
Khan, S. (2012). Q&A: Khan Academy Creator Talks About K-12 Innovation. Education Week, March 7, 2012. Accessed May 1, 2012 from [http://www.edweek.org/ew/articles/2012/03/07/23biz-qanda- khan.h31.html?qs=Khan+Academy].
King, A. (1993). From sage on the stage to guide on the side. College Teaching, 41: 30-35.
Knight J.K., Wood W.B. (2005). Teaching more by lecturing less. Cell Biology Education, 4(4):298-310.
Knight, R.D., (2004). Five Easy Lessons – Strategies for Successful Physics Teaching. Pearson Education, Addison Wesley.
Lage, M.J., and Platt, G.J. (2000). The internet and the inverted classroom. Journal of Economic Education, 31: 31-43.
Lentz, N.H., and Cifuentes, O.E. (2009). Web-Based Learning Enhancements: Video Lectures Through Voice-Over Powerpoint in a Majors-Level Biology Course. Journal of College Science Teaching, Nov/Dec: 38-46.
Loverude, M.E., Kautz, C.H., and Heron, P.R.L. (2002). Student understanding of the first law of thermodynamics: Relating work to the adiabatic compression of an ideal gas. American Journal of Physics, 70(2), 137–148.
McCray, R.A., R. DeHaan, and J.A. Schuck (Eds) (2003). Improving undergraduate instruction in science, technology, engineering, and mathematics: Report of a workshop. National Research Council, Washington, DC: The National Academies Press. [http://www.nap.edu/catalog.php?record_id=10711].
McFarlin, B.K. (2008). Hybrid lecture-online format increases student grades in an undergraduate exercise physiology course at a large urban university. Advances in Physiology Education, 22: 86-91.
Means, B., Toyama, Y., Murphy, R., Bakia, M., Jones, K. (2009). Evaluation of Evidence-Based Practices in Online Learning: A Meta-Analysis and Review of Online Learning Studies. U.S. Department of Education. Accessed on May 1, 2012 from [http://repository.alt.ac.uk/629/].
Mervis, J. (2010). Better Intro Courses Seen as Key to Reducing Attrition of STEM Majors. Science, 330: 306.
National Research Council. (2000). How People Learn: Brain, Mind, Experience, and School. Washington, DC: National Academies Press. [http://www.nap.edu/catalog/9853.html].
National Research Council. (2003). Improving Undergraduate Instruction in Science, Technology, Engineering, and Mathematics: Report of a Workshop. National Academies Press. Washington, DC. [http://www.nap.edu/catalog.php?record_id=10711].
National Research Council. (2003). BIO2010: Transforming Undergraduate Education for Future Research Biologists. Washington, DC: National Academies Press. [http://www.nap.edu/openbook.php?isbn=0309085357].
National Science Foundation. (1996).Report of the Advisory Committee to the NSF Directorate for Education and Human Resources. Shaping the Future: New Experiences for Undergraduate Education in Science, Mathematics, Engineering, and Technology. http://www.nsf.gov/pubs/1998/nsf9873/nsf9873.doc].
Nielsen, N. (2011). Promising Practices in Undergraduate Science, Technology, Engineering, and Mathematics Education: Summary of Two Workshops. Planning Committee on Evidence on Selected Innovations in Undergraduate STEM Education. National Research Council Board on Science Education, Division of Behavioral and Social Sciences and Education. The National Academies Press. Washington, DC. [http://www.nap.edu/catalog.php?record_id=13099].
Noschese, F. (2012). Khan Academy Criticism. Action-Reaction Blog: Reflections on the Dynamics of Teaching: http://fnoschese.wordpress.com/tag/khan-academy/. Accessed on May 9, 2013.
Ojalvo, H.E., and Doyne, S. (2011). Five Ways to Flip Your Classroom. The New York Times, December 8, 2011. Accessed March 5, 2012 from [http://learning.blogs.nytimes.com/2011/12/08/five-ways-to-flip-your-classroom-with-the-new-york-times/?partner=rss&emc=rss].
Picciano, A. (2010). Blending With Purpose: the Multimodal Model. Journal of Asynchronous Learning Networks, 13: 7-18.
PCAST [President’s Council of Advisors on Science and Technology] (2012). Report to the President: Engage to Excel: Producing One Million Additional College Graduates with Degrees in Science, Technology, Engineering, and Mathematics. Accessed March 5, 2014 from [http://www.whitehouse.gov/sites/default/files/microsites/ostp/pcast-engage-to-excel-final_2-25-12.pdf.]
Prince, M. (2004). Does Active Learning Work? A Review of the Research. Journal of Engineering Education, 93: 223-231.
Project Kaleidoscope (2004). TwentyFirst Century Science & the Facilities of the Future. What Works, What Matters, What Lasts, Volume IV. Accessed April 14, 2012 from [http://www.pkal.org/documents/21stCenturyScienceAndFacilities.cfm].
Project Kaleidoscope (2012). TwentyFirst Century Science & the Facilities of the Future. What Works, What Matters, What Lasts, Volume IV. Accessed April 14, 2012 from [http://www.pkal.org/documents/21stCenturyScienceAndFacilities.cfm].
Reasons, S.G., Valadares, K., and Slavkin, M. (2005). Questioning the Hybrid Model: Student Outcomes in Different Course Formats. Journal of Asynchronous Learning Networks, 9: 83-94.
Reiss, K. (1955). Biophysics in the Premedical Curriculum. American Journal of Physics, 23: 5.
Riffell, S., and Sibley, D. (2005). Using web-based instruction to improve large undergraduate biology courses: An evaluation of a hybrid course format. Computers & Education, 44: 217-235.
Ruiz-Primo, M.A., Briggs, D., Iverson, H., Talbot, R., Shepard, L.A. (2011). Impact of undergraduate science course innovations on learning. Science, 331: 1269–1270.
Sadaghiani, H. (2012). Online Prelectures: An Alternative to Textbook Reading Assignments. The Physics Teacher, 50:301-303.
Sams, A. (2011). The Flipped Class: Shedding light on the confusion, critique, and hype. The Daily Riff. [http://www.thedailyriff.com/articles/the-flipped-class-shedding-light-on-the-confusion-critique-and-hype-801.php].
Sundberg, M.D. (2002). Assessing student learning. Cell Biology Education, 1: 11–15.
Smith, M.K., Wood W.B., Krauter, K., Knight J.K. (2011). Combining peer discussion with instructor explanation increases student learning from in-class concept questions. CBE Life Sciences Education, 10(1): 55-63.
Stokstad, E. (2001). Reintroducing the Intro Course. Science, 293: 1608-1610.
Springer, L., Stanne, M. E., and Donovan, S. S. (1999). Effects of small-group learning on undergraduates in science, mathematics, engineering, and technology: A meta-analysis. Review of Educational Research, 69: 21-51
Terenzini, P. T., Cabrera, A. F., Colbeck, C. L., Parente, J. M., Bjorklund, S. A. (2001). Collaborative Learning vs. Lecture/Discussion: Students’ Reported Learning Gains. Journal of Engineering Education, 90, 123–130.
Udovic, D., Morris, D., Dickman, A., Postlethwait, J., and Wetherwax, P. (2002). Workshop Biology: Demonstrating the Effectiveness of Active Learning in an Introductory Biology Course. Bioscience, 52: 272- 281.
Vision and Change in Undergraduate Biology Education (2011). http://visionandchange.org/files/2011/03/Revised-Vision-and-Change-Final-Report.pdf, Accessed on April 1, 2014.
Walker, J.D., Cotner, S. & Beermann, N. (2011). Vodcasts and Captures: Using Multimedia to Improve Student Learning in Introductory Biology. Journal of Educational Multimedia and Hypermedia, 20: 97-111. Accessed on April 10, 2012 from http://www.editlib.org/p/35300.
Wood, B.W., and Gentile, J.M. (2003). Teaching in a research context. Science, 302: 1510.
Wright, J.C., Millar, S.B., Kosciuk, S.A., Penberthy, D.L., Williams, P.H., and Wampold, B.E. (1998). A novel strategy for assessing the effects of curriculum reform on student competence. Journal of Chemical Education, 75(8), 986–992.
Pingback: Collaboration: A Primer | Mnanfito@work
Pingback: Collaboration: A Primer | The Academic Commons