Instruments to explore blending learning: Modifying a method to analyze online communication for the analysis of face-to-face communication

De Leng, Bas A., et al. “Instruments to Explore Blended Learning: Modifying a Method to Analyse Online Communicaiton for the Analysis of Face-to-Face Communication.” Computers & Education 55.2 (2010): 644-51.

De Leng, B. A., Dolmans, D. H.J.M., Muijtjens, A. M.M., & van der Vleuten, C. P.M. (2010). Instruments to explore blended learning: Modifying a method to analyze online communication for the analysis of face-to-face communication. Computers & Education, 55(2), 644-51.

Abstract

In the complex practice of today’s blended learning, educators need to be able to evaluate both online and face-to-face communication in order to get the full picture of what is going on in blended learning scenarios. The aim of this study was to investigate the reliability and feasibility of a practical instrument for analyzing face-to-face communication that is in alignment with tools for analyzing online communication. We examined whether an existing coding scheme for online communication could be modified to evaluate face-to-face communication. We investigated the reliability, validity and usability of a modified version of the Transcript Analysis Tool (TAT) in analyzing communication among students during small-group sessions comprising presentations by students and discussion. The modified TAT proved to be effective in obtaining a detailed description of students’ participation in the communication and of the focus of the communication. Evaluation was more efficient when direct video analysis was used than when a transcript of the same video was analyzed. Direct analysis took about 2 h for 20 min of video recordings and had an inter-rater reliability of k = 0.56. The modified TAT appears to be an effective and efficient instrument to face-to-face communication in small group sessions. Its combined use with an evaluation instrument for online learning may create a powerful evaluation instrument for blended learning. Further studies are needed to establish the value of such an instrument.

Summary
Communication research is made up of two types of analysis; conversations (interaction) and content. This study uses Fahy’s Transcript Analysis Tool (TAT) because it allows for coding of both the interaction and the content components of communication research. TAT is broken up into three different categories, which include questioning, statements and quotations. TAT was formed for coding online texts, but it is believed that the TAT coding process would work for face-to-face communication as well. This goal of this study is to determine if a modified version of the TAT could be used to study both the online and face-to-face communication styles.

The following research questions were addressed:

1. Is the modified TAT a reliable instrument for analyzing face-to-face communication?

- What is the inter-rater reliability of the modified TAT when it is used for direct analysis of video recording of face-to-face communication?

- What is the inter-rater reliability of the modified TAT when it is used to analyses transcripts of video recordings of face-to-face communication?

2. Is the modified TAT an efficient instrument to analyze a video recording of face-to-face communication?

- Is there a correlation between analyses, using the modified TAT, of video recording and of transcript of the same recording of communication in a small group of students?

- How does the time needed for direct video analysis using TAT compare with the time needed to analyze a transcript of the same recoding using TAT?

3. Are there benefits in using the modified TAT for direct video analysis of face-to-face communication among students during presentations and discussions in small groups?

Students at an undergraduate medical curriculum in their 4th and 5th years were used for this study. This group was picked for the study because of the use of presentations in their coursework. The study size was small, only 8 students. A 20 minute long segment was analyzed using the modified version of TAT by dividing utterances into four different categories. The two raters of the section did not rate every section exactly the same, “but clear concordances could be indentified based on the identities of the different speakers” (646). The raters eventually decided on 82 different utterances.

The results indicate that the modified Fahy’s Transcript Analysis Tool can be efficient for studying communication in a small group setting. The biggest benefit of analyzing the direct video communication “is that it yield a rich description of these sessions” (649). The biggest limitation of this study is that such a small amount of videos were analyzed, which included only a small amount of participants. More studies with more groups will need to be done.