Introduction

According to Pintrich, Marx and Boyle (1993), learning processes involve more than “cold cognition” and include additional factors such as affective and social variables which need to be taken into account when investigating learning processes. More recent psychological studies emphasise the significance of emotions in learning and achievement (Mayring & Rhöneck, 2003; Pekrun, 2000). Students experience learning and achievement situations individually in a different manner depending on their previous experiences, their social context, their own personal goals, their individual interests, and a number of other personality factors (Götz, Zimgibl, Pekrun, & Hall, 2003). For example, Allen (2010) found that pupils who experienced more intense emotions during an educational intervention demonstrated the most gains after the process.

However, until recently there has been a general paucity of data on the role of emotions in classroom instruction (Gläser-Zikuda, Fuß, Raukenmann, Metz & Randler, 2005; Pekrun, Götz, Titz, & Perry, 2002) and where research has been done it has tended to focus on the cognitive- emotional construct „interest". One reason why emotions have not been studied in instructional situations was due to the fact that they were seen as interfering with learning and achievement (Gläser-Zikuda et al., 2005). At best, they are mentioned as motivational aptitude or as affective learning outcome (Fraser, Walberg, Welch, & Hattie, 1987). However, Värlander (2008) argues that emotions should not be considered as hindering learning, but rather as being a natural part of it and as having a focal role in the process, and the positive impact of interest on learning processes has been confirmed with regard to both knowledge domains and subject areas (Hidi, Bem- dorff, & Ainley, 2002). Teachers" didactic competencies, students" academic achievement and interest, and social interactions have been found to correlate with successful learning processes at school (Hascher, 2003), and boredom has been shown to be related to attention problems and negative intrinsic motivation (Pekrun, Götz, Daniels, Stupnisky, & Perry, 2010).

Much of the previous work on emotions (and especially on interest) is based on trait-survey studies (assuming that a given trait is a personality variable that is resistant to short and sudden changes) by using questionnaires in cross-sectional designs assessing, e.g. interest, across grades and school topics, often in retrospect. In our study, we used a concept that distinguishes between current situational emotions and biographically developed and enduring "trait"- emotions (Ulich & Mayring, 1992) as interest may fluctuate during a lesson (Palmer, 2009). The idea behind this distinction can be clarified by the following example: Pupils may experience a particular lesson, e.g. a hands-on lesson or a specific topic, as interesting even though they do not have a general interest in the subject itself (see Raukenmann et al., 2003). Here, we define "situational emotions" as emotions that are sensitive to changes and that are not developed as a stable trait factor (as, e.g. general interest in a specific topic). Rike many psychological variables, state and trait components exist simultaneously (Spielberger, Gorsuch, & Rushene, 1970).

A central aspect of implementation and of treatments in learning studies is the need to assess situational emotions as some kind of moderating variables, because they are related to learning success. There were just very few approaches that measured situational emotions (see as examples, Gläser-Zikuda, 2010; Gläser-Zikuda et al., 2005). The main focus of our present work is to further modify a short scale to measure differences in emotions during learning processes; this specific scale is supposed to be applied in different educational settings, such as in out-of-school as well as in typical school settings, and from 5^th grade up to university level, as well as in formal and informal learning environments. The short scale has its benefit because it is less time consuming and can be applied many times during an educational unit at the end of lessons.

As there is a need to assess situational emotions as a moderating variable when implementing educational interventions and, because there are very few approaches that measure situational emotions (Gläser-Zikuda, 2010; Gläser-Zikuda et al., 2005), the main focus of our study was to investigate the reliability and validity of a short scale to measure situational emotions in different educational settings from 5 ^th grade up to university level (the benefit of the short scale being that it is less time consuming and can be applied many times during an educational unit at the end of individual lessons). For the purposes of the study we differentiated between a more cognitive-evaluative (satisfaction) and a more affective (joy) dimension in terms of the concept ‘well-being' (c.f., Strack, Argyle, & Schwarz, 1990; Mayring, 2009). Interest is defined as a specific subject-topic-relationship which specifically includes importance and utility (c.f., Hidi, Renninger & Krapp, 1992), and boredom is defined by the components lack of action and interest, as well as subject-related boredom (c.f. Bellebaum, 1990; Csikszentmihalyi & LeFevre, 1987). Well-being is more related to a subjective positive feeling during the lessons, while interest has a more cognitive orientation, and boredom finally is related to a lack of action and interest.

Previous work assessed situational emotions immediately after school lessons by using different versions of a situational emotion scale (Gläser-Zikuda et al., 2005; Gläser-Zikuda & Fuß, 2008; Laukenmann et al., 2003; Randler, 2004, 2009). In this study we investigate the reliability and validity of a much shorter scale based on just three items for each of the dimensions; interest, well-being and boredom. The resultant nine-item scale is easy to apply in most school situation, for example at the end of a lesson or a field trip. We present four different studies based on four different pupil/student populations. The first study was done to provide evidence for the factor structure of the three dimensions (interest, well-being, boredom) of the scale, the second study sought evidence for sensitivity across different subjects and across different grades (we assumed that pupils would assess different lessons and subjects differently - as is known from trait surveys). The third study was done to assess sensitivity on an individual basis (individual reactions to different lessons in biology, namely zoology versus botany, were presented). Finally, we used another scale based on a different motivational theory to seek external validation of the shortened scale by correlating these scores with our scores when using a university student sample.

Methods

As noted above, we based our validation on four different studies based on four different samples (three at school level, one at University level). In all studies, the response to the nine-item questionnaire was at the end of the lesson (usually within the last five minutes).

Participants and Data Collection

The first study was based on 393 pupils (188 boys, 205 girls) from South-Western Germany. The pupils (all 5^th and 6^th graders) participated in an educational unit about bird flight (Hummel & Randler, 2010). All teaching was based on the same kind of instruction. The teaching materials were standardised by using a booklet where a hands-on educational unit was offered. Fourteen different teachers participated in the study. Of course, there may still be differences between the individual teachers, but we aimed for at least some kind of standardisation by providing standardised teaching materials and a booklet guiding pupils through their learning process. At the end of the lesson, pupils filled in the questionnaire and we used these data for the confirmatory factor analysis.

The second study was based on 141 pupils (74 girls, 67 boys), aged 12-19 years consisting of 7^th (12) 8^th (24) 9* (50) 10^th (55) graders. Different subjects such as biology, chemistry, German language and social sciences were taught by different teachers to assess differences between subjects and grades. At the end of a lesson, the pupils filled in the questionnaire and we compared the situational emotions. We suppose that situational differences should occur between subject and grades.

The third study used one lesson (90 minutes) about the water lily from a standardised educational program developed by Randler & Bogner (2009), and a 90 minute lesson about snail eco
logy designed by Hummel & Randler (2010). Ninety-seven pupils (48 boys, 49 girls; 53 5^th and 44 6^th graders) participated in this aspect of the study. We focused on assessing intra-individual differences by applying two similar instructions (hands-on experiments) based on different topics, one instruction dealing with a zoological topic (snail), and the other was a botanical topic (water lily). As we know from previous research in interest in biology (Randler & Bogner, 2007), pupils generally rate zoological topics higher and as more interesting than botanical ones. As such, we expected that pupils should give higher scores in interest and well-being and lower scores in boredom after the zoological topic, than they would do for the botanical topic. However, these previous ratings have only been measured using trait scales (as compared to situational scales).

The fourth study sought additional external validity and was done with University students at the University of Education Heidelberg. We compared measures of our situational emotions scale with another short scale as provided by Wilde, Bätz, Kovaleva, & Urhahne (2009; labelled KIM - Kurzskala zur Intrinsischen Motivation - short scale for intrinsic motivation). This construct is based on measuring motivation based on the self-determination theory of Deci & Ryan (1985, 2003), which contains the domains interest/enjoyment, competence, pressure/tension (negative), and perceived choice. At the end of each course, students filled in both questionnaire; the KIM and our scale for situational emotions. We calculated the overall means over 11 course days (which were assessed each time) and used the individual data to calculate correlation coefficients as an indicator of external validity.

Statistical Methods

Confirmatory factor analysis (study I): As the three independent factors have been derived from theoretical constructs (Gläser-Zikuda et al., 2005), we assessed the model by using a confirmatory factor analysis (Brown, 2006) using EISREE 8.80 (Jöreskog & Sörbom, 1993). In study II, we used t-tests and one-way-ANOVA to compare the answers according to grade, subject and gender. To assess different variables in parallel, we used a general linear model (GEM). In a first step, all independent variables entered the model simultaneously, and in the second step, all nonsignificant variables were removed to produce the final model. In study III, we compared the answers of the pupils to the zoological and botanical topic by using paired t-Tests. Finally, for study IV, the validation with the KIM, we used Pearson's correlations to assess the relationship between the three emotional constructs of our scale and the four constructs of the KIM.

Results

Study I - Factor Structure of the Scale

Study II Differences between Subjects

Reliability was calculated using Cronbach's a of .79 for well-being, of .73 for interest, and of .72 for boredom. As this study was based on different age groups and grades of the pupils and on different types of lessons and subjects, we expected differences between the variables (e.g., gender, grade, age groups). As a first step, we used a general linear model (GLM) with age as covariate, gender, subject and grade as fixed factors. In this model, we revealed no significant main effect for gender and for age (covariate), thus, these variables were removed from the model. The final model (Table 1) showed a significant influence of subject and of grade. The assessment of interest, well-being and boredom is different in different grades and in different subjects. This provides evidence for the fact that different instructions are assessed differently by the pupils which, in turn, confirms the situational characteristic of the scale. Also, different instructions, teachers and lessons may have evoked different situational emotions.

Study III - Intra-individual Differences

We obtained reliabilities of .82 (snail) and .83 (water lily) for well-being, of .73 (snail) and .74 (water lily) for interest, and of .80 (snail) and .82 (water lily) for boredom. In this study, the same pupils received two different lessons that were identical in structure. Both were biological hands- on lessons (see methods) but they differed in the subject matter (zoological versus botanical topic): We found significant differences between the two lessons (snail ecology versus water lily). Pupils rated the zoological topic snail as more interesting and less boring than the botanical topic water lily, and felt better (Table 2). This indicates that the scale is sensitive to changes on the individual level because the same individuals rated the different lessons in a different manner (a typical trait variable should be constant over time and not sensitive towards changes).

Study IV - External Validity

The fourth study sought additional external validity with a well-known instrument derived from Deci & Ryan (2003; in the shortened German version, labeled KIM: Kurzskala zur Intrinsischen Motivation - short scale for assessing intrinsic motivation). As noted earlier, this scale can be used at the end of a lesson or at the end of an educational unit and assesses four dimension of motivation. There were significant correlations between the different dimensions, e.g., the KIM interest scale was positively related to the interest and well-being scales of our scale and negatively related to the boredom scale. Similarly, KIM competence was positively related to interest and well-being and negatively to boredom of our scales. KIM pressure/tension was negatively related to well-being and positively to boredom, suggesting that students feeling pressed experienced higher boredom and lower well-being. KIM perceived choice showed the lowest correlations because this measures a slightly different construct, but perceived choice was positively related to well-being.

Discussion

Our study provides evidence that a shortened scale is sufficient to measure short-term learning situational emotions, and therefore is a suitable means for educational research of this nature. The shortness of the scale allows an application more than once during an educational unit (as noted earlier, longer scales may stir up aversion or reactance). Overall the emotional variables were rated differently with regard to grades and/or subject matter, which suggest that our scale is suitable to measure differences between lessons, and within pupil cohorts. In study-II, no differences in age and gender were revealed which suggests that the scale can be applied effectively from age 10 onwards until (and including) university age students. We do, however, recognize that such differences may emerge from a larger sample size - something which may provide an interesting issue for investigation.

The within subject differences from study III point towards a real situational characteristic of our scale because the same pupils rated the different lessons differently (as expected), and the teacher was identical, thus removing any teacher effects from the results (Randler & Bogner, 2009). This suggests that the scale is sensitive towards changes. Furthermore, study III adds external validity, because trait survey studies on interest in biology revealed that the zoological topics were more interesting to the pupils than the botanical ones (see, e.g., Löwe, 1992; Hong, Shim, & Chang, 1998; Lindemann-Matthies, 2005). This difference between zoological and botanical topics found in trait interest studies is reflected in our measurements of situational interest.

In study IV, we sought external validity by correlating our situational emotion scale with an established measurement on motivation, the KIM based on the motivational theory of Deci & Ryan (1985), as provided by Wilde et al. (2009). Both interest scales correlated with each other and in both cases boredom was negatively related to intrinsic motivation (see also Pekrun et al., 2010)     suggesting that the scale, although shortened, remains valid. In turn, the Cronbach's a scores attributed to the data attest to the internal reliability of the instrument.

References

1. Allen, M. (2010). Learner error, affectual stimulation, and conceptual change. Journal of Research in Science Teaching, 47, 151-173.


2. Bellebaum, A. (1990). Langeweile, Überdruß und Lebenssinn. Opladen: Westdeutscher Verlag. Brown, T. A. (2006). Confirmatory factor analysis for applied research. The Guilfford Press.


3. Csikszentmihalyi, M. & LeFevre, J. (1987). The experience of work and leisure. Journal of Personality and Social Psychology, 56, 815-822.


4. Deci, E. L. & Ryan, R. M. (1985). Intrinsic motivation and self-determination in human behavior. New York: Plenum.


5. Deci, E. L. & Ryan, R. M. (2003). Intrinsic Motivation Inventory. http://www.psych.rochester.edu/SDT/measures/intrins.html.


6. Eraser, B. J., Walberg, H. J., Welch, W. W. & Hattie, J. A. (1987). Synthesis of educational productivity research. International Journal of Educational Research, 11, 145-252.


7. Gläser-Zikuda, M. (2010). Interventionsstudien zu Emotionen im Bildungskontext. In T. Hascher & B. Schmitz (Hrsg.). Handbuch Pädagogische Interventionsforschung (pp. 111-132). Weinheim: Juventa.


8. Gläser-Zikuda, M. & Fuß, S. (2008). Impact of teacher competencies on students' emotion - a multi-method approach. International Journal of Educational Research, 47, 136-147.


9. Gläser-Zikuda, M., Fuß, S., Laukenmann, M., Metz, K. & Randler, C. (2005). Promoting students“ emotions and achievement - conception and evaluation of the ECOLE approach. Learning and Instruction, 15, 481-495.


10. Götz, T., Zimgibl, A., Pekrun, R. & Hall, N. (2003). Emotions, learning, and achievement from an educational-psychological perspective. In P. Mayring & C. v. Rhöneck (Eds.). Learning emotions. The influence of affective factors on classroom learning (pp. 9-28). London: Peter Lang.


11. Hascher, T. (2003). Well-being in school—Why students need social support. In P. Mayring & C. v.Rhöneck (Eds.). Learning emotions. The influence of affective factors on classroom learning (pp. 127-142). London: Peter Lang.


12. Hidi, S., Renninger, K.-A. & Krapp, A. (1992). The role of interest in learning and development. Hillsdale: Erlbaum.


13. Hidi, S., Bemdorff, D. & Ainley, M. (2002). Children's argument writing, interest and self- efficacy: an intervention study. Learning and Instruction, 12, 429-446.


14. Hong, J.-L., Shim, K.-C. & Chang, N.-K. (1998). A study of Korean middle school students' interests in biology and their implications for biology education. International Journal of Science Education, 20, 989-999.


15. Hummel, E. & Randler, C. (2010). Experiments with living animals - effects on learning success, experimental competency and emotions. Procedia - Social and Behavioral Sciences, 2, 3823-3830.


16. Jöreskog, К. G. & Sörbom, D. (1993). LISREL 8: User's reference guide. SSI Scientific Software International: Lincolnwood.


17. Laukenmann, M., Bleicher, M., Fuß, S., Gläser-Zikuda, M., Mayring, P. & Rhöneck, C. (2003). An investigation on the influence of emotions on learning in physics. International Journal of Science Education, 25, 489-507.


18. Lindemann-Matthies, P. (2005). ‘Loveable' mammals and ‘lifeless' plants: how children's interests in common local organisms can be enhanced through observation of nature. International Journal of Science Education, 27, 655-677.


19. Löwe, В. (1992). Biologieunterricht und Schülerinteresse an Biologie. Weinheim: Beltz.


20. Mayring, P. & v. Rhöneck, C. (2003). Learning Emotions. The influence of affective factors on classroom learning. Peter Lang, Frankfurt, London.


21. Mayring, P. (2009). Freude und Glück. In V. Brandstätter & J.H. Otto (Hrsg.), Handbuch der Allgemeinen Psychologie - Motivation und Emotion (pp. 585 - 595). Göttingen Hogrefe.


22. Palmer, D. H. (2009). Student interest generated during an inquiry skills lesson. Journal of Research in Science Teaching, 46, 147-165.


23. Pekrun, R., Götz, T., Titz, W. & Perry, R. P. (2002). Academic emotions in students' selfregulated learning and achievement: A program of qualitative and quantitative research. Educational Psychologist, 37, 91-105.


24. Pekrun, R. (2000). A social-cognitive, control-value theory of achievement emotions. In J. Heckhausen (Ed.), Motivational psychology of human development. Developing motivation and motivating development (pp. 143-163). New York, NY: Elsevier.


25. Pekrun, R, Götz, T., Daniels, L. M., Stupnisky, R. H. & Perry, R. P. (2010). Boredom in achievement settings: exploring control-value antecedents and performance outcomes of a neglected emotion. Journal of Educational Psychology, 102, 531-549.


26. Pintrich, P. R, Marx, R. W. & Boyle, R. A. (1993). Beyond cold conceptual change. The role of motivational beliefs and classroom contextual factors in the process of conceptual change. Review of Educational Research, 63, 167-199.


27. Randler, C. (2004). Kognitive und emotionale Faktoren des Lernens. Am Beispiel einer Biologieunterrichtseinheit „Lebensraum See“. Hamburg: Kovac.


28. Randler, C. & Bogner, E X (2007). Pupils' interest before, during and after a curriculum dealing with ecological topics and its relationship with achievement. Educational Research and Evaluation, 13, 463-478.


29. Randler, C. & Bogner, F. X. (2009). Efficacy of two different instructional methods involving complex ecological content. International Journal of Science and Mathematics Education,! ,315-337.


30. Randler, C. (2009). Association between emotional variables and school achievement. International Journal of Instruction, 2, 3-10.


31. Spielberger, C. D., Gorsuch, R. L. & Lushene, R E (1970). State-Trait Anxiety! Inventory, Manual for the State-Trait Anxiety! Inventory!', Palo Alto, CA: Consulting Psychologist Press.


32. Sturm, H. & Bogner, E X (2007). Student-oriented versus teacher- centred: The effect of learning at workstations about birds and bird flight on cognitive achievement and motivation. International Journal of Science Education, 30, 941-959.


33. Strack, F., Argyle, M. & Schwarz, N. (Eds.). (1990). Subjective well-being. Oxford: Pergamon.


34. Ulich, D. & Mayring, P. (1992). Psychologie der Emotionen. Stuttgart: Kohlhammer.


35. Wilde, M., Bätz, К , Kovaleva, A., & Urhahne, D. (2009). Überprüfung einer Kurzskala intrinsischer Motivation (KIM). Testing a short scale of intrinsic motivation. Zeitschrift für Didaktik der Naturwissenschaften, 15, 31-45.


36. Värlander, S. (2008). The role of students' emotions in formal feedback situations. Teaching in Higher Education, 13, 145-156.