INTRODUCTION

Objectives

This paper is part of a larger mixed method study tiiat developed and assessed an elementary garden-based science curriculum on insects, which used a school garden as an informal learning setting. Over the course of the study, we became interested in measuring and tracking shifts in students' attitudes toward the environment. In this paper, we report on an unintended phenomenon associated with the use of a school garden, tiiat is, the shift in students’ attitudes toward the environment. The question tiiat drove this part of the study was: "Do students' attitudes toward the environment shift over the course of their engagement in a school garden curriculum?" Our objective is to report on the challenges associated with measuring these shifts in attitudes, provide evidence both for and against these shifts, and to propose a direction we see as profitable — looking at these shifts through an informal learning lens. To that end, we provide details about the curriculum design, background literature tiiat informed our work, contradictory results from multiple sources, and suggestions for future research. This examination of student attitudes provides a new and important contribution to the small, but growing, body of research on garden-based learning and provides further evidence of the benefits of these out-of-school settings (Dierking, Falk, Rennie, Anderson, & Ellenbogen, 2003).

Background

Environmental issues gained a conspicuous position on the national policy agenda in the U.S. and the international community during the 1970s (Carter & Simmons, 2010; Dunlap, Van Liere, Mertig, & Jones, 2002; Palmer, 1998), a position which includes educational policy evidenced by President Obama’s inclusion of environmental literacy in the U.S. Department of Education budget since 2010 (NCLI Coalition, 2010). More broadly, several recent international research initiatives respond to global environmental change and achieving global sustainability. Developing young scientists who will potentially find the solutions to global environmental change is an important activity of such international research organizations (International Social Science Council, 2014). Environmental education, which produces environmentally literate and responsible citizens, is a means to this goal (Knapp, 2000). Europe has been at the foreground for advancing environmental education. An exploratory report commissioned by the Environment Directorate-General of the European Commission addressed how environmental education is taught in different countries, including subjects in which it is taught and teaching approaches that have been adopted (Stokes, Edge & West, 2001). Another report by the Parliamentary Assembly (1994) declared that environmental education is the most important means for addressing environmental problems and achieving sustainability, but not enough teachers are practicing environmental education in their classrooms. In this report, they recommend that the Council of Europe institute an action program for environmental education in teacher training. One venue for environmental education is outdoor learning opportunities. Dillon, Rickinson, Teamey, Morris, Choi, Sanders, and Benefield (2006) reviewed research conducted in Europe, Australia, and the United States on the value of outdoor learning experiences. They found that outdoor learning opportunities have many positive impacts, including improving students’ attitudes about the environment (Dillon et al., 2006). Similarly, Mittelstaedt, Sanker, and Vanderveer’s (1999) study of 46 U.S. children attending a five-day environmental summer program found that “although students arrived with a positive attitude toward the environment, they left the program with an even stronger environmental attitude” (p. 147).

Environmental Attitudes

Environmental attitudes are defined as “a psychological tendency expressed by evaluating the natural environment with some degree of favor or disfavor” (Milfont & Duckit, 2009, p. 81). There is no gold standard for measuring environmental attitudes; however, direct, self-reporting techniques, such as scales and inventories, are the most widely used (Milfont & Duckit, 2009). While many instruments designed to measure environmental attitudes are study specific, there are three widely used instruments (Dunlap & Jones, 2003): a) The Ecology Scale (Maloney & Ward, 1973), b) The Environmental Concern Scale (Weigel & Weigel, 1978), and c) The New Environmental Paradigm (NEP) (Dunlap & Van Liere, 1978). One of the significant challenges of assessing changes in children’s environmental attitudes is finding an age-appropriate instrument. Manoli, Johnson & Dunlap (2007) modified the NEP, originally designed for adult populations, for use with children ages 10-12. After interviewing fifth-grade students, the authors found that reducing the number of items on the NEP to 10 from 15 and revising the wording made it appropriate for use with upper elementary students. (Manoli et al., 2007). However, instruments for measuring the environmental attitudes of very young children (younger than 10 years old) are project-specific. For example, Ratcliffe (2007) created an Ecoliteracy Survey instrument that included statements about students’ ecological attitudes toward extinction, organic produce, water pollution, land conservation, littering, and energy and water conservation. Ratcliffe (2007) explains, “These eco-attitudes were identified as ‘things environmental people cared about’ and are conceptualizations of environmentally responsible behaviors found in the literature (Bunting & Cousins, 1983; Jaus, 1982)” (p. 78). The Ecoliteracy Survey was then administered to 236 sixth-grade students participating in a gardening experience (Ratcliffe, 2007). Findings from this study indicate that these students improved their ecological knowledge and performance of environmentally positive behaviors, but did not improve their environmental attitudes (Ratcliffe, 2007). Since Ratcliffe’s (2007) Ecoliteracy Survey was used to measure changes in environmental attitudes as a result of a school garden experience and was mo st-closely age-appropriate, we based our survey questions on dais instrument, but slightly modified the language and length as described in the methods section below. Sample survey items are found in Figure 2.

Another challenge in selecting and/or developing an instrument to measure environmental attitudes comes from the inherent complexity of the structure of environmental attitudes. Historically, environmental attitudes are described as containing three components: affect, beliefs, and behavior. However, contemporary studies on attitude structure demonstrate that affect, beliefs, and behaviors interact with attitudes, rather than being their constituent components of attitudes (Milfont & Duckit, 2009). Accordingly, we treat environmental attitudes as a singularcomponent and did not measure behavior. With the limitations of study specificity, age appropriateness, and the inherent complexity of environmental attitudes in mind, we leveraged research on studies of young peoples’ environmental attitudes and the garden learning literature.

Fancovicova & Prokop (2011) conducted a study on 34 Slovakian fifth-grade students who participated in an outdoor education program. Comparing pre/post measures from 17 treatment and 17 control students, they found students’ attitudes toward plants, as measured by the Plant Attitude Questionnaire (PAQ), shifted in a positive direction in the treatment group. Similarly, Carrier (2009) conducted a quasi-experimental pre/post study with four- fourth- and fifth-grade classes who participated in a 14-week environmental education program in a southeastern state in the U.S. The treatment group participated in outdoor activities while the comparison group participated in activities in their classrooms. Using the Children’s Attitudes Toward the Environment Scale (Musser & Malleus, 1994) they found interesting differences between boys and girls where boys increased their environmental attitudes when they participated in outdoor activities, while girls scored the same in both settings. Smith-Sebasto & Cavern (2006) measured the impact of adding pre/post in-class activities to a three-day environmental education program for 169 seventh-grade students. The Environmental Adaptation Environmental Trust and Pastoralism Subscales of the Children’s Environmental Response Inventory (Bunting & Cousins, 1983) were used to measure changes in environmental attitudes. Students who received both pre- and post-activities had statistically significant gains on the Environmental Adaptation Subscale.

Only two research studies investigated environmental attitude change in conjunction with school gardens, both employing the Project GREEN (Garden Resources for Environmental Education Now) curriculum. Project GREEN is a program that uses a garden to teach about the environment and sustainability (Skelly & Zajicek, 1998). Skelly & Zajicek(1998) surveyed second- and fourth-grade students (n=153) from four elementary schools in Texas who participated in the garden program and compared them to a control group (n=84) that did not participate in the garden program. Using the Children’s Environmental Response Inventory, Skelly & Zajicek(1998) found garden program students demonstrated more positive environmental attitudes. For example, they noted higher scores in pastoralism, or “enjoyment of the natural environment in an intellectual and aesthetic fashion,” than those students without the garden experience (Skelly & Zajicek, 1998, p. 579). Similarly, Waliczek and Zajicek (1999), who studied 589 second- through eighth-grade students from seven schools in Texas and Kansas, found that environmental attitudes changed in a positive direction on a project-specific environmental attitudes scale called The School Garden Program Environmental Attitude Inventory after experiencing Project GREEN gardening activities. According to these studies, student attitudes toward the environment shift in a positive direction when they participate in garden-based curriculum experiences. In contrast, Blair’s (2009) review of the U.S. research literature on school gardening outcomes found that student environmental attitudes do not consistently improve with gardening.

School Gardens

In the U.S., over 3,000 school gardens are being used for educational purposes and as a tool to get students outdoors (National Gardening Association, 2010). School gardens also are gaining momentum internationally, as evidenced by organizations, such as British-based Royal Horticultural Society Campaign for School Gardening and the increasing popularity of kitchen gardens in Australia (Public Schools NSW, 2013). Several research studies have shown that school gardens provide a variety of environmental stewardship opportunities (Alexander, North, & Hendren, 1995; Blair, 2009; Brunotts, 1998; Brynjegard, 2001; Canaris, 1995; Faddegon, 2005; Moore, 1995; Thorp & Townsend, 2001).

School-based instructional strategies that use a garden of some kind as a teaching tool are often referred to as garden-based learning. Beginning in the mid-1990s, a number of researchers started to explore the effects of school garden programs. Most of the studies that have been conducted have been in the area of nutrition education and have been small-scale and quantitative in methodology. .Evidence shows that school gardens improve nutritional habits by encouraging children to eat more vegetables (Lineburger & Zajiceck, 2000; Nanney, Johnson, Elliot, & Haire-Joshu, 2006; Public Schools NSW, 2013). A small number of studies explored how school gardens affect children’s’ environmental attitudes (Skelly & Zajicek, 1998; Waliczek & Zajicek, 1999) and social and emotional growth (Desmond, Grieshop, & Subramaniam, 2002; Waliczek, Bradley, Lineberger, & Zajicek, 2000). In addition, four studies looked at learning by evaluating specific garden-based curricula and academic achievement by students in science (Dirks & Orvis, 2005; Klemmer, Waliczek & Zajicek, 2005a & b; Smith & Mostenbocker, 2005). A study
conducted in die United Kingdom found diat students participating in gardening at school were better at problem solving and displayed more independence (National Foundation for Educational Research, 2011).

In spite of interest in exploring school garden programs, their impacts are still poorly understood. As we argue elsewhere (Fisher-Maltese & Zimmerman, 2014), we believe existing studies on garden-based programs approach learning through a school-based perspective, that is, these other studies focus mainly on content learning. We believe this perspective on learning is unduly narrow and limits the conclusions that can be drawn about the benefits of these settings. In dais paper, we present evidence of shifts in environmental attitudes of students participating in a garden-based learning program; we argue that student attitudes are a component often associated with informal learning contexts.

Informal Science

Our study comes at a time when there is growing interest in understanding how people learn science in informal settings. In die spring of 1999, die board of die National Association of Research in Science Teaching (NARST) established an Ad Hoc committee focused on out-of-school science education. The consensus policy statement, issued after two years of collaboration, recognized a broad view of learning which states “much of what people come to know about die world, including the world of science content and process, derives from real-world experiences within a diversity of appropriate physical and social contexts, motivated by an intrinsic desire to learn” (Dierking, Falk, Rennie, Anderson & Ellenbogen, 2003, p. 109). This perspective directly connects to die categories of learning we documented in a school garden study. There is a growing body of research demonstrating that learning in informal settings is socioculturally mediated, derived from real-world experiences in an authentic setting, self motivated and guided by learners needs and interests, voluntary, and lifelong (Ash, 2003; Ash, Crain, Brandt, Eoomis, Wheaton, & Bennett, 2007; Bell, Eewenstein, Shouse, & Fedder, 2009; Borun, Chambers, & Cleghorn, 1996; Dierking et al., 2003; Eberbach & Crowley, 2005; Rowe, 2002).

In 2009, die National Research Council (NRC) released Teaming science in informal environments: People, places, and pursuits (Bell, et al., 2009) in which learning in informal science contexts is described as “learner-motivated, guided by learner interests, voluntary, personal, ongoing, contextually relevant, collaborative, nonlinear, and open-ended” (p. 11). From these documents, and die associated research literature, we developed a framework for what we call an informal learning lens (see Table 1) (Fisher-Maltese & Zimmerman, 2014). Although we acknowledge that this framework is a work in progress, it serves as a guide for our analysis of learning that occurs in a school garden. We argue elsewhere (Fisher-Maltese & Zimmerman, 2014) that in order to understand learning in school gardens, researchers should adopt a broader view of learning from an informal learning perspective. This paper further expands this argument to incorporate shifts in students’ attitudes toward die environment associated with participation in school garden programs. Of particular importance for this paper, we will be focusing on “life-long” learning (highlighted in blue) as an integral component of informal learning. According to Dierking et al.’s (2003) definition, life-long learning encompasses science knowledge and attitudes that are “cumulative and emerging over time through myriad experiences” (p. 109). As we will discuss in the recommendations section, an informal learning perspective points to life-long methods (e.g., longitudinal studies) as an effective means to measure changes in attitude in dais context.

METHODS

This paper is part of a larger case study of a garden-based, science curriculum on insects, which uses a school garden as an informal learning setting.

Study Context

This study took place in four second-grade classrooms within a K-3 elementary school, located in an affluent, predominantly White (60%) and Asian (40%) school district in central New Jersey. Sixty-six second graders participated in the study, along with four teachers, and one principal (n = 71).

The school garden at this school consists of four large and two small raised beds surrounded by mulched paths and a deer- and rodent-proof fence. Teachers, students, and parents grow vegetables (e.g., peas, tomatoes, carrots, etc.), herbs (e.g., basil, dill), fruit (e.g., blueberries, strawberries), and flowers (e.g., zinnias, marigolds, cosmos) and maintain the garden. The fence is lined with an internal and external border of perennial plants. One section of the border contains perennial plants that are food sources for local butterflies. The garden is located on the school’s property, although a distance from the building and across a parking lot. Students primarily use the garden during class time accompanied by a teacher. Teachers use the school garden at every grade level (I<-3) at Penn Valley Elementary School (a pseudonym).

Table 1. Informal Learning Lens

Garden-based Curriculum and Framework

The second-grade science curriculum at Penn Valley Elementary School includes a unit on insects during the spring. Typically specimens are ordered from a science supply company and raised in the classroom to demonstrate their life cycle changes. Painted lady butterflies are the most common insect observed in classrooms at the school. Teachers from Penn Valley also chose to study ladybugs and praying mantises since they are beneficial to the garden and served as a practical means to connect the insect curriculum to the school garden. However, ladybugs pose a unique challenge to observing the different phases of the life cycle since most science supply companies typically ship adults, because larva are fragile and tend to die during transport.

Following a со-design approach (Penuel, Roschelle, & Shechtman, 2007), we developed a four-week standards- based science curriculum on insects collaboratively with the four participating teachers, utilizing the school garden as an informal learning context. The students participated in classroom and garden insect lessons every day during the curriculum. The lead author facilitated lessons by supporting the teachers and co-teaching the lessons in the school garden. Lessons were focused around week-long themes including anatomy, life cycles, helpful and harmful insects, butterfly and larva identification, and designing a butterfly garden (see Table 2). While preliminary work (Fisher-Maltese & Zimmerman, 2014) did point to the possibility of shifts in students’ environmental attitudes, we did not design the curriculum with this content or pedagogical goal.

The curriculum was designed using a framework called Learning Across Contexts (LAC) (Zimmerman, 2005, 2010, 2011), a curriculum design framework that addresses the need to capture evidence of learning across the gaps between informal and formal science learning settings (Zimmerman, 2005, 2011). A goal of the curriculum was to promote opportunities for students to connect and reinforce concepts learned in the garden to those learned in the classroom. Drawing from the research on informal learning, LAC involves a three-phase pedagogical model: a) previsit preparatory activities, b) activities and tasks during the field trip (or visit to the school garden), and c) post-visit reflection activities (Zimmerman, 2005, 2010). The LAC model was enacted through the garden curriculum when students learned important terminology and content information before an experience in the school garden, then went out to the garden to see real-life examples of what they were studying in the classroom, and, last, engaged in post-experience reflection activities, such as writing in a science journal or completing one of four written assignments (e.g., making an insect life cycle timeline). According to the LAC framework, the informal learning experience, in this case lessons in the school garden, is viewed as an integral part of the curriculum, instead of a supplementary or disconnected activity (Zimmerman, 2005, 2010).

Table 2. Curriculum Overview__________________________________

Lesson 1: Using die 5 senses to observe and explore die school garden

Role of the researcher. Over die course of die study, the role of die principal investigator, the lead author, ductuated between direct observer and participant observer (Creswell, 2007). She was previously a second-grade teacher at this school and, six years ago, led the initiative to plant the school garden used in this study. Due to die close connections between die principal investigator and the teachers, school garden, and garden-based curriculum die teachers were implementing, we exercised caution to avoid bias. For example, die second author served as an objective reviewer of die curriculum, instruments, and data and conducted inter-rater reliability during data analysis. We also made a conscious effort to look for contradictory evidence. While bias could still conceivably limit our work, personal connections to die study setting and participants facilitated access to the school, and background knowledge, which otherwise would not have been possible. This access helped us understand die participants’ perspectives in important ways. Lessons were facilitated by modeling for die teachers how to teach die content in die garden and manage their students in an outdoor learning setting. Throughout die study, we encouraged the teachers to take die lead so we could remain in die role of observer.

DATA SOURCES

Though we collected many forms of data, for this paper, we will only describe those data sources relevant to our discussion of shifts in students’ attitudes toward the environment. Over the course of the four-week curriculum, we collected several forms of data related to attitudinal shifts by students. Relevant complementary data sources included: a) pre/post tests, b) pre/post environmental attitude surveys, c) interviews, and d) student conversations in the garden.

Pre/post tests. We administered pre/post tests to assess science content knowledge and student attitudes toward the environment. Pre-tests were administered the same week the curriculum was initiated and post-tests within one week of curriculum completion. Pre/post tests included multiple choice and open-ended questions designed to elicit students' understanding of insect anatomy, life cycles, behavior, habitats, and attitudes toward insects and habitat loss.

Pre/post surveys. To capture shifts in students’ environmental attitudes over the course of the curriculum, we used a pre-existing survey instrument designed by Ratcliffe (2007). We modified the instrument by reducing the number of items and simplifying the language (Ratcliffe’s instrument was designed for use with sixth-grade students while we worked with second-grade students). In our abbreviated version of Ratcliffe’s (2007) Ecoliteracy Survey, there were a total of seven attitudinal statements, which included a 5-point Eikert scale (e.g., 1 = strongly agree, 5 = strongly disagree). For example, one statement from the survey was, “Trying to protect the environment is my responsibility,” with response options ranging from “agree” to “disagree” across a 5 point Eikert scale. Another statement was, “I think people should build more parks for animals.”'For all but two of the statements (2 and 8), a 1, or strongly agree, was the most desirable response. For example, statement 1 read “I am worried about animals that are going extinct.” For statements 2 and 8, the inverse was the most desirable response so the responses were re-coded for consistency (i.e., a 1 became a 5, a 2 became a 4, etc.).

Interviews. We conducted semi-structured pre-/post-curriculum interviews with four students in each classroom (total = 16). Interview questions included the following: “Do you think it’s important to protect where insects live? If yes, why? How can you protect where insects live? Is there anything you can do?” Interviews were audio recorded and videotaped for accuracy and later transcribed.

Student conversations. We digitally audio-recorded student conversations during lessons in die school garden to capture in situ learning and attitudinal shifts. We placed small digital recorders in students’ pockets and lapel microphones on their shirts to capture their conversations.

DATA ANALYSIS

Data analysis followed a multi-step process; quantitative and qualitative data were analyzed separately and then examined for triangulation purposes.

Pre/post tests. Pre/post tests primarily assessed science content knowledge, but also contained one question which measured attitudes toward the environment. Pre/post test data were analyzed using a rubric developed by the researchers. Inter-rater reliability was conducted and yielded 94% reliability. Paired sample t-tests were conducted using the statistical software, SPSS, on the pre/post tests.

Pre/post surveys. Students’ responses to our survey instrument were entered into an Excel spreadsheet. Responses were then added together to create an index (Index A = pre-test, Index В = post-test). Indices provided a general measure of environmental attitudes over time (i.e., from pre- to post-test). Statistical analysis involved paired sample t-tests using SPSS.

Interviews and student conversations. Interview and student conversation data were first transcribed and organized by data source. The second step involved describing the data set with several rounds of coding that were created deductively from the research question guiding the study: environmental attitudes. For subsequent passes of data coding, subcodes were created both deductively from the literature and inductively from the data, following recommended qualitative data analysis protocols (Creswell, 2007). Table 3 describes the coding scheme we used, including the code, criteria, and examples. Codes for environmental attitudes included “protect habitat,” “fear of insects,” and “desire to protect insects/compassion towards.”

RESULTS

Several forms of data were used to assess whether students’ attitudes toward the environment changed throughout their use of the school garden. In this section, the following results are discussed: responses to a specific question on the pre/post test, pre/post environmental attitudes survey, interviews, and student conversations in the garden.

Quantitative Results

Pre/post test. Of the 88 students who took the pre-test, 63 also took the post-test. Therefore, a total of 63 paired pre/post tests were collected. Only one question assessed students’ environmental attitudes on the pre/post test. Question 13 read: “Is there anything you can do to protect where butterflies live? Do you think this is important? If you do, why?” However, this one question was separated into the separate sections, each coded independently. For the first part of this question, among answers coded as “correct” were responses such as plant seedlings for nectar plants (i.e., those with flowers from which butterflies obtain nectar), don’t pull important plants thought to be weeds, and don't harm habitats. These responses also can be coded as pro-environmental responses and thus relate to students’ attitudes toward the environment. If students provided some “other” response, it was considered incorrect. While many students answered, “I don’t know” (n = 53) to question 13 on the pre-test, posttest answers included a variety of responses. Many students had ideas for things they could do to protect where butterflies live (question 13: n = 36 answered “1” for a positive behavior), such as “plant food for the butterflies to eat” and “ask my parents to stop spraying our lawn [with pesticides].” On this part of the question, students’ pro-

Table 3. Coding Table

Table 4. Responses to “Do you think it is important to protect where butterflies live?”

Table 5. Responses to “If you do [think it’s important to protect where butterflies live], why?”

environmental responses increased by 32% and the number of student having no opinion decreased by 17% (see Table 4)

For the third part of the question, 36% more students provided a “good reason” for why it is important to protect butterflies. Good reasons included: “butterflies are helpful insects because they pollinate flowers,” ‘Ъеір plants grow,” and “are living things.” (“Not a good reason” usually was an unrelated response, e.g., “butterflies have three body parts,” “butterflies are different colors.”) 21% fewer children had no opinion on the post-test compared to the pre-test (see Table 5).

Pre/Post environmental attitudes survey. Sixty-three students completed both the pre- and post-survey; only these repeated measures were analyzed. Analysis of these pre/post surveys did not result in a statistically significant pre-post change (Index A (pre) M — 17.84, SD — 4.43; Index В (post) M — 17.81, SD — 4.86, a lower number indicates a better score; paired t-test yielded t(63) = 0.076, p = .94).

QUALITATIVE RESULTS

Interviews. Sixteen students (four in each of the second-grade classes) were interviewed before and after the curriculum. Pre/post curriculum student interviews included the questions, “Do you think it’s important to protect where insects live? If yes, why? How can you protect where insects live? Is there anything you can do?” In total, 6 out of 16 students’ interview responses showed a positive shift in environmental attitudes from pre to post curriculum (see Table 6); the other 10 students had a positive attitude toward the environment at the start of the curriculum which remained positive at the end of the curriculum (i.e., there was no change in their attitudes toward the environment).

Table 6 demonstrates the six students’ shifts in attitudes from pre to post curriculum. In summary, Pamela¹All names are pseudonyms and Margaret exemplify students who had a complete attitude change. They changed their attitude from “no, you should not protect where insects live” in the pre-curriculum interview to “yes, because some insects are actually helpful, and not all are harmful.” Similarly, Pamela, Carson, Margaret, and Noah seemed to regard insects favorably in the post interview because some insects are helpful. Isaac and Noah did not change their opinion that insects’ habitats should be protected, but their reasoning in the post-interview was much more sophisticated. During the preinterview, both explained that you should protect insects because you do not want more animals added to the endangered animals list. However, in the post-interview, Isaac explained how people are responsible for the butterflies dying due to spraying pesticides and habitat destruction and Noah explained how insects are important for pollination. Carson and Kyle explained that you should protect where insects live because insects will hurt you if you don’t protect their habitat. In the post-curriculum interview, Carson expressed that you should protect the insects’ habitats that are helpful. Kyle seems to have developed some compassion towards insects. He thinks he should help them since they need a habitat to survive.

At least four students communicated a fear of insects in the pre-interview. Clearly, students had either been taught or learned through personal experience that insects are frightening. For example, Darren explained in an interview, “I don’t like insects. Eike I can draw an insect, but when people talk about them a lot, I start to shiver and then I feel like I have bugs and insects crawling on me.” Darren refused to touch the plastic creatures we asked him to sort into two groups during the interview: insects and non-insects. He felt more comfortable pointing as we moved them for him into two different piles. Interestingly, Darren seemed to overcome or forget about his fear during the lesson in the garden which involved catching insects with tweezers and nets and observing them in bug boxes. In the audio-recorded conversation between him and his partner, Darren does not once express fear and seems engaged in the activity.

Student conversations in the garden. Student conversation data also provided support that students had a positive shift in attitude toward the environment. Students’ comments fell into two categories: expressing concern for insects and wanting to protect them and expressing excitement about catching insects as part of the curriculum (see Table 7).

DISCUSSION

Findings from our study are in line with conclusions from Blair’s (2009) review that found students’ environmental attitudes do not consistently improve with gardening. We attempted to use triangulation to corroborate our results, but instead we found interesting differences between the survey results and the pre/post test, interview, and student conversation data. The quantitative survey data for this study show no statistically significant shifts in attitudes. However, in contrast to the survey data, data from the pre/post test, interviews, and student conversations suggest an improvement in students’ attitudes toward a more empathic view of nature, specifically insects. Students’ changing their opinion of insects as a result of studying them is not unique to this study. For example, Ratcliffe (2007) found that teachers from her study reported that students became “more insect friendly” and that “not all kids want to make their hands dirty, but.. .they got used to it and [then].. .they wanted to touch the worms and insects ” (Ratcliffe, 2007, p. 80). In addition, our pre/post test, interview, and student conversation data are in-line with other research studies that show positive shifts in environmental attitudes for students as a result of outdoor education programs generally (Carrier, 2009; Fancovicova & Prokop, 2011; Farmer, Knapp & Benton, 2007) and experiences in school gardens, in particular (Skelly & Zajicek, 1998; Waliczek & Zajicek, 1999).

This inconsistency in our results led us to question the survey tool we used and its reliability for this population and for this context. Many environmental attitude research instruments are study specific. Therefore, there is no standard for measuring environmental attitudes, making it difficult to generalize across studies in this field (Milfont & Duckit, 2009). In addition, there are challenges to developing a good instrument for detecting changes in environmental attitudes (Johnson & Manoli, 2010) including the complexity and multidimensional structure of environmental attitudes (Milfont & Duckit, 2009) so we sought out an existing tool that would serve our needs. During our search for a valid, reliable instrument for measuring environmental attitudes, we found only three instruments that had strong measures. However, only one of these instruments had been modified for research with children (Manoli, Johnson & Dunlap, 2007). We ultimately chose the only instrument we could find that was developed for students participating in a gardening activity. We believe we saw no change from pre- to post-survey because of the limitations of the instrument and a possible ceiling effect. In hindsight, the questions from the survey we used were too general and did not match the specific curriculum content. For example, our students’ shifts in environmental attitudes were often about insects specifically. Data suggest that perhaps another tool would have resulted in quantitative pre-post changes. For instance, a scale that included fear toward nature (or specifically insects) would have captured changes in the students’ environmental attitudes. In addition, similar to interview responses in Mittelstaedt et al.’s (1999) study, a majority of our students (10 out of 16) began the curriculum with a positive attitude toward the environment, however, we were not able to demonstrate that they finished the curriculum with an even stronger environmental attitude. This indicates a possible ceiling effect, where the items on the instrument limit the possible answer choices in a way that constrains possible higher measures.

RECOMMENDATIONS

We conclude with three recommendations for how the field should moving forward with future work: 1) the use of mixed methods, 2) better survey tools to measure attitudinal shifts with this population in this context, and 3) a longitudinal approach.

First, we recommend the use of mixed methods because they reveal possible drawbacks associated with a single method approach. While several of our data sources (pre/post tests, interviews, and student conversations) indicate positive shifts for students toward the environment, the survey data show no statistically significant changes in attitude. There are advantages to doth qualitative and quantitative methods being used; dual methodologies add to the richness of data analysis (Firestone, 1987; Fraser & Tobin, 1993; Orion & Hoftstein, 1994; Sieber, 1973). Fraser and Tobin (1992) explain the rationale for a combined method:

• ...the complexity of qualitative observational data and quantitative data added to the richness of the data base as a whole...Through triangulation of quantitative data and qualitative information, greater credibility could be placed in findings because they emerged consistently from data obtained using a range of different data collection methods (p. 290)

In the case of this study, triangulation of the data show positive shifts in attitude that would have been missed if quantitative methods were used in exclusion. Second, we think the field should proceed with better measures in this context. Surveys that are age-appropriate and specifically match curriculum content would potentially capture pre/post changes and therefore yield more reliable results.

Third, given the challenges associated with measuring shifts in environmental attitudes, we encourage other researchers to explore the use of informal learning perspectives as a framework for future garden-based research. For example, one way to solve these challenges is to take a life-long learning approach. Dierking et al. (2003) explain that shifts in student attitudes are a result of indicators of life-long learning.

Rather, learning, in general, and science learning in particular, is cumulative, emerging over time through myriad experiences, including but not limited to experiences in museums and schools.. .The experiences children and adults have in these various situations dynamically interact to influence the ways individuals construct scientific knowledge, attitudes (italics added), behaviors, and understanding, (p. 109)

We need to consider life-long methods (i.e., longitudinal studies), longitudinal studies might allow researchers to disentangle the messiness of data on this hard-to-measure construct. Longer term or repeated experiences with gardens, and more free-choice elements to garden curricula could potentially reveal changes in attitude.

References