Introduction

Although the history of the environmental education dates back to Stockholm Conference held in 1972, the concept of sustainable development got attention and came into the focus with the publication of Brundtland Report (Brundtland, 1987), and then Education for Sustainability (EfS) was proposed in 1997 by means of “Thessaloniki Decleration” (UNESCO, 1997a), as a carrier of the common and single message of hope for the future (Knapp, 1997). The message of Thessaloniki was that education, first and foremost, should be in the center of the international, regional and national agendas (UNESCO, 1997b) and it is the foundation for all other “pillars” of sustainability, namely, regulation, innovation and economy. Most recently, UNESCO proclaimed the Decade of Education for Sustainable Development (UNDESD) for the period of 2005-2014 which emphasize the importance of education for achieving sustainable development and integrating sustainable development into education system at all levels (UNESCO, 2005).

Today, educators face a compelling responsibility to serve society by fostering the transformations needed to set us on the path to sustainable development. The time has come to ensure that the concepts of education for sustainability - in the broadest sense - are discussed and woven into a framework upon which current and future educational policy (de Rebello, D., 2003; Wright, 2002) is based. Moreover, achieving a sustainable future will not happen unless our educational system trains citizens and specialists who understand the interconnections among the environmental, economic, and social disciplines. EfS, therefore, provides an opportunity to craft the future we want for a sustainable world. Higher Education plays a profound and pivotal role in making this vision of a sustainable future a reality (de Ciurana & Filho, 2006). It prepares most of the professionals who develop, lead, manage, teach, work in and influence society's institutions. In addition to training future teachers, higher education strongly influences the learning framework of K-12 education. Higher Education plays a critical role in creating and disseminating the knowledge, skills and values for society (Ciegis & Gineitiene, 2006). It has also unique academic freedom and the critical mass and diversity of skill to develop new ideas; to comment on society and its challenges; and to engage in bold experimentation in sustainable living.

Awareness of shared needs and common ground is the first step for EfS. New approaches to learning may offer significant benefits. New approaches will be more readily accepted if the benefits of teaching EfS are understood. Up to now, educators have identified a number of obstacles that are impeding the integration of information about the environment and sustainability in formal learning settings (Moore, 2005). One obstacle is that the interdisciplinary content of EfS does not easily fit into a discipline-oriented educational process (McKeown, 2002). Other obstacles are the lack of general agreement among professional educators that EfS is a priority and there is insufficient professional preparation for teaching the core content of sustainability issues (Hopkins & McKown, 2001). Until recently, there was a lack of consensus on an effective system for evaluating programs and materials in order to ensure quality. As stated in Agenda 21, the document produced by the 1992 United Nations Conference on the Environment and Development, education is critical for promoting sustainable development (United Nations, 1992). Understanding the principles of sustainability and the interdependence of the environment, the economy, and social systems can help us learn to make the changes necessary to become effective stewards of natural resources and the environment.

The convergence of a number of trends and events in recent years suggests that a unique opportunity to advance education for sustainability has arrived. On the one hand, the field is benefiting from increased attention from professional societies, continued surges of public concern over local and national and international environmental issues and ongoing engagement by nongovernmental organizations. Besides, when they graduate, the students would be able to bring their knowledge, skills and values of sustainability to their future employment, consumption decisions, lifestyle choices, and to the improvement of communities in which they live.

This article reported and discussed an evaluation of a lecture, designed to change university students’ values related with sustainable ways of living. Outcomes of the study were those obtained from the evaluation of the course titled “Education and Awareness for Sustainability”, which has been given in the Middle East Technical University, Department of Elementary Education for 6 semesters. Therefore, the educational problem we set out to address is to improve the course schedule by means of referring the needs and pertinent problems as expressed by the students.

The following research questions were addressed accordingly:

1. What are the expectations of university students as far as a course related with “sustainability” is concerned?


2. How do students’ needs meet with the course outline and applications?


3. What are the means to improve the course to meet both with the students need and the target of the course?

Methodology

This study was designed as a course evaluation study which consisted of (1) needs assessment, (2) formative evaluation and (3) summative evaluation respectively. Both quantitative and qualitative research methods were made use of throughout the study.

The Course

The course is titled “Education and awareness for sustainability (EAfS)” and its basic concept has been originated from the light of the idea that developing environmental awareness of a society requires the formation of an “environmentally-sensitive” culture. At the beginning, the course was offered only to students in the Faculty of Education and after two semesters it was offered to all students of the university, because of the interdisciplinary feature of the subject. The lecturer has an environmental engineering background and has been giving environmental education related courses in the Department of Elementary Education for three years. The target of the course is to influence environmental attitudes and practices of students towards more environmentally - sustainable ways of living. The goals seek to help students to understand how daily life and work can be adapted to improve the environment; to acquire awareness and sensitivity to the total environment; to acquire social values, strong feelings of concern for the environment and motivation for actively participating in its protection and improvement; to acquire a personal view of general and global environmental issue and to ensure that students understand that they are part of the natural circle. The course content was designed by the lecturer and rearranged at the beginning of each semester based upon feedbacks provided by students.

Sample of the Study

The participants of the study were the ones who enrolled in the course of Education and Awareness for Sustainability. The number of the students participating in the study was 85 (26 male, 59 female). The course was given as two groups (sections); the number of the students in the first group was 50 and that in the second group were 35. 68 of the students were from the Faculty of Education, 4 of the students from Faculty of Economy & Administration and 13 from Faculty of Engineering. Table 1 summarizes background information about number of students according to group, gender, faculties and classes they enrolled.

Data Collection Instruments

Various data collection instruments were developed by the researchers and then used for different purposes in different time. These instruments and their descriptions are given below.

Need Assessment Questionnaire (NAQ); NAQ was developed to investigate undergraduate students’ needs, expectations and pre-knowledge about the course of EAfS. It consisted of three parts, eight sub-parts and total 53 items. 17 of the items were open-ended whereas 36 of them were closed ended on a Likert type scale. These parts addressed to (1) background information, (2) course implementation with regard to instructor, student, instructional methods and techniques, instructional materials, and evaluation techniques, and (3) students’ attitude toward the course.

Observation Schedule for Formative Evaluation (OSFE); OFSE was developed to observe classroom context and course implementation (instruction) during the semester. OFSE consisted of the items pertaining to course objectives, course content, teacher and student roles, instructional methods and techniques, materials and equipment, evaluation procedure, interdisciplinary feature of the course and real life experiences integrated into the course implementation.

Open-ended Questionnaire for Formative Evaluation (OQFEf OQFE was developed to get students’ opinions regarding course implementation. It included only seven open-ended questions. The participants were required to respond to these open-ended questions by considering their initial needs and implementation of the course. With this questionnaire, it was also aimed to determine the students’ opinions about the field trip, which was organized depending on the results of the NA.

Summative Evaluation Questionnaire (SEQ); SEQ was developed in line with the results obtained from the former steps (Need assessment and Formative evaluation). SEQ was designed to determine whether students’ expectations and need were satisfied or not. SEQ consisted of three parts, each addressing to (1) background information and general opinions about the course, (2) contributions of the course to professional and social life, and (3) general evaluation of the course.

Tablel. Students’ profile

Reliability and Validity of the Instruments

Reliability and validity were assured by obtaining various evidences from different sources for each of the instruments. For content validity of the instruments, two experts were invited to the study for consulting their opinions. One of the experts specialized on curriculum evaluation and the other expert specialized on education for sustainability. The experts were required to assess the instruments with regard to their content coverage and their consistency with the aims of the present study. They believed that the items in the instruments were in line with the aims. But, they suggested revisions of some of the items (i.e. wording). Based on their suggestions, the instruments were re-designed for the administration.

For assuring the reliability of the instruments, the Cronbach’s alpha reliability coefficient (CX) were computed by means of reliability analysis in SPSS version 11.5 for only NAQ and SEQ. Since both of these instruments included several parts, statistical reliability analysis was only applied to Likert type items. Cronbach’s alpha reliability coefficient (ar) of this part of the NAQ was found .77 and of the SEQ was found .94. Moreover, one of the observations was conducted through the use of OSFE by three observers for the purpose of assuring the interrater reliability among the observers/coders. The codes emerging from each of these observations were almost consistent. As for the reliability of OQSE, the obtained data through OQSE were firstly coded, and then the codes were re-checked by another researcher. The codes emerging from first coding were confirmed in the second check. Based on all of these evidences, it is possible to state that the data collected through different instruments were complementary to one another and that the triangulation of the data was ensured.

Data Collection Procedure

The study was carried out during the spring semester of 2004-2005 academic year. It was designed to have three stages: (1) need assessment, (2) formative evaluation and (3) summative evaluation. Different data collection instruments were used in each step in order to triangulate the findings. Furthermore, the DIPO model developed by the authors depending on these three steps was utilized for evaluating the selected course. The model shown in Figure 1 mainly emphasizes the needs-objectives relationships and the use of qualitative procedures seeking for in-depth information. Throughout the evaluation study, the standards (utility, feasibility, propriety, and accuracy) suggested by Joint Committee on Standards for Educational Evaluation (1994) were strictly considered for effective evaluation. The major considerations of the model are; determining students’ needs (and/or expectations, priorities), adapting the program to the emerged needs and detecting if these needs are matched with the program objectives. The stages of DIPO are parallel to those of the CIPP model (Stufflebeam, 2003). Figure 2 illustrates the steps and further sub-steps followed for evaluating the course.

NA was initially conducted with three main sub-steps: (1) obtaining background information about the course and students (preparation or planning step), (2) developing and applying an instrument (development and implementation step) and (3) analyzing and reporting the data collected (reporting step). In the first sub-step of NA, NAQ was developed depending on the informal interviews realized with the lecturer and several former students of the course. In the second sub-step of NA, the questionnaire was administered at the beginning of the semester so as to reveal the needs, expectations and priorities of the students in terms of course objectives, instructional materials and medium, teaching methods and evaluation procedures. In the last sub-step of NA, a report including emerged needs and problems of students was prepared in order to be ready for second step which was FE.

Step 2. Formative Evaluation (FE)

The major goals of this step were to (1) determine the strengths and weaknesses of the course, (2) observe whether the instructor are integrating students’ needs into course implementation process, and (3) suggest ways for improvements and for redesigning the course, if it is needed. This step was performed during the instructions in line with the emerged needs in the NA step. Likewise to the NA step, three sub-steps were followed throughout FE; (1) developing data collection instrument(s) based on the NA results, (2) administering instruments and (3) analyzing and reporting the data. In the first sub-step, two different, but related, instruments were designed depending on the themes emerged in the NA. These instruments were open-ended observation schedule and open-ended questionnaire. In the second sub-step, the classroom was observed five times by making use of the observation sheet. Four of them were conducted in the class environment and one was during a field trip. In order to ensure the inter-rater reliability among the coders, three PhD students observed the classroom through making use of the observation sheet. Most of the emerged codes seemed to be similar or same. Furthermore, the open-ended questionnaire was applied to seven students, each of whom was selected from different departments. In the last sub-step of the FE, collected data was classified and then subjected to content analysis. A report including the FE and its steps was prepared and were shared with the instructor so that she could easily consider them, and re-design and make necessary changes in the course implementation accordingly.

Step 3. Summative Evaluation (SE)

The aim of this step of evaluation was to get the overall picture of the quality of the course. As well as other main steps, the SE also included three sub-steps which were (1) developing SE instrument, (2) administering the instrument and (3) preparing final report.

In the first step of SE, a questionnaire including both open and close-ended questions was developed. In the second sub-step of SE, the instrument was administered to 70 students (25 male, 45 female) in the classroom environment. Although there were 85 students in the NA step, 15 students did not take place in this sub-step because of several reasons, such as nonattendance and unwillingness. Finally, after going through all these steps, a final report was prepared and then the results were presented and discussed with the students in the classroom.

Data Analysis

Numerical data were analyzed by means of descriptive statistics (means, percentages...etc.). On the other hand, the data collected from open ended questions were subjected to content analysis (Patton, 1987, 1990). The codes that emerged in the content analysis constituted the themes discussed in the result part.

Results

Needs Assessment

One of the first open ended items of the Need Assessment Questionnaire (NAQ) was about the meaning of and relation among the concepts: education, awareness, and sustainability. Although there were a few misconceptions, particularly about the term awareness, most of the students satisfactorily defined the terms and set out a relationship indicating that teaching sustainability is a way to make people aware of the environmental problems and this can be realized by education. On the other hand, when students were asked about their expectations
of the course, it was not possible to make a one general comment out of the 85 answers. Expectations were scattered especially according to their subjects. Some were interested in learning the balance between production - natural sources - consumption and environment whereas others were interested in developing active responsible behavior. Still, some were interested in how to teach environmental issues to their students. One of the students for example stated, “it is hard for me to realize how our daily activities disturb the environment, so my expectation is to understand this”. Another replied the question in the following way: “I would like to learn how I disturb the environment and when and how should I stop this?”. Moreover, some of the students’ expectations were more profession related: “I would like to learn how environmental impact assessment procedure affects investment decisions?”, or “How could I integrate environmental issues into my professional life?”. As an overall result, it can be concluded from the responses given to this item only that students were very interested in the environmental problems, sustainability and taking responsibility for helping to achieve sustainable development. One of the items that most of the students gave a similar answer was about the necessity of a field trip for such a course. Almost all of the students agreed that field trips were essential for such a course because they stated, “an environment related course can be learned best in the natural environment”. Upon the item about the instructor’s role in the class, students stated such characteristics as democratic, motivator, informer, sometimes active, sometimes passive, presenter, intellectual, sensitive, illuminating, real, cheerful and environmentalist. When the students were asked about their role in the classroom, on the other hand, they defined themselves as being a participator, a critical thinker, a problem solver, and careful listener, sometimes active, sometimes passive, an alternative thinker and an observer.

In the Likert type scale part of the questionnaire, students were asked about their expectations related to the course in general, teaching techniques and materials. As a result of the descriptive analyses, this part revealed that students expected to be informed about the targets (90%), rules and content (81%) of the course as well as the lecturer’s expectations (79%). Based upon the students’ expectations for a field trip, a trip to the solid waste recycling centre of the university was realized. The feedback obtained from this trip, on the other hand, was discussed in the following sections.

Regarding the instructional methods used for the course, according to the frequencies obtained from the descriptive analysis, students preferred discussion (94%), teacher presentation (93%) and brain storming (80%) techniques mostly. Journal writing (35%) and tutorial (34.9) seemed to be the least preferred methods among students.

Formative Evaluation

As stated before, this step was composed of a series of classroom observations and an open ended questionnaire application. There were 8 themes in the observation sheets designed according to the NA results. Therefore, FE was realized according to these themes.

Objectives: As the NA results displayed, the course objectives stated at the beginning of the course live up to the students’ expectations. The best part of this section was that both students and the instructor put up the main target as understanding how daily life and work can be adapted to improve the environment.

Course content: The answers students gave to the question “what do you expect to learn from this course?” coincided with the pre-announced content of the course. However, although none of the students mentioned the real cases, the instructor emphasized that the most attractive part of the course, as far as the students’ expectations are concerned, was the real cases presented in the form of stories.

Teacher and student roles: Several points about a lecture hour, from the observers’ point of view, would be well-placed to give the results of the FE for this issue. In fact, one student commented as follows: “a summary was given to the students about the last topics at the beginning of each lecture and this was done by asking questions to the students and reminding them about the past subjects. ...The lecturer was always smiling and made jokes making students more comfortable. She was very flexible, but always controlled the classroom”. Thus, the teacher roles, as being a democrat, a motivator, an informer, sometimes active, sometimes passive, a presenter, an intellectual, a sensitive individual, illuminator, a realist, a cheerful individual and an environmentalist, as defined as a result of the NA, were regarded as satisfactory by the observer. As far as the student’ roles are concerned, on the other hand, as reported by three observers, students were highly interested in the course and were voluntarily attending the classes all the time. All the students seemed anxious and curious about the environmental problems and they felt responsible to protect the environment. They were enthusiastic and motivated to gain knowledge and act accordingly. They acted as a creative and reflective thinker, a problem solver, an active participator, and responsible, decision maker during the discussions. Their active participation in the field trip, which was to the recycling material storage of the university, and their sensitivity to the subject were two of the most noticeable observations of their above mentioned roles.

Instructional methods and techniques: Discussion, presentation and brain storming were the three techniques preferred by the students in the NA step. As a result of the FE step, on the other hand, it was observed that although discussion and brain storming methods were used frequently during the lectures, the main technique that the instructor used was lecturing by the use of PowerPoint presentations, always supported with photographs and sometimes with music. Questioning - answering sessions were always realized before and after presentations. Students were always encouraged to present their activities, experiences and assignments in the same way. In fact, it was reported by the observers that the field trip to the recycling centre of the university was one of the best activities realized. This was obvious from the students’ comments, as one of the students stated, “I am really impressed. I am going to separate my wastes and bring them herefrom now on”.

Materials and equipment: The mostly used medium during the lectures was the computer. PowerPoint presentations, internet connections, music and VCD sessions were all realized by means of a computer. Examples from the media, newspapers, brochures and magazines were also used as supportive materials. For example, one of the sessions which got the students’ attention involved footprint calculations by means of the related internet sites. Getting different footprints for different life styles, even for those living in the same city was the point that made students become aware of the impacts and importance of living habits on the environmental damage.

Evaluation procedure: The students' preferences for evaluation varied in terms of teacher evaluation and self evaluation. Evaluation was realized by the teacher by means of the assignments and attendance to the lectures. Since the assignments mostly contributed to student comments and experiences, a higher percentage of the evaluation was composed by the attendance scores.

Interdisciplinary feature of the course: The course, for sure, was interdisciplinary as far as both the content and the student profile were concerned. The subjects integrated into the
course content included, history, geography, sociology, mathematics, science, technology, engineering, management, health, media and literature.

Real life experiences: Apart from the field trip mentioned above, real life cases discussed during the lectures were the best examples to describe the feature of the course that the issues were supported with the real life examples which were enough to give students a vision to carry out these experiences in their professional and daily lives. A story about how people feel and act about leaving their homes, gardens and memories during the dam construction process in the north part of Turkey (Qoruh River case), for example, made students to think how dam construction may change people and environment. Or through the history of the Black Sea, they realized that a sea may have a past and a future and this impact people’s lives, culture and trade.

The major result of the FE step was that although students’ needs were satisfied mostly for the lecture, the most attractive part of the lecture turned out to be the field trip added to the syllabus upon the students’ request. Moreover, case studies and videos were the other important issues of the lecture that made students impressed and helped them to be aware of the issues stated by the content of the lecture.

Summative Evaluation

The data obtained from the SE procedure were analyzed by making use of both qualitative and quantitative methods. The themes and codes emerging from the SE are presented in Table 2. When the codes emerging from this step are compared with the NA, it is clear that students’ needs, expectations, priorities and wills seemed to be satisfied. The only expectation that arose as a result of the NA, and that was not included in the syllabus at the beginning, was the wish for afield trip. In response to this preliminary result, a trip to the university recycling center was organized within the semester. In addition, two outdoor activities, tree planting and making a recycling stand during the university spring festival were organized with the students. These activities together with the field trip made “recycling” one of the most favorable subjects and all the students became very sensitive for collecting recyclable materials.

Therefore, as a result of SE, designed under the light of NA and FE procedure, it can be concluded that students felt sensitive, conscious, responsible and active as far as the sustainable use of natural resources concerned. As students claimed, the course titled “education and awareness for sustainability” helped them to develop awareness on the concept of sustainability and also to develop insights on how to integrate environmental issues into their profession and social life.

Discussion and Conclusions

What made students feel so comfortable about the course and develop a sense of responsibility? Was it the classroom environment, instructional methods and techniques, theoretical part of the lecture, material and equipment, practical part, relation to real life or role and attributes of instructor? As is obvious from the results of both the NA and the SE, integrating the issues in the syllabus with real life is the correct answer for this question. Visiting university recycling centre and making an “awareness stand” during the spring festival, for example, were the most popular activities of the semester and students became very sensitive about solid wastes produced and recycling. According to the responses given to the open ended questions during the SE, on the other hand, the most challenging point of the course is the real life cases presented related to each subject.

Table 2. Codes emerging from summative evaluation

Among these, the story of the aluminum can production impressed the students very much, especially after they saw the “mountains of cans” in the recycling centre. Related to history
and legends, on the other hand, the story of the Black Sea was another subject which attracted students’ interest. History of the Black Sea, its relation with the legend of Noah’s ark and that it is one of the major fish sources of Turkey today caused students to establish connections with past, present and future. The section titled “what is on media” was another part of the lecture that focused students’ attention on the TV programs, newspapers and documentaries related with environmental issues. Some of them, for example, suggested including just a few environmental responsibility related scenes into the most popular sit-coms. Some others also reported in the SE part that they started to become aware of and interested in the news and discussions in newspapers and on TV that they did not even care about before. Calculating their own footprints and realizing that they were already high, compared to the world average, was another point that impressed students and made them think how to decrease them. Me Millan et al (2004) reported a similar result in their study about the impact of university-level environmental studies class on students’ values. Sixty five percent of their respondents mentioned the ecological footprint tutorial as being very influential and one interview respondent of their study described the concept as a “real eye opener". Moreover, results of this study also reflect different attitudes of students from different faculties (Table 1). Although students’ overall evaluation of the course was similar, differences became evident when the students were asked about the reflections of this course on their professional lives. The important point to be mentioned here is that students from different fields were capable of integrating the concept of sustainability into their own fields. While all the pre-service teachers reported that they were planning to make their own students aware of the concept, engineering students reported that the fact that they became aware that the environment is an important issue to be considered, besides economic and technical issues, when evaluating a project. The assumption that students became capable of integrating the concept into their daily life was also evident from the assignments they did. For the first assignment, for example, they were asked to report a case from their experiences in life, which they think was related with the concept of “sustainability”. Many of the students reported a memory related with their former school years; students from rural places wrote cases about their home town and their elders (grandparents) and their common statement was that, they were just realized and integrate these memories with sustainability. When the point comes to the “decision making” step, on the other hand, as Arvai et al (2004) discussed in detail, as environmental issues become more complex and challenging and the need to act becomes more urgent, students require more than an appreciation for this complexity and urgency that is typically facilitated through typical curriculum content. They also require an appreciation for the complexities of decision making and must learn the skills that can help them to make higher quality choices. Along these lines, as the authors reported, teaching the theory and skills to address this requirement must receive prominent placement in curricula.

On the whole, the course had positive results on the students’ thinking on the sustainable ways of living. In their responses to the open ended questions in the SE, students attributed the changes in themselves to the course. However, it cannot be said for certain, as there are various limitations of the study, such as having no control groups. But, the results are consistent with the relevant literature, which ended up with positive change due to university level environmental classes. Almost all the studies in the literature related to the impact of universitylevel environmental studies class on students’ values result with positive changes (Learning et al., 1993; Mangas et al., 1997; McMillan, 2003; McMillan et al., 2004). As an overall, although the reported changes are encouraging, it is not known if they are temporary or long lasting ones. Therefore, future research would be recommended to follow the students over the long term and confirmatory evaluation might be a way for monitoring participants of the study. Moreover, gaining such a point of view and attitude toward sustainable living, the next
step for the education of the students, studying to teach science in primary schools, might be enabling them to introduce aspects of education for sustainability to their pupils. In this way only, the course “education and awareness for sustainability" will have a tiny role for constructing a sustainable future. Gayford’s article (2004) and his findings on this subject, on the other hand, can be the author’s inspiration to design such a study for the future.

References

Arvai, J. L., Campbell, V. E. A., Baird, A., & Rivers, L. (2004). ‘Teaching students to make better decisions about the environment: lessons from decision sciences”. The Journal of Environmental Education, 36(1), 33-44.

Bruntland, G. (ed.), (1987). Our common future: The World Commission on Environment and Development. Oxford: Oxford University Press.

Ciegis, R., & Gineitiene, D. (2006). The role of universities in promoting sustainability. Engineering Economics, 3(48), 56-62.

de Ciurana, A. M. G., & Filho, W. L. (2006). Education for sustainability in university studies: Experiences from project involving European and Latin American universities. International Journal of Sustainability in Higher Education, 7(1), 81-93.

de Rebello, D. (2003). Shaping the practical role of higher education for a sustainable development. A paper presented at International Conference on Education for a Sustainable Future. Charles University, Karolinum, Prague, Czech Republic, 10-11 September. Retrieved on October 08, 2006 from http://www.unesco.org/iau/sd/pdf/ Rebello.pdf

Gayford, C. (2004). A model for planning and evaluation of aspects of education for sustainability for students training to teach science in primary schools, Environmental Education Research, 10(2), 255-271.

Hopkins, C., & McKeown R. (2001). Education for Sustainable Development: past experience, present action and future prospects, Educational Philosophy and Theory, 33(2), 231-244.

Joint Committee on Standards for Educational Evaluation (1994). Standards for evaluations of educational programs projects and materials. Donneley & Sons Company.

Knapp, D. (1997). Twenty years after Tbilisi: UNESCO interregional workshop on reorienting environmental education for sustainable development. NAI’s Environmental Education Newsletter, 7(1), 6.

Leeming, F.C., Dwyer, W.O., Porter, B.E., & Cobem M. K. (1993). Outcome research in environmental education: A critical review. The Journal of Environmental Education, 24(4), 8-21.

Mangas, V., Martinez, P., & Pedauye, R. (1977). Analysis of environmental concepts and attitudes among biology degree students. The Journal of Environmental Education, 29 (1), 28-33

McKeown, R. (2002). Progress has been made in education for sustainable development. Applied Environmental Education and Communication, 1, 21-23.

McMillan E. E. (2003). A Method for Evaluating the Impact of an Introductory Environmental Studies Class on the Values of Students. Applied Environmental Education and Communication, 2, 91- 98.

McMillan, E.E., Wright, T., & Beazly, K. (2004). Impact of a University-Level Environmental studies class on students’ values. The Journal of Environmental Education, 35(3), 19-28.

Moore, J. (2005). Barriers and pathways to creating sustainability education programs: policy, rhetoric and reality. Environmental Education Research, 11(5), 537-555

Patton, M. Q. (1987). How to use qualitative methods in evaluation. In Newbury Park: CA. Sage.

Patton, M. Q. (1990). Qualitative evaluation and research methods (2^nd Ed.). Newbury Park: CA. Sage.

Stufflebeam, D. L., McCormick, С. H, Brinkerhoff, R. O., & Nelson, С. O. (1984). Conducting educational needs assessments. Massachusetts: Kluwer Academic Publishers.

UNESCO (2005). United Nations decade for education for sustainable development, 2005-2014: International implementation scheme. Paris, France: UNESCO.

UNESCO (1997a). International Conference Environment and Society: Education and Public Awareness for Sustainability; Declaration of Thessaloniki. Retrieved on October 02, 2006 from http://ncseonlme.org/ncseconference/2003conference/thessalonild_declaration.pdf

UNESCO (1997b). Educating for sustainable future: A Transdisciplinary vision for concerted action. Paris, France: UNESCO.

United Nations (1992). Chapter 36: Promoting education, public awareness and training. In Earth Summit: Agenda 21, The United Nations Programme of Action from Rio. New York, NY: United Nations.

Wright, T.S.A. (2002). Definitions and frameworks for environmental sustainability in higher education. International Journal of Sustainability in Higher Education, 3(3), 203-220.