21st century skills research

Toward an understanding of 21st-century skills: From a systematic review

• October 2021
• International Journal for Educational and Vocational Guidance 23(2)

Abstract and Figures

Discover the world's research

• 25+ million members
• 160+ million publication pages
• 2.3+ billion citations
• Mona Salah Abd-Allah Othman

• Kathleen Kupiec

• Dilber Sağır

• J COMPUT ASSIST LEAR

• Dandan Chen
• Bradley Kaptur

• Educ Inform Tech

• Jinming Zhang

• Nurakmal Baharum
• APPL PSYCH MEAS

• Qingshun Guo
• Huiyou Chang

• Recruit researchers
• Join for free
• Login Email Tip: Most researchers use their institutional email address as their ResearchGate login Password Forgot password? Keep me logged in Log in or Continue with Google Welcome back! Please log in. Email · Hint Tip: Most researchers use their institutional email address as their ResearchGate login Password Forgot password? Keep me logged in Log in or Continue with Google No account? Sign up

Integrating 21st century skills into education systems: From rhetoric to reality

Subscribe to the center for universal education bulletin, ramya vivekanandan rv ramya vivekanandan senior education specialist, learning assessment systems - gpe secretariat.

February 14, 2019

This is the third post in a series about  education systems alignment in teaching, learning, and assessing 21st century skills .

What does it mean to be a successful learner or graduate in today’s world? While in years past, a solid acquisition of the “three Rs” (reading, writing, and arithmetic) and mastery in the core academic subjects may have been the measure of attainment, the world of the 21 st century requires a radically different orientation. To participate effectively in the increasingly complex societies and globalized economy that characterize today’s world, students need to think critically, communicate effectively, collaborate with diverse peers, solve complex problems, adopt a global mindset, and engage with information and communications technologies, to name but just a few requirements. The new report from Brookings, “ Education system alignment for 21st century skills: Focus on assessment ,” illuminates this imperative in depth.

Recognizing that traditional education systems have generally not been preparing learners to face such challenges, the global education community has increasingly talked about and mobilized in favor of the changes required. This has resulted in a suite of initiatives and research around the broad area of “21st century skills,” which culminated most notably with the adoption of Sustainable Development Goal 4 and the Education 2030 agenda, including Target 4.7, which commits countries to ensure that learners acquire knowledge and skills in areas such as sustainable development, human rights, gender equality, global citizenship, and others.

In this landscape, Global Partnership for Education (GPE) has a core mandate of improving equity and learning by strengthening education systems. GPE supports developing countries, many of which are affected by fragility and conflict, to develop and implement robust education sector plans. Depending on the country, GPE implementation grants support a broad range of activities including teacher training, textbook provision, interventions to promote girls’ education, incentives for marginalized groups, the strengthening of data and learning assessment systems, early childhood education, and many other areas.

This work is buttressed by thematic work at the global level, including in the area of learning assessment. The strengthening of learning assessment systems is a strategic priority for GPE because of its relevance to both improving learning outcomes and ensuring effective and efficient education systems, which are two of the three key goals of the GPE strategic plan for the 2016-2020 period . The work on learning assessment includes the Assessment for Learning (A4L) initiative, which aims to strengthen learning assessment systems and to promote a holistic measurement of learning.

Under A4L, we are undertaking a landscape review on the measurement of 21st century skills, using a definition derived from Binkley et. al . and Scoular and Care :

“21st century skills are tools that can be universally applied to enhance ways of thinking, learning, working and living in the world. The skills include critical thinking/reasoning, creativity/creative thinking, problem solving, metacognition, collaboration, communication and global citizenship. 21st century skills also include literacies such as reading literacy, writing literacy, numeracy, information literacy, ICT [information and communications technologies] digital literacy, communication and can be described broadly as learning domains.”

Using this lens, the landscape review examines the research literature, the efforts of GPE partners that have been active in this space, and data collected from a sample of countries in sub-Saharan Africa and Asia in regard to the assessment of these skills. These research efforts were led by Brookings and coordinated by the UNESCO offices in Dakar and Bangkok. As another important piece of this work, we are also taking stock of the latest education sector plans and implementation grants of these same countries (nine in sub-Saharan Africa and six in Asia), to explore the extent to which the integration of 21st century skills is reflected in sector plans and, vitally, in their implementation.

Though the work is in progress, the initial findings provide food for thought. Reflecting the conclusions of the new report by Brookings, as well as its earlier breadth of work on skills mapping, a large majority of these 15 countries note ambitious objectives related to 21st century skills in their education sector plans, particularly in their vision or mission statements and/or statements of policy priorities. “Skills” such as creativity and innovation, critical thinking, problem-solving, decisionmaking, life and career skills, citizenship, personal and social responsibility, and information and communications technology literacy were strongly featured, as opposed to areas such as collaboration, communication, information literacy, and metacognition.

However, when we look at the planned interventions noted in these sector plans, there is not a strong indication that countries plan to operationalize their intentions to promote 21st century skills. Not surprisingly then, when we look at their implementation grants, which are one of the financing instruments through which education sector plans are implemented, only two of the 15 grants examined include activities aimed at promoting 21st century skills among their program components. Because the GPE model mandates that national governments determine the program components and allocation of resources for these within their grant, the bottom line seems to echo the findings of the Brookings report: vision and aspiration are rife, but action is scarce.

While the sample of countries studied in this exercise is small (and other countries’ education sector plans and grants may well include integration of 21st century skills), it’s the disconnect between the 15 countries’ policy orientation around these skills and their implementation that is telling. Why this gap? Why, if countries espouse the importance of 21st century skills in their sector plans, do they not concretely move to addressing them in their implementation? The reasons for this may be manifold, but the challenges highlighted by the Brookings report in terms of incorporating a 21 st century learning agenda in education systems are indeed telling. As a field, we still have much work to do to understand the nature of these skills, to develop learning progressions for them, and to design appropriate and authentic assessment of them. In other words, it may be that countries have difficulty in imagining how to move from rhetoric to reality.

However, in another perspective, there may be a challenge associated with how countries (and the broader education community) perceive 21st century skills in general. In contexts of limited resources, crowded curricula, inadequately trained teachers, fragility, weak governance, and other challenges that are characteristic of GPE partner countries, there is sometimes an unfortunate tendency to view 21st century skills and the “basics” as a tradeoff. In such settings, there can be a perception that 21st century skills are the concern of more advanced or higher-income countries. It is thus no wonder that, in the words of the Brookings report, “a global mobilization of efforts to respond to the 21CS [21st century skills] shift is non-existent, and individual countries struggle alone to plan the shift.”

This suggests that those who are committed to a holistic view of education have much work to do in terms of research, sharing of experience, capacity building, and advocacy around the potential and need for all countries, regardless of context, to move in this direction. The Brookings report makes a very valuable contribution in this regard. GPE’s landscape review, which will be published this spring, will inform how the partnership thinks about and approaches 21st century skills in its work and will thereby provide a complementary perspective.

Related Content

Helyn Kim, Esther Care, Alvin Vista

January 30, 2019

Tserennadmid Nyamkhuu, Jun Morohashi

February 5, 2019

Global Education K-12 Education

Global Economy and Development

Center for Universal Education

August 2, 2024

Modupe (Mo) Olateju, Grace Cannon, Kelsey Rappe

July 29, 2024

Sweta Shah, Donald Wertlieb, Charlotte Vuyiswa McClain-Nhlapo, Ruchi Kulbir Singh, Kathy Hirsh-Pasek

July 24, 2024

• Open access
• Published: 25 November 2019

Developing student 21 st Century skills in selected exemplary inclusive STEM high schools

• Stephanie M. Stehle   ORCID: orcid.org/0000-0003-4017-186X 1 &
• Erin E. Peters-Burton 1  

International Journal of STEM Education volume  6 , Article number:  39 ( 2019 ) Cite this article

127k Accesses

115 Citations

Metrics details

There is a need to arm students with noncognitive, or 21 st Century, skills to prepare them for a more STEM-based job market. As STEM schools are created in a response to this call to action, research is needed to better understand how exemplary STEM schools successfully accomplish this goal. This conversion mixed method study analyzed student work samples and teacher lesson plans from seven exemplary inclusive STEM high schools to better understand at what level teachers at these schools are engaging and developing student 21 st Century skills.

We found of the 67 lesson plans collected at the inclusive STEM high schools, 50 included instruction on 21 st Century skills. Most of these lesson plans designed instruction for 21 st Century skills at an introductory level. Few lesson plans encouraged multiple 21 st Century skills and addressed higher levels of those skills. Although there was not a significant difference between levels of 21 st Century skills by grade level, there was an overall trend of higher levels of 21 st Century skills demonstrated in lesson plans designed for grades 11 and 12. We also found that lesson plans that lasted three or more days had higher levels of 21 st Century skills.

Conclusions

These findings suggest that inclusive STEM high schools provide environments that support the development of 21 st Century skills. Yet, more can be done in the area of teacher professional development to improve instruction of high levels of 21 st Century skills.

Introduction

School-aged students in the USA are underperforming, particularly in science, technology, engineering, and mathematics (STEM) subjects. National Assessment of Educational Progress (U.S. Department of Education, 2015a ) scores show that in science, only 34% of 8th graders are performing at or above proficiency and 12th grade students at or above proficient US students drop to 22%. Similarly, mathematics scores show 33% of 8th graders and 22% of 12th graders were at or above proficiency (U.S. Department of Education, 2015a ). Additionally, the US mathematics scores for the Programme for International Student Assessment (PISA) for 2015 were lower than the scores for 2009 and 2012 (Organisation for Economic Co-operation and Development; OECD, 2018 ). US students not only underachieve in mathematics and science, but are also not engaging successfully in engineering and technology. At the secondary level, there are relatively few students in the USA that take engineering (2%) and computer science (5.7%) (National Science Board, 2016 ). The NAEP technology and engineering literacy (TEL) assessment found that for technology and engineering literacy, only 43% of 8th graders were at or above the proficiency level (U.S. Department of Education, 2015b ). This consistent trend of underperformance has focused many national, state, and local efforts to improve student experiences in integrated STEM subjects (cf. President’s Council of Advisors on Science and Technology, 2010 ; Texas Education Association ( n.d. ) for school-aged students and beyond.

The efforts for improvement in STEM teaching in K-12 environments have yielded a slight increase in the enrollment of STEM majors recently (National Science Board, 2016 ). However, roughly half of students who declare a STEM major when entering college either switch majors or drop out of college (National Science Board, 2016 ). One approach to helping students persist in undergraduate education is a stronger foundation in content knowledge, academic skills, and noncognitive skills (Farrington et al., 2012 ). Academic skills, including analysis and problem solving skills, allow students to engage with content knowledge at higher levels of cognition. Noncognitive skills, including study skills, time management, and self-management, assist students in optimizing their ability to gain content knowledge and use their academic skills to solve problems. Students who possess these skills have high-quality academic behaviors, characterized by a pursuit of academic goals despite any setbacks (Farrington et al., 2012 ).

Because academic skills, noncognitive skills, and content knowledge have fluid definitions and may not be directly observable, for the purposes of this study we used 21 st Century skills consisting of knowledge construction, real-world problem solving, skilled communication, collaboration, use of information and communication technology for learning, and self-regulation (Partnership for 21 st Century Learning, 2016 ). Graduates who possess 21 st Century skills are sought out by employers (National Research Council, 2013 ). In the environment of rapid advancements in technology and globalization, employees need to be flexible and perpetual learners in order to keep up with new developments (Bybee, 2013 ; Johnson, Peters-Burton, & Moore, 2016 ). There is a need to ensure that students who graduate the K-12 system are adept in 21 st Century skills so that they can be successful in this new workforce landscape (Bybee, 2013 ).

Not only do 21 st Century skills help students be successful in all areas of formal school, these skills are also necessary for a person to adapt and thrive in an ever changing world (Partnership for 21 st Century Learning, 2016 ). One movement embracing the need for the development of student 21 st Century skills is the proliferation of inclusive STEM high schools (ISHSs), schools that serve all students regardless of prior academic achievement (LaForce et al., 2016 ; Lynch et al., 2018 ). ISHSs promote student research experiences by using inquiry-based curricular models to scaffold independent learning and encourage personal responsibility (Tofel-Grehl & Callahan, 2014 ). The goal for ISHSs to facilitate this type of student-centered learning is to build students’ 21 st Century skills such as adaptability, communication, problem solving, critical thinking, collaboration, and self-management (Bybee, 2013 ; Johnson et al., 2016 ; LaForce et al., 2016 ). Although there has been some evidence that not all ISHSs are advantageous in offering STEM opportunities (Eisenhart et al., 2015 ), there is an accumulation of evidence that ISHSs can increase college and career readiness for students from groups who are typically underrepresented in STEM careers (Erdogan & Stuessy, 2015 ; Means, Wang, Viki, Peters, & Lynch, 2016 ). As the number of inclusive STEM schools continue to increase across the USA, there is a need to understand the ways these schools successfully engage students in 21 st Century skills. The purpose of this paper is to systematically analyze teacher-constructed lessons and student work from seven exemplar ISHSs in order to better understand how teachers are engaging and developing student 21 st Century skills.

Specifically, this study looked at the extent to which teachers at these exemplar ISHSs ask students to practice the 21 st Century skills and at the level of student performance of the following categories: (a) knowledge construction, (b) real-world problem solving, (c) skilled communication, (d) collaboration, (e) use of information and communication technology (ICT) for learning, and (f) self-regulation (SRI International, n.d. -a; SRI International, n.d. -b). An examination of the lesson plans and student work products at exemplar ISHSs provides insight into effective development of student 21 st Century skills in a variety of contexts.

Conceptual framework

In an attempt to clearly define the skills, content knowledge and literacies that students would need to be successful in their future endeavors, the Partnership for 21 st Century Learning (P21; 2016) created a framework that includes (a) life and career skills; (b) learning and innovation skills; (c) information, media, and technology skills; and (d) key subjects (Partnership for 21 st Century Learning, 2016 ). The first three parts of the framework, (a) life and career skills, (b) learning and innovation skills, and (c) information, media, and technology skills, describe proficiencies or literacies students should develop and can be integrated and developed in any academic lesson. The fourth piece, key subjects, suggests 21 st Century interdisciplinary themes or content to engage students in authentic study (Partnership for 21 st Century Learning, 2016 ).

Due to the need to build 21 st Century skills, this study focused on the teaching and learning of (a) learning and innovation skills; (b) information, media, and technology skills; and (c) life and career skills at exemplar ISHSs. In order to operationalize and measure the three categories, we searched for instruments that measured the learning of 21 st Century skills. Microsoft, in collaboration with SRI Education, developed two rubrics that are designed to assess the extent to which 21 st Century skills are present in lessons and the extent to which students demonstrate the skills from these lessons (SRI International, n.d. -a; SRI International, n.d. -b). The 21 st Century Learning Design Learning Activity Rubric examined the proficiency of teacher lesson plans for the development of 21 st Century skills while the 21 st Century Learning Design Student Work Rubric examined the level of competency for each 21 st Century skill. Although the rubrics did not align exactly with the P21 Framework, we felt that there was enough alignment with the categories that the rubrics would be useful in measuring the extent to which lessons in ISHSs taught 21 st Century skills and the extent to which students demonstrated these skills. The rubrics had the same categories for lesson assessment and student work assessment: (a) knowledge construction, (b) real-world problem solving, (c) skilled communication, (d) collaboration, (e) use of ICT for learning, and (f) self-regulation in teacher lesson plans and student work samples (SRI International, n.d. -a; SRI International, n.d. -b). Table 1 shows how the categories assessed in the two rubrics align with the categories in the P21 Framework. Further, as we reviewed the literature on these categories, a model of their relationship emerged. Our literature review discusses the individual categories followed by the conceptual model of how these categories work together in 21 st Century skill development.

• Knowledge construction

Knowledge construction occurs when students create new knowledge themselves rather than reproducing or consuming information (Prettyman, Ward, Jauk, & Awad, 2012 ; Shear, Novais, Means, Gallagher, & Langworthy, 2010 ). When students participate in knowledge construction rather than reproduction, they build a deeper understanding of the content. Learning environments that are designed for knowledge construction promote self-regulated and self-directed learners as well as building grit (Carpenter & Pease, 2013 ).

Although knowledge construction helps students to build deep understandings and skills to be self-directed and resilient learners, many students are unfamiliar with this approach to learning and frequently need scaffolding to take on joint responsibility of learning (Carpenter & Pease, 2013 ; Peters, 2010 ). When transitioning to a more student-centered learning environment that supports knowledge construction, the teacher becomes more of a facilitator rather than a lecturer (McCabe & O’Connor, 2014 ). A student-centered learning environment encourages students to shift from a paradigm of expecting one convergent answer and toward deeper meaning-making when learning (Peters, 2010 ). Knowledge construction anchors the development of 21 st Century skills because students need to be able to have background knowledge in order to perform the skills in an authentic context.

• Real-world problem solving

Sometimes called project-based learning (Warin, Talbi, Kolski, & Hoogstoel, 2016 ), real-world problem solving is characterized by students working to solve problems that have no current solution and where the students can implement their own approach (Shear et al., 2010 ). When solving a real-world problem, students work to identify the problem, propose a solution for a specific client, test the solution, and share their ideas (Prettyman et al., 2012 ; Warin et al., 2016 ). The design aspect of the process encourages students to be creative and learn from failures (Carroll, 2015 ). When using real-world problem solving, students develop knowledge in a meaningful way (White & Frederiksen, 1998 ), must regulate their cognition and behavior in a way to reach their goals (Brown, Bransford, Ferrara, & Campione, 1983 ; Flavell, 1987 ), and gain experience defending their choices through evidence and effective communication skills (Voss & Post, 1988 ).

Teachers can develop real-world problem solving skills in their students by modeling inquiry after research actual scientist are involved in, using databases with real-life data, and evaluating evidence from current events (Chinn & Malhortra, 2002 ). Designing real-world problem scenarios for the classroom provide a framework by which students can engage in 21 st Century learning and can help to encourage a more positive attitude towards STEM careers (Williams & Mangan, 2016 ). Together, knowledge construction and real-world problem solving create the foundation from which students can engage in self-regulation, collaboration, and communication.

• Self-regulation

Self-regulation is a key 21 st Century skill for independent learners. Students who are self-regulated plan their approach to problem solving, monitor their progress, and reflect on their work given feedback (Shear et al., 2010 ; Zimmerman, 2000 ). During the self-regulation process, a student motivates himself or herself to control impulses in order to efficiently solve problems (Carpenter & Pease, 2013 ; English & Kitsantas, 2013 ). Fortunately, these skills are teachable; however, students need time to accomplish regulatory tasks and guidance for the key processes of reflection and revision (Zimmerman, 2000 ). Therefore, long-term projects give a more appropriate time frame than short-term projects to hone these regulatory skills.

Students have different levels of self-regulation (English & Kitsantas, 2013 ) and teachers may need to integrate strategies and ways of monitoring students into lessons (Bell & Pape, 2014 ; English & Kitsantas, 2013 ). Incorporating self-regulated learning strategies helps students to stay engaged and deal with any adversity that may come up in the process (Boekaerts, 2016 ; Peters & Kitsantas, 2010 ). A tangible way teachers can support student self-regulation is by using Zimmerman’s ( 1998 ) four-stage model of self-regulated learning support: modeling, emulation, self-control, and self-regulation (Peters, 2010 ). First, teachers explicitly model the target learning strategy that the student should acquire, pointing out key processes (modeling). Second, teachers can provide students with verbal or written support for the key processes of the learning strategy while the student attempts to emulate the modeling from the teacher (emulation). Once students can roughly emulate the learning strategy, the teacher can fade support and have the student try to do the learning strategy on their own (self-control). After students attempt it on their own, the teacher provides feedback to the student to help them improve their attempted learning strategy (self-regulation). When a student can successfully perform the learning strategy on their own, they have become self-regulated in that aspect of their learning. Students who have mastered self-regulated learning have the ability to be proactive in knowledge building and in problem solving, which are characteristics that STEM industry employers value.

• Collaboration

Collaboration occurs when students take on roles and interact with one another in groups while working to produce a product (Shear et al., 2010 ). Collaborative interactions include taking on leadership roles, making decisions, building trust, communicating, reflecting, and managing conflicts (Carpenter & Pease, 2013 ). Students who collaborate solve problems at higher levels than students who work individually because students respond to feedback and questions to create solutions that better fit the problem (Care, Scoular, & Griffin, 2016 ). Collaboration is an important skill to enhance knowledge building and problem solving. Conversations among peers can support student self-regulated learning through modeling of verbalized thinking.

• Skilled communication

“Even the most brilliant scientific discovery, if not communicated widely and accurately, is of little value” (McNutt, 2013 , p. 13). For the purpose of this paper, skilled communication is defined as types of communication used to present or explain information, not discourse communication. Skilled communicators present their ideas and demonstrate how they use relevant evidence (Shear et al., 2010 ). An important part of being able to communicate successfully is the ability to connect a product to the needs of a specific audience or client (Warin et al., 2016 ). In doing so, the students need to take into account both the media they are using and the ideas they are communicating so that it is appropriate for the audience (Claro et al., 2012 ; van Laar, van Deursen, van Dijk, & de Haan, 2017 ). Like collaboration, skilled communication is a necessary process to successfully employ knowledge construction and real-world problem solving.

Use of information and communication technology for learning

When students use information and communication technology (ICT) for learning, they are designing, creating, representing, evaluating, or improving a product, not merely demonstrating their knowledge (Koh, Chai, Benjamin, & Hong, 2015 ). In doing so, they need to choose how and when to use the ICT as well as know how to recognize credible online resources (Shear et al., 2010 ). The effective use of ICT requires self-regulation in order to use these tools independently and to keep up with technological advances. Given the continuous advancements in technology, it is essential that students know how to manage and communicate information in order to solve problems (Ainley, Fraillon, Schulz, & Gebhardt, 2016 ).

Conceptual Model of 21 st Century Skills

The six 21 st Century skills presented above are critical for students to develop to prepare for both college (National Science Board, 2016 ) and the future employment (Bybee, 2013 ; Johnson et al., 2016 ). Twenty-first century skills do not exists in isolation. By building one skill, others are reinforced. For example, knowledge construction and real-world problem solving can be enhanced by self-regulation. Likewise, collaboration requires skilled communication to build knowledge and solve problems. These skills coalesce to build the necessary toolkit for students who can learn on their own. Figure 1 shows a working hypothesis of how these six skills, (a) knowledge construction, (b) real-world problem solving, (c) skilled communication, (d) collaboration, (e) use of ICT for learning, and (f) self-regulation, interact to foster lifelong learning for student.

Working hypothesis of how 21 st Century skills work together to build a 21 st Century student

Knowledge construction and real-world problem solving are the keystones of the model and typically represent the two main goals of student-centered lessons. Knowledge construction is the conceptual formation while real-world problem solving represents the process skills that students are expected to develop. Knowledge construction and real-world problem solving feed each other in a circular fashion. Knowledge construction is built through the inquiry process of real-world problem solving. At the same time, real-world problem solving requires new knowledge to be constructed in order to solve the problem at hand. The connection between knowledge construction and real work problem solving is mediated by collaboration and communication.

While communication and collaboration allow a student to work with others to build their conceptual knowledge and work toward a solution to their real-world problem, self-regulation is an internal process that occurs simultaneously. The student’s self-regulation guides the student’s individual connections, reflections, and revisions between knowledge construction and real-world problem solving.

Information and communication technology provides tools for the students to facilitate communication and collaboration as well as other 21 st Century skills. ICT helps to simplify and assist the communication and collaboration for groups of students. ICT can help streamline the process of analysis and record keeping as well as facilitating the sharing ideas with others. It allows students to more easily document their progress and express their ideas for later reflection. Although ICT is not directly connected with other elements in the model, the use of ICT allows for the learning process to be more efficient.

The six 21 st Century skills addressed in this study, (a) knowledge construction, (b) real-world problem solving, (c) skilled communication, (d) collaboration, (e) use of ICT for learning, and (f) self-regulation, are important facets of STEM education. This study documented the extent to which each of the 21 st Century skills were present in both lesson plans and in student work at seven exemplar ISHSs. Given that the schools in the study were highly regarded, understanding the structure and student outcomes of lessons could provide a model for teachers and teacher educators. With that in mind, the study was driven by the following research questions:

To what extent do teacher lesson plans at exemplar ISHSs exhibit 21 st Century learning practices as measured by the 21 st Century Learning Design Learning Activity and Student Work Rubrics?

Do teacher lesson plans and student work samples from exemplar ISHSs show differences in rubric scores by grade level?

During the analysis of these questions, a third research question emerged regarding the duration of lessons. The question and rationale can be found in the data analysis section.

This study is part of a larger multiple instrumental case study of eight exemplar ISHSs. The larger study (Opportunity Structures for Preparation and Inspiration in STEM; OSPrI) examined the common features of successful ISHSs (Lynch et al., 2018 ; Lynch, Peters-Burton, & Ford, 2014 ). OSPrI identified 14 critical components (CC; Table 2 ) that successful ISHSs possess (Behrend et al., 2016 ; Lynch et al., 2015 ; Lynch, Means, Behrend, & Peters-Burton, 2011 ; Peters-Burton, Lynch, Behrend, & Means, 2014 ). Three of the 14 critical components involve the application of 21 st Century skills in the classroom. This study addresses these three critical components: (a) CC1: STEM focused curriculum for all, (b) CC2: reform instructional strategies and project-based learning, and (c) CC3: integrated, innovative technology use.

Cross-case analysis of the eight schools found similarities in how the schools addressed two specific critical components: CC1: college-prep, STEM focused curriculum for all and CC2: reform instructional strategies and project-based learning. From these two critical components, curriculum and instruction, four themes emerged: (a) classroom-related STEM opportunities, (b) cross-cutting school level STEM learning opportunities, (c) school-wide design for STEM opportunities, and (d) responsive design (Peters-Burton, House, Han, & Lynch, 2018 ). The theme of classroom-related STEM opportunities was characterized by the expectation that teachers act as designers of the curriculum and look beyond the typical textbook for resources. While designing the curriculum, teachers took a mastery learning approach and provided students multiple opportunities to master the material. Through the use of collaborative group projects, summative projects, culminating projects, and interdisciplinary studies, the schools demonstrated a cross-cutting school level approach to the STEM learning. School-wide STEM opportunities included a rigorous curriculum, incorporating engineering classes and/or engineering design thinking, emphasizing connections between the curriculum and real-world examples, as well as building strong collaboration between teachers. Finally, these ISHSs had systems such as data-driven decision making and supports for incoming ninth graders built into their schools as a responsive design. In summary, these schools worked to improve students’ 21 st Century skill such as collaboration, problem solving, information and media literacy, and self-directed learning (Lynch et al., 2018 ).

Research design

This study was designed as a conversion mixed methods approach (Tashakkori & Teddlie, 2003 ) in that qualitative data were transformed into quantitative data using established rubrics. Document analysis was used as a tool to identify occasions of evidence within lessons plans and student work products related to the identified 21 st Century skills (Krippendorff, 2012 ). In this conversion approach, the 21 st Century skill demonstrated qualitatively in the documents was scored using the rubrics, ergo integrating qualitative and quantitative methods in the analysis.

Participating schools

The eight exemplar ISHSs for this study came from the same quintain as used by the OSPrI project (Lynch et al., 2018 ). Because this origin project was a cross-case analysis and the IRB did not allow for school to school comparison, the data collected from individual schools was aggregated as one data source. Protocol for inclusion in the OSPrI study was that the school had no academic admission requirements, self-identified as a STEM school, was in operation for grades 9 through 12, and intentionally recruited students typically underrepresented in STEM. For more information on the demographics of the schools and the selection process, see Lynch et al., 2018 . Of the eight schools that were in the original OSPrI project, seven provided teacher lesson plans and/or student work samples during the school visit. All schools have given permission to use their actual names. The sample size from each school was inconsistent, therefore, we treated the data set as one combined group that included all seven schools.

Data sources

Student work samples and teacher lesson plans were collected during OSPrI site visits to the seven schools, which were each visited once between 2012 and 2014. Researchers requested paper copies of typical lesson plans and student work that resulted in an average performance from the lesson plan that was observed at all eight ISHSs during the site visits. Because this was a convenience sample, not all teachers submitted lesson plans, and only a few teachers submitted the student work products related to those lessons. Unfortunately, few parents consented to release student work products. As a result, 67 teacher lesson plans and 29 student work samples were collected from seven of the eight schools. We decided to keep the student work products in the descriptive portion of the analysis, but not the inferential analysis in the study because this is a unique opportunity to gain even a small insight into student work from STEM schools that were considered exemplary and served students who are typically underrepresented in STEM. Table 3 describes the content matter and grade level(s) associated collected teacher lesson plan and corresponding student work product.

Each teacher lesson plan was analyzed using the 21 st Century Learning Design (21CLD) Learning Activity Rubric and each student work product was analyzed using the 21 st Century Learning Design Student Work Rubric (SRI International, n.d.-a; SRI International, n.d.-b). These instruments were found to be valid and reliable for use in high school classrooms, and Shear et al., 2010 reports the details of the development and validation of the rubrics. Although the student work products were related to the teacher lesson plans, they were analyzed independently according to the protocol of the 21CLD rubrics. The 21CLD Activity Rubric and the 21CLD Student Work Rubric were designed by Microsoft Partner’s in Learning with a collaboration between ITL Research and SRI International (SRI International, n.d.-a; SRI International, n.d.-b). These two 21CLD rubrics were the result of a multi-year project synthesizing research-based practices that promote 21 st Century skills (Shear et al., 2010 ). The rubrics, each 44-pages in length, are available online for public use ( https://education.microsoft.com/GetTrained/ITL-Research ). The 21CLD rubrics assess teacher lesson plans or student work products on six metrics aligned with 21 st Century skills: (a) knowledge construction, (b) real-world problem solving, (c) skilled communication, (d) collaboration, (e) use of ICT for learning, and (f) self-regulation (SRI International, n.d.-a; SRI International, n.d.-b). Collaboration, knowledge construction, and use of ICT score ratings range from one to five while real-world problem solving, self-regulation, and skilled communication score ratings range from one to four.

Data analysis

The teacher lessons and student work samples were assessed on (a) knowledge construction, (b) real-world problem solving, (c) skilled communication, (d) collaboration, (e) use of ICT for learning, and (f) self-regulation using the 21CLD Learning Activity and the 21CLD Student Work Rubrics respectively. Examples of excerpts from teacher lesson plans and student work products for each category can be found in Table 4 . Two raters were used to establish interrater reliability. Both raters have a background as secondary science teachers and were trained on the use of the rubric. One rater has a terminal degree in education and the other rater is a doctoral student in education. The two raters met and discussed the rubric scores until the interrater reliability was 100%. Once consensus scores were established, tests for assumptions, descriptive, and inferential statistics were run.

During the analysis of research questions one and two, unique trends of short-term and long-term lesson plans were noted. From this, a third research question emerged from the analysis:

Are there differences in the 21 CLD Learning Activity scores of short-term lessons and long-term lessons?

The 21CLD Learning Activity and the 21CLD Student Work Rubrics required a lesson to be long-term order to assess self-regulation. The rubric defined long-term as “if students work on it for a substantive period of time” (SRI International, n.d.-a, p. 32). From our reading of the lesson plans, lessons that were scheduled for three or more days met the criterion of a substantive period of time, while lesson that were scheduled for 1 or 2 days did not meet this criterion. For the purposes of this study, we decided to refine the definition of long-term to be a lesson lasting three or more class periods and a short-term lesson lasting less than three class periods. The analyses for all research questions separated lessons into long-term and short-term in order to clarify the category of self-regulation.

The data were checked for normality, skewness, and outliers; only the teacher lesson plans met all assumptions for an ANOVA (comparison of grade levels) and t test (long-term versus short-term). Due to the small number of student work samples collected (see Table 6 ), the data related to student work did not meet the assumptions needed to run a t test therefore was not included in this analysis.

Overall rubric scores

To answer the first research question, a descriptive analysis was run for each of the six categories on the rubric and the total score (found in Tables 5 and 6 ). The average score for all teacher lesson plans was less than 2 for all six categories (out of a total of 4 or 5). Likewise, overall student work sample averages scored below 2 except on the category of Knowledge Construction. Table 6 also shows the median score for long-term student work sample categories to better describe central tendencies of the data. Figure 2 shows the distribution of total rubric scores for all teacher lesson plans. Seventeen of the 67 lessons scored a 6, the lowest possible score. Only 16 of the 67 lessons scored higher than 13 points, half of the total possible points. Out of those 16 scoring over 50%, only three lessons scored 20 points or more out of the possible 27.

Distribution of total 21CLD rubric scores for all lessons

Figure 3 illustrates the quantity of 21 st Century skills found in each lesson. Nearly 75% of the teacher lesson plans included at least one 21 st Century skill in the lesson and 67% addressed two or more 21 st Century skills. Although most of the lessons at the ISHSs introduced multiple 21 st Century skills, the overall scores for the quality were low.

Distribution of number of 21 st Century skills addressed in a lesson

21 st Century learning by grade

To answer the second research question, an ANOVA was conducted to compare lesson scores by grade level. There were no statistically significant differences between grade level scores for the total rubric score. Data were separated into short-term and long-term lessons by rubric category. There were no significant differences in short-term lessons by grade level (Fig. 4 ). However, there were significant differences across grades for long-term lessons. Total rubric score for grade 12 lessons were significantly higher than grade 9 ( p = 0.023) and grade 11 ( p = 0.032). Difference in total rubric scores for grade 12 lessons were approaching significance with grade 10 ( p = 0.063). As seen in Fig. 5 , category scores for long-term learning activities have small differences in 9th, 10th, and 11th grades but peaks noticeably in 12th grade. The exception to this trend is use of ICT which peaks in 11th grade.

The average rubric metric scores for short-term lessons, sorted by grade level for the lesson

The average rubric metric scores for long-term lessons, sorted by grade level for the lesson

Long-term versus short-term assignments

To answer the second research question, a t test with Bonferroni correction was performed to compare long-term and short-term lessons for each of the categories. A statistically significant difference was found between short-term ( N = 35) and long-term ( N = 32) lessons on total score, knowledge construction, use of ICT, self-regulation, and skilled communication (Table 7 ). The effect sizes for these categories as calculated by Hedges g (Lakens, 2013 ) were all above 0.8 indicated large effect size (Table 7 ). In all of those categories, long-term lessons scored higher than short-term lessons (Table 5 ). The category of real-world problem solving was approaching statistical significance with the t-score not showing significance [ t = − 2.67, p = .001] but a statistically significant confidence interval [− 1.23, 0.003] and a medium effect size (Table 7 ).

• 21 st Century skills

Overall, the teacher lesson plans collected at the ISHSs showed evidence of addressing 21 st Century skills. Nearly 75% of the lessons included at least one 21 st Century skill with 67% addressing two or more. Although the majority of lessons addressed multiple 21 st Century skills, the rubric scores for these lessons were low because they addressed these skills at a minimal level. For example, a minimal level of collaboration would be instructions to form a group. A high level of collaboration would include defining roles, explicit instructions on how to share responsibility, and evidence of interdependence. Only five lessons showed evidence of multiple 21 st Century skills implemented at the highest level, as measured by the 21CLD Learning Activity Rubric.

While assessing the lesson plans, we noted that more explicit instructions in the teacher lesson plans would have resulted in higher rubric scores. Placing students in groups, structuring peer feedback, and having students design a final project for a particular audience are three small changes not seen frequently in the lesson plans that are articulated in the Lesson Plan rubrics to encourage multiple 21 st Century skills. When students work in groups, they improve their collaboration and communication skills while constructing knowledge and solving problems (Care et al., 2016 ; Shear et al., 2010 ). When teachers incorporate peer feedback into their lesson, students engage in collaboration. Peer feedback also gives students the opportunity to revise their work based on feedback, increasing self-regulation (Shear et al., 2010 ; Zimmerman, 2000 ). When students design their final project for a specific target audience, rather than simply displaying their knowledge for the teacher, they work on their skilled communication processes (Claro et al., 2012 ; van Laar et al., 2017 ; Warin et al., 2016 ). In summary, placing students in groups, structuring peer feedback, and having students design a final project for a particular audience provides opportunities for students to practice 21 st Century skills.

When lessons addressed more than one 21 st Century skill, they usually demonstrated the use of collaboration or communication in real-world problem solving and knowledge construction (Care et al., 2016 ; Carpenter & Pease, 2013 ). Thirty-three lesson plans in which real-world problem solving or knowledge construction was evident, 31 showed evidence of collaboration or communication. Similarly, 13 of the 18 student work samples showed evidence of collaboration or communication when real-world problem solving or knowledge construction was practiced. The results from the indirect measures of the rubric build support for a conceptual model connecting the components of 21 st Century skills (Fig. 1 ). There was some evidence demonstrating the support that collaboration and communication have for knowledge construction and real-world problem solving.

The findings of this study point to the likelihood of self-regulation being connected to other 21 st Century skills. Each time self-regulation was present in a teacher lesson plan, there was evidence of at least one other 21 st Century skill in that lesson. Seventeen of the 23 lesson plans addressing self-regulation included at least three other 21 st Century skills, showing evidence that self-regulation is a skill that is related to knowledge construction and real-world problem solving. Our findings reflect the findings of other researchers, in that self-regulation guides the students’ individual connections, reflections, and revisions between knowledge construction and real-world problem solving (Brown et al., 1983 ; Carpenter & Pease, 2013 ; Flavell, 1987 ; Shear et al., 2010 ).

Evidence from the lessons showed that there was no consistent connection to the use of ICT and the presence of the other 21 st Century skills. ICT was seen in both low-scoring lessons as the sole 21 st Century skill, as well as in high-scoring lessons in tandem with multiple other 21 st Century skills. As in our model, technology is a tool to help facilitate but is not necessary in the development of the other 21 st Century skills (Koh et al., 2015 ; Shear et al., 2010 ). After examining the data, our model remained unchanged for all 21 st Century skills and their relationship to each other.

Grade level differences

Overall, there were no statistically significant differences in the total 21CLD scores across grade levels. This is consistent with the missions of the ISHSs in this study to shift responsibility for learning to the students by weaving 21 st Century skills throughout high school grade levels (Lynch et al., 2017 ). When looking at trends in long-term projects, there was a jump in total 21CLD score for 12th grade. Again, this aligns with the participating schools’ goals of creating an environment where students have a more independent learning experience during their senior year internships, college classes, and specialized programs CC1 (Lynch et al., 2018 ). This is consistent with the goal of many of the schools to have the students work independently during their senior year either by taking college classes, completing an internship, or taking a career specific set of classes.

Short-term vs. long-term lessons

The data showed that long-term lesson planning had significantly higher scores on the rubric as compared to the short-termed lessons. This difference is consistent with the literature regarding the need for students to have time to develop and practice skills (Lynch et al., 2017 ; NGSS Lead States, 2013 ). The extended time allows students to monitor and reflect on their progress while working toward self-regulation of the skill (Carpenter & Pease, 2013 ; English & Kitsantas, 2013 ). To truly become self-regulated, students need repeated supported attempts to be able to do it on their own (Zimmerman, 2000 ).

Although not significant, collaboration was the only rubric metric where the short-term lessons averaged a higher collaboration score than the long-term lessons. Evidence from the lessons show students worked in pairs or groups, but infrequently shared responsibility, made decisions together, or worked interdependently. This leads to the possibility that incorporating the higher levels of collaborations is difficult, even in long-term projects. In addition, evaluating the higher levels of collaboration is difficult to make based solely on documents. Observations would be required to evaluate how the students within the group were interacting with one another.

Limitations

Because this study used data collected as part of a larger study, there were several limitations. The work collected is a snapshot of the work students were doing at the time of the observation and does not allow for a clear longitudinal look at student growth over time. As stated before, the small student work sample limited what we were able to do with the analysis.

By only analyzing paper copies of the student work, it was not possible to determine a true collaboration score for many of the projects. Higher levels of collaboration such as sharing responsibility, making decisions together, and working interdependently require observation or more detailed notes from the students or teachers. Some lessons may have scored higher in the metric of collaboration had the student interactions been observed or noted.

This study confirmed the presence of all identified 21 st Century skills in the lesson plans at the selected exemplar ISHSs serving underrepresented students in STEM: (a) knowledge construction, (b) real-world problem solving, (c) skilled communication, (d) collaboration, (e) use of information and communication technology (ICT) for learning, and (f) self-regulation. In light of the patterns that emerged from the rubrics, we posit that in the lesson plans communication and collaboration are the core 21st Century skills that facilitate knowledge construction and real-world problem solving, while student self-regulation creates efficiencies resulting in improved knowledge construction and real-world problem solving. We also saw in the lesson plans that ICT provides tools to support communication and reflection which leads to knowledge construction and real-world problem solving. To further develop knowledge about how 21 st Century skills addressed in lesson plans help to support student work, our model can be a hypothesized starting point to investigate interactions.

While teachers were successful at including 21 st Century skills into lessons, very few lessons practiced higher levels of those skills. This could be an indication that high levels of 21 st Century skills are difficult to teach explicitly at the high school level. Future studies may investigate why teachers are not frequently incorporating higher level 21 st Century skills into their lessons to answer questions as to whether teachers feel that (a) they need more training on incorporating 21 st Century skills, (b) students need more practice and scaffolding to build up to higher levels of 21 st Century skills, or (c) they need more time for long-term projects to work on the higher level skills.

The use of the 21CLD rubric is a tangible way for teachers to self-assess the level of 21 st Century skills in their lessons. Self-evaluation helps encourage reflection, promote professional growth, and recommendations for new aspects of lessons (Akram & Zepeda, 2015 ; Peterson & Comeaux, 1990 ). This can also help teachers make the instructions for the development of 21 st Century skills more explicit in their lesson. In conducting a self-evaluation, teachers may realize that they do not have a deep understanding of the characteristics of 21 st Century skills. If teachers are new to incorporating these skills into their lessons, the teachers may need time to learn the skills themselves before they can incorporate them into their lessons (Yoon et al., 2015 ). Further studies may examine how teachers use the 21CLD rubric to improve their lesson.

Students need time to grapple with and learn new skills (Lynch et al., 2017 ; NGSS Lead States, 2013 ). While we were able to see evidence of higher rubric scores for 21 st Century skills for 12th grade students in the lesson plans, due to the convenience sampling of lesson plans and student work samples, we were not able to look at how students’ 21 st Century skills were built over time. There is a desire to better understand how ISHSs successfully develop these skills. This includes how schools incorporate and build the 21 st Century skills (a) within multiple lessons in one course, (b) across multiple classes over the course of a school year, and (c) throughout the students’ entire high school sequence. Future research may look at a longitudinal study that follows one student’s work over an entire school year to see how the 21CLD scores change. In addition, future studies may also look at how the short-term projects build the skills needed for the students to incorporate higher levels of 21 st Century skills in long-term projects.

Availability of data and materials

The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request.

Abbreviations

21 st Century Learning Design

Critical component

Information and communication technology

Inclusive STEM high school

National Assessment of Educational Progress

Next-generation science standards

Opportunity Structures for Preparation and Inspiration in STEM

Partnership for 21 st Century Learning

Programme for International Student Assessment

Science, technology, engineering, and mathematics

Technology and engineering literacy

Ainley, J., Fraillon, J., Schulz, W., & Gebhardt, E. (2016). Conceptualizing and measuring computer and information literacy in cross-national contexts. Applied Measurement in Education, 29 , 291–309. https://doi.org/10.1080/08957347.2016.1209205 .

Article   Google Scholar  

Akram, M., & Zepeda, S. J. (2015). Development and validation of a teacher self-assessment instrument. Journal of Research and Reflections in Education, 9 (2), 134–148.

Google Scholar  

Behrend, T. S., Peters-Burton, E. E., Hudson, C., Matray, S., Ford, M., & Lynch, S. J. (2016). STEM High School Inventory. [Measurement instrument]. Retrieved from https://ospri.research.gwu.edu/sites/ospri.research.gwu.edu/files/downloads/CC%20Inventory_FINAL.pdf .

Bell, C. V., & Pape, S. J. (2014). Scaffolding the development of self-regulated learning in mathematics classrooms. Middle School Journal, 45 (4), 23–32.

Boekaerts, M. (2016). Engagement as an inherent aspect of the learning process. Learning and Instruction, 43 , 76–83. https://doi.org/10.1016/j.learninstruc.2016.02.001 .

Brown, A. L., Bransford, J., Ferrara, R., & Campione, J. (1983). Learning, remembering, and understanding. In P. H. Musen (Ed.), Handbook of child psychology (Vol. III, pp. 77–166). New York: Wiley.

Bybee, R. W. (2013). The case for STEM education . Arlington: NSTA press.

Care, E., Scoular, C., & Griffin, P. (2016). Assessment of collaborative problem solving in education environments. Applied Measurement in Education, 29 , 250–264. https://doi.org/10.1080/08957347.2016.1209204 .

Carpenter, J. P., & Pease, J. S. (2013). Preparing students to take responsibility for learning: The role of non-curricular learning strategies. Journal of Curriculum & Instruction, 7 (2), 38–55. https://doi.org/10.3776/joci.2013.v7n2p38-55 .

Carroll, M. (2015). Stretch, dream, and do—A 21 st century design thinking & STEM journey. Journal of Research in STEM Education, 1 (1), 59–70.

Chinn, C. A., & Malhortra, B. A. (2002). Epistemologically authentic inquiry in schools: A theoretical framework for evaluating inquiry tasks. Science Education, 86 , 175–218. https://doi.org/10.1002/sce.10001 .

Claro, M., Preiss, D. D., San Martin, E., Jara, I., Hinostroza, J. E., Valenzuela, S., Cortes, F., & Nussbaun, M. (2012). Assessment of 21 st century ICT skills in Chile: Test design and results from high school level students. Computers & Education, 59 , 1042–1053. https://doi.org/10.1016/j.compedu.2012.04.004 .

Eisenhart, M., Weis, L., Allen, C. D., Cipollone, K., Stich, A., & Dominguez, R. (2015). High school opportunities for STEM: Comparing inclusive STEM-focused and comprehensive high schools in two US cities. Journal of Research in Science Teaching, 52 (6), 763–789. https://doi.org/10.1002/tea.21213 .

English, M. C., & Kitsantas, A. (2013). Supporting student self-regulated learning in problem- and project based learning. Interdisciplinary Journal of Problem-Based Learning, 7 (2), 127–150. https://doi.org/10.7771/1541-5015.1339 .

Erdogan, N., & Stuessy, C. (2015). Examining the role of inclusive STEM schools in the college and career readiness of students in the United States: A multi-group analysis on the outcome of student achievement. Educational Sciences: Theory & Practice, 15 (6), 1517–1529. https://doi.org/10.12738/estp.2016.1.0072 .

Farrington, C. A., Roderick, M., Allensworth, E., Nagaoka, J., Keyes, T. S., Johnson, D. W., & Beechum, N. O. (2012). Teaching adolescents to become learners. The role of noncognitive factors in shaping school performance: A critical literature review . Chicago: University of Chicago Consortium on Chicago School Research.

Flavell, J. H. (1987). Speculations about the nature and development of metacognition. In F. Weinert & U. R. Kluwe (Eds.), Metacognition, motivation, and understanding (pp. 21–29). Hillsdale: Erlbaum.

Johnson, C. C., Peters-Burton, E. E., & Moore, T. J. (Eds.). (2016). STEM road map: A framework for integrated STEM education . New York: Routledge.

Koh, J. H. L., Chai, C. S., Benjamin, W., & Hong, H. Y. (2015). Technological pedagogical content knowledge (TPACK) and design thinking: A framework to support ICT lesson design for 21st century learning. Asia-Pacific Education Researcher (Springer Science & Business Media B.V.), 24 (3), 535–543. https://doi.org/10.1007/s40299-015-0237-2 .

Krippendorff, K. H. (2012). Content analysis: An introduction to its methodology (3rd ed.). Los Angeles: SAGE Publications, Inc..

LaForce, M., Noble, E., King, H., Century, J., Blackwell, C., Holt, S., Ibrahim, A., & Loo, S. (2016). The eight essential elements of inclusive STEM high schools. International Journal of STEM Education, 3 (21), 1–11. https://doi.org/10.1186/s40594-016-0054-z .

Lakens, D. (2013). Calculating and reporting effect sizes to facilitate cumulative science: A practical primer for t-tests and ANOVAs. Frontiers in Psychology, 4 , 863. https://doi.org/10.3389/fpsyg.2013.00863 .

Lynch, S. J., House, A., Peters-Burton, E., Behrend, T., Means, B., Ford, M., Spillane, N., Matray, S., Moore, I., Coyne, C., Williams, C., & Corn, J. (2015). A Logic model that describes and explains eight exemplary STEM-focused high schools with diverse student populations . Washington DC: George Washington University OSPrI Project Retrieved from http://ospri.research.gwu.edu .

Lynch, S. J., Means, B., Behrend, T., & Peters-Burton, E. (2011). Multiple instrumental case studies of inclusive STEM-focused high schools: Opportunity Structures for Preparation and Inspiration (OSPrI). Retrieved from http://ospri.research.gwu.edu

Lynch, S. J., Peters-Burton, E. E., Behrend, T., House, A., Ford, M., Spillane, N., Matray, S., Han, E., & Means, B. (2018). Understanding inclusive STEM high schools as opportunity structures for underrepresented students: Critical components. Journal of Research in Science Teaching., 55 (5), 712–748. https://doi.org/10.1002/tea.21437 .

Lynch, S. J., Peters-Burton, E. E., & Ford, M. (2014). Building STEM opportunities for all. Educational Leadership, 72 (4), 54–60.

Lynch, S. J., Spillane, N., House, A., Peters-Burton, E., Behrend, T., Ross, K. M., & Han, E. M. (2017). A policy-relevant instrument case study of an inclusive STEM-focused high school: Manor New Tech High. International Journal of Education in Mathematics, Science and Technology, 5 (1), 1–20. https://doi.org/10.18404/ijemst.75656 .

McCabe, A., & O’Connor, U. (2014). Student-centered learning: The role and responsibility of the lecturer. Teacher in Higher Education, 19 (4), 350–359. https://doi.org/10.1080/13562517.2013.860111 .

McNutt, M. (2013). Improving scientific communication. Science, 342 , 13. https://doi.org/10.1126/science.1246449 .

Means, B., Wang, H., Viki, Y., Peters, V. L., & Lynch, S. J. (2016). STEM-focused high schools as a strategy for enhancing readiness for postsecondary STEM programs. Journal of Research in Science Teaching, 53 (5), 709–736. https://doi.org/10.1002/tea.21313 .

National Research Council. (2013). Monitoring progress toward successful K-12 STEM education: A nation advancing? Washington, DC: National Academies Press. https://doi.org/10.17226/13509 .

Book   Google Scholar  

National Science Board. (2016). Science and engineering indicators 2016. (NSB-2016-1) . Arlington: National Science Foundation.

NGSS Lead States. (2013). Next generation science standards: For states, by states . Washington, DC: The National Academies Press.

Organisation for Economic Co-operation and Development. (2018). PISA 2015 results in focus. Retrieved from https://www.oecd.org/pisa/pisa-2015-results-in-focus.pdf

Partnership for 21st Century Learning. (2016). Framework for 21st century learning. Retrieved from www.p21.org/about-us/p21-framework .

Peters, E. E. (2010). Shifting to a student-centered science classroom: An exploration of teacher and student changes in perceptions and practices. Journal of Science Teacher Education, 21 (3), 329–349. https://doi.org/10.1007/s10972-009-9178-z .

Peters, E. E., & Kitsantas, A. (2010). The effect of nature of science metacognitive prompts on science students’ content and nature of science knowledge, metacognition, and self-regulatory efficacy. School Science and Mathematics, 110 , 382–396. https://doi.org/10.1111/j.1949-8594.2010.00050.x .

Peters-Burton, E. E., House, A., Han, E., & Lynch, S. (2018). Curriculum and instruction at inclusive STEM high schools. Journal of Research in STEM Education, 4 (2), 193–212.

Peters-Burton, E. E., Lynch, S. J., Behrend, T. S., & Means, B. B. (2014). Inclusive STEM high school design: 10 critical components. Theory into Practice, 53 (1), 64–71. https://doi.org/10.1080/00405841.2014.862125 .

Peterson, P. L., & Comeaux, M. A. (1990). Evaluating the systems: Teachers’ perspectives on teacher evaluation. Educational Evaluation and Policy Analysis, 12 (1), 3–24. https://doi.org/10.3102/01623737012001003 .

President’s Council of Advisors on Science and Technology. (2010). Prepare and inspire: K-12 education in science, technology, engineering, and math (STEM) for America’s future . Washington, DC: Executive Office of the President.

Prettyman, S. S., Ward, C. L., Jauk, D., & Awad, G. (2012). 21st century learners: Voices of students in a one-to-one STEM environment. Journal of Applied Learning Technology, 2 (4), 6–15.

Shear, L., Novais, G., Means, B., Gallagher, L., & Langworthy, M. (2010). ITL research design . Menlo Park: SRI International Retrieved from https://www.sri.com/sites/default/files/publications/itl_research_design_15_nov_2010.pdf .

SRI International. (n.d.-a). 21CLD learning activity rubrics. Retrieved from https://education.microsoft.com/GetTrained/ITL-Research

SRI International. (n.d.-b). 21CLD student work rubrics. Retrieved from https://education.microsoft.com/GetTrained/ITL-Research

Tashakkori, A., & Teddlie, C. (2003). Handbook of mixed methods in social & behavioral research . Thousand Oaks: Sage.

Texas Education Association (n.d.). Texas science, technology, engineering and mathematics initiative (T-STEM). Retrieved from https://tea.texas.gov/T-STEM/

Tofel-Grehl, C., & Callahan, C. M. (2014). STEM high school communities: Common and differing features. Journal of Advanced Academics, 25 (3), 237–271. https://doi.org/10.1177/1932202X14539156 .

U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics. (2015a). The nation’s report card: 2015 mathematics and reading assessments. (NCES No. 2015136). Retrieved from https://www.nationsreportcard.gov/reading_math_2015/#?grade=4

U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics. (2015b). The nation’s report card: Technology and engineering literacy. (NCES No. 2016119). Retrieved from https://www.nationsreportcard.gov/tel_2014/

van Laar, E., van Deursen, A. J. A. M., van Dijk, J. A. G. M., & de Haan, J. (2017). The relation between 21 st -century skills and digital skills: A systematic literature review. Computers in Human Behavior, 77 , 577–588. https://doi.org/10.1016/j.chb.2017.03.010 .

Voss, J. F., & Post, T. A. (1988). On the solving of ill-structured problems. In M. T. H. Chi, R. Glaser, & M. J. Farr (Eds.), The nature of expertise (pp. 261–285). Hillsdale: Lawrence Erlbaum.

Warin, B., Talbi, O., Kolski, C., & Hoogstoel, F. (2016). Multi-role project (MRP): A new project-based learning method for STEM. IEEE Transactions on Education, 59 (2), 137–146. https://doi.org/10.1109/TE.2015.2462809 .

White, B. Y., & Frederiksen, J. R. (1998). Inquiry, modeling, and metacognition: Making science accessible to all students. Cognition and Instruction, 16 (1), 3–18.

Williams, P. J., & Mangan, J. (2016). The effectiveness of using young professionals to influence STEM career choices of secondary school students. Journal of Research in STEM Education, 2 (1), 2–18.

Yoon, S. A., Anderson, E., Koehler-Yom, J., Klopfer, E., Sheldon, J., Wendel, D., Schoenfeld, I., Scheintaub, H., Oztok, M., & Evans, C. (2015). Designing curriculum and instruction for computer-supported complex systems teaching and learning in high school science classrooms. Journal of Research in STEM Education, 1 (1), 4–14.

Zimmerman, B. J. (1998). Developing self-fulfilling cycles of academic regulation: An analysis of exemplary instructional models. In D. H. Schunk & B. J. Zimmerman (Eds.), Self-regulated learning: From teaching to self-reflective practice (pp. 1–19). New York: The Guilford Press.

Zimmerman, B. J. (2000). Attaining self-regulation: A social cognitive perspective. In M. Boekaerts, P. R. Pintrich, & M. Zeidner (Eds.), Handbook of self-regulation (pp. 13–39). San Diego: Academic Press.

Chapter   Google Scholar  

Download references

Acknowledgments

Publication of this article was funded in part by the George Mason University Libraries Open Access Publishing Fund.

This work was conducted by the OSPrI research project, with Sharon Lynch, Tara Behrend, Erin Peters-Burton, and Barbara Means as principal investigators. Funding for OSPrI was provided by the National Science Foundation (DRL 1118851). Any opinions, findings, conclusions, or recommendations are those of the authors and do not necessarily reflect the position or policy of endorsement of the funding agency.

Author information

Authors and affiliations.

George Mason University, Fairfax, USA

Stephanie M. Stehle & Erin E. Peters-Burton

You can also search for this author in PubMed   Google Scholar

Contributions

Both authors contributed equally to this manuscript. Both authors read and approved the final manuscript.

Corresponding author

Correspondence to Stephanie M. Stehle .

Ethics declarations

Competing interests.

The authors declare that they have no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License ( http://creativecommons.org/licenses/by/4.0/ ), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

Reprints and permissions

About this article

Cite this article.

Stehle, S.M., Peters-Burton, E.E. Developing student 21 st Century skills in selected exemplary inclusive STEM high schools. IJ STEM Ed 6 , 39 (2019). https://doi.org/10.1186/s40594-019-0192-1

Download citation

Received : 21 December 2018

Accepted : 04 October 2019

Published : 25 November 2019

DOI : https://doi.org/10.1186/s40594-019-0192-1

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• STEM schools

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

• View all journals
• Explore content
• About the journal
• Publish with us
• Sign up for alerts
• Published: 09 May 2022

Teaching of 21st century skills needs to be informed by psychological research

• Samuel Greiff   ORCID: orcid.org/0000-0003-2900-3734 1 &
• Francesca Borgonovi   ORCID: orcid.org/0000-0002-6759-4515 2  

Nature Reviews Psychology volume  1 ,  pages 314–315 ( 2022 ) Cite this article

300 Accesses

5 Citations

3 Altmetric

Metrics details

• Problem solving
• Science, technology and society

The technological advancements and globalization of the 21st century require a broad set of skills beyond traditional subjects such as mathematics, reading, and science. Research in psychological science should inform best practice and evidence-based recommendations for teaching these skills.

This is a preview of subscription content, access via your institution

Relevant articles

Open Access articles citing this article.

Behavioral patterns in collaborative problem solving: a latent profile analysis based on response times and actions in PISA 2015

• , Florian Krieger
•  …  Samuel Greiff

Large-scale Assessments in Education Open Access 13 November 2023

Access options

Subscribe to this journal

Receive 12 digital issues and online access to articles

55,14 € per year

only 4,60 € per issue

Buy this article

• Purchase on Springer Link
• Instant access to full article PDF

Prices may be subject to local taxes which are calculated during checkout

Frank, M. R. et al. Toward understanding the impact of artificial intelligence on labor. Proc. Natl Acad. Sci. USA 116 , 6531–6539 (2019).

Article   Google Scholar  

OECD. Skills Outlook 2021: Learning for Life (OECD Publishing, 2021).

OECD. PISA 2015 Results: Collaborative Problem Solving (OECD Publishing, 2017).

Autor, D. H. Skills, education, and the rise of earnings inequality among the “other 99 percent”. Science 344 , 843–851 (2014).

Hattie, J. A. C. & Donoghue, G. M. Learning strategies: a synthesis and conceptual model. NPJ Sci. Learn. 1 , 16013 (2016).

Sonnleitner, P., Brunner, M., Keller, U. & Martin, R. Differential relations between facets of complex problem solving and students’ immigration background. J. Edu. Psy. 106 , 681–695 (2014).

Graesser, A. C., Sabatini, J. P. & Li, H. Educational psychology is evolving to accommodate technology, multiple disciplines, and twenty-first century skills. Annu. Rev. Psychol. 73 , 547–574 (2022).

National Academy of Sciences. Investigating the Influence of Standards. A Framework for Research in Mathematics , Science and Technology Education (National Academies Press, 2022).

Markovitz, D. The Meritocracy Trap: How America’s Foundational Myth Feeds Inequality, Dismantles the Middle Class, and Devours the Elite (Penguin, 2020).

Download references

Author information

Authors and affiliations.

Department of Behavioural and Cognitive Sciences, University of Luxembourg, Esch-sur-Alzette, Luxembourg

• Samuel Greiff

Social Research Institute, University College London, London, UK

Francesca Borgonovi

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Samuel Greiff .

Ethics declarations

Competing interests.

The authors declare no competing interests.

Rights and permissions

Reprints and permissions

About this article

Cite this article.

Greiff, S., Borgonovi, F. Teaching of 21st century skills needs to be informed by psychological research. Nat Rev Psychol 1 , 314–315 (2022). https://doi.org/10.1038/s44159-022-00064-w

Download citation

Published : 09 May 2022

Issue Date : June 2022

DOI : https://doi.org/10.1038/s44159-022-00064-w

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

This article is cited by

• Florian Krieger

Large-scale Assessments in Education (2023)

Quick links

• Explore articles by subject
• Guide to authors
• Editorial policies

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Twenty-First Century Skills: From Theory to Action

• First Online: 09 November 2017

Cite this chapter

• Esther Care 6  

Part of the book series: Educational Assessment in an Information Age ((EAIA))

5955 Accesses

21 Citations

3 Altmetric

This chapter provides a general introduction to issues and initiatives in the assessment of twenty-first century skills, the implications of assessment for the teacher and teacher training, the role played by technologies not only for demonstration of skills but for their measurement, and a look to the future. Frameworks that have informed a gradual shift in the aspirations of education systems for their students are described, followed by evidence of implementations globally and regionally. The role of the Assessment and Teaching of 21st Century Skills (ATC21S; Griffin et al. (Eds.) (2012), Assessment and teaching of 21st century skills. Springer, Dordrecht) in reflecting and acting on a call by global consortia is outlined. This provides the context for the book contents, with the chapters briefly described within their thematic parts. The chapters provide a clear picture of the complexities of the introduction of teaching and assessment strategies based on skills rather than content.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save.

• Get 10 units per month
• Download Article/Chapter or eBook
• 1 Unit = 1 Article or 1 Chapter
• Cancel anytime
• Available as PDF
• Read on any device
• Instant download
• Own it forever
• Available as EPUB and PDF
• Compact, lightweight edition
• Dispatched in 3 to 5 business days
• Free shipping worldwide - see info
• Durable hardcover edition

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Introducing Assessment Tools for 21st Century Skills in Finland

The Explicit Nature of Educational Goals for the Twenty-first Century

Key Skills in the Context of Twenty-First-Century Teaching and Learning

Ahonen, A. K., Häkkinen, P., & Pöysä-Tarhonen, J. (2018). Collaborative problem solving in Finnish pre-service teacher education: A case study. In E. Care, P. Griffin, & M. Wilson (Eds.), Assessment and teaching of 21st century skills: Research and applications . Dordrecht: Springer.

Google Scholar  

Australian Curriculum, Assessment and Reporting Authority (ACARA). (2013). General capabilities in the Australian curriculum. Retrieved from http://www.australiancurriculum.edu.au/GeneralCapabilities/Pdf/Overview .

Binkley, M., Erstad, O., Herman, J., Raizen, S., & Ripley, M. (2012). Defining 21st century skills. In P. Griffin, B. McGaw, & E. Care (Eds.), The assessment and teaching of 21st century skills . Dordrecht: Springer.

Bressler, D. (2014). Better than business-as-usual: Improving scientific practices during discourse and writing by playing a collaborative mystery game. Double Helix, 2 , 1–13.

Bujanda, M. E., Muñoz, L., & Zúñiga, M. (2018). Initiatives and implementation of 21st century skills teaching and assessment in Costa Rica. In E. Care, P. Griffin, & M. Wilson (Eds.), Assessment and teaching of 21st century skills: Research and applications . Dordrecht: Springer.

Care, E., & Kim, H. (2018). Assessment of 21st century skills: The issue of authenticity. In E. Care, P. Griffin, & M. Wilson (Eds.), Assessment and teaching of 21st century skills: Research and applications . Dordrecht: Springer.

Care, E., & Luo, R. (2016). Assessment of transversal competencies: Policy and practice in the Asia-Pacific Region . Bangkok: UNESCO.

Care, E., Anderson, K., & Kim, H. (2016). Visualizing the breadth of skills movement across education systems , Skills for a Changing World . Washington, DC: The Brookings Institution.

Comfort, K., & Timms, M. (2018). A 21st skills lens on the common core standards and the next generation science standards. In E. Care, P. Griffin, & M. Wilson (Eds.), Assessment and teaching of 21st century skills: Research and applications . Dordrecht: Springer.

Delors, J., Al Mufti, I., Amagai, I. A., Carneiro, R., Chung, F., Geremek, B., et al. (1996). Learning: The treasure within . Paris: UNESCO.

Funke, J. (2010). Complex problem solving: A case for complex cognition? Cognitive Process, 11 , 133–142.

Article   Google Scholar  

Funke, J., Fischer, A., & Holt, D. V. (2018). Competencies for complexity: Problem solving in the 21st century. In E. Care, P. Griffin, & M. Wilson (Eds.), Assessment and teaching of 21st century skills: Research and applications . Dordrecht: Springer.

Gordon, J., Halasz, G., Krawczyk, M., Leney, T., Michel, A., Pepper, D., Putkiewicz, E., & Wisniewski, J. (2009). Key competences in Europe: Opening doors for lifelong learners across the school curriculum and teacher education . Warsaw: Center for Social and Economic Research.

Graesser, A., Foltz, P. W., Rosen, Y., Shaffer, D. W., Forsyth, C., & Germany, M. (2018). Challenges of assessing collaborative problem-solving. In E. Care, P. Griffin, & M. Wilson (Eds.), Assessment and teaching of 21st century skills: Research and applications . Dordrecht: Springer.

Griffin, P., & Care, E. (Eds.). (2015). Assessment and teaching of 21st century skills: Methods and approach . Dordrecht: Springer.

Griffin, P., McGaw, B., & Care, E. (Eds.). (2012). Assessment and teaching of 21st Century Skills . Dordrecht: Springer.

Gulikers, J. T. M., Bastiaens, T. J., & Kirschner, P. A. (2004). A five-dimensional framework for authentic assessment. Educational Technology Research and Development, 52 , 67–86.

Hesse, F., Care, E., Buder, J., Sassenberg, K., & Griffin, P. (2015). A framework for teachable collaborative problem solving skills. In P. Griffin & E. Care (Eds.), Assessment and teaching of 21st century skills: Methods and approach . Dordrecht: Springer.

Kozma, R. B. (2011). A framework for ICT policies to transform education. In Transforming education: The power of ICT policies . Paris: UNESCO.

Krkovic, K., Mustafic, M., Wüstenberg, S., & Greiff, S. (2018). Shifts in the assessment of problem solving. In E. Care, P. Griffin, & M. Wilson (Eds.), Assessment and teaching of 21st century skills: Research and applications . Dordrecht: Springer.

Learning Metrics Task Force. (2013a). Toward universal learning: What every child should learn . Montreal: UNESCO Institute for Statistics and Brookings Institution.

Learning Metrics Task Force. (2013b). Toward universal learning: Recommendations from the Learning Metrics Task Force . Montreal: UNESCO Institute for Statistics and Brookings Institution.

Nieveen, N., & Plomp, T. (2018). Curricular and implementation challenges in introducing 21st century skills education. In E. Care, P. Griffin, & M. Wilson (Eds.), Assessment and teaching of 21st century skills: Research and applications . Dordrecht: Springer.

OECD. (2013). Draft PISA collaborative problem solving framework . Paris, France: OECD Publishing.

OECD. (2016). Global competency for an inclusive world . Paris: OECD Publishing.

Pellegrino, J. W., & Hilton, M. L. (Eds.). (2012). Education for life and work: Developing transferable knowledge and skills in the 21st century . Washington, DC: The National Academies Press.

Ramalingam, D., & Adams, R. J. (2018). How can use of data from computer-delivered assessments improve measurement? In E. Care, P. Griffin, & M. Wilson (Eds.), Assessment and teaching of 21st century skills: Research and applications . Dordrecht: Springer.

Roblyer, M. D. (2006). Integrating educational technology into teaching . Upper Saddle River: Pearson.

Rychen, D. S., & Salganik, L. H. (Eds.). (2001). Defining and selecting key competencies . Göttingen: Hogrefe & Huber.

Scalise, K. (2018). Next wave for integration of educational technology into the classroom: Collaborative technology integration planning practices. In E. Care, P. Griffin, & M. Wilson (Eds.), Assessment and teaching of 21st century skills: Research and applications . Dordrecht: Springer.

Scoular, C., & Care, E. (2018). Teaching 21st century skills: Implications at system levels in Australia. In E. Care, P. Griffin, & M. Wilson (Eds.), Assessment and teaching of 21st century skills: Research and applications . Dordrecht: Springer.

Scoular, C., Care, E., & Hesse, F. (2018). Designs for operationalizing collaborative problem solving for automated assessment. Journal of Educational Measurement, 54 (1), 12–35.

Tan, J. P., Caleon, I., Koh, E., Poon, C. L., & Ng, H. L. (2018). Collective creativity and collaborative problem-solving among Singapore secondary school students. In E. Care, P. Griffin, & M. Wilson (Eds.), Assessment and teaching of 21st century skills: Research and applications . Dordrecht: Springer.

UNESCO. (2015a). Sustainable development goals: 17 goals to transform our world . http://www.un.org/sustainabledevelopment/sustainable-development-goals/ . Paris: UNESCO.

UNESCO. (2015b). 2013 Asia-Pacific Education Research Institutes Network (ERI-Net) regional study on transversal competencies in education policy & practice (phase 1) . Paris: UNESCO.

UNESCO. (2016). 2014 Asia-Pacific Education Research Institutes Network (ERI-Net) regional study on transversal competencies in education policy & practice (phase 11): School and teaching practices for twenty-first century challenges: Lessons from the Asia-Pacific Region . Paris: UNESCO.

Voogt, J., & Roblin, N. P. (2012). A comparative analysis of international frameworks for 21st century competences: Implications for national curriculum policies. Journal of Curriculum Studies, 44 (3), 299–321.

Wilson, M., Scalise, K., & Gochyyev, P. (2015). Rethinking ICT literacy: From computer skills to social network settings. Thinking Skills and Creativity, 18 , 65–80.

Wilson, M., Scalise, K., & Gochyyev, P. (2018a). Learning in digital networks as a modern approach to ICT Literacy. In E. Care, P. Griffin, & M. Wilson (Eds.), Assessment and teaching of 21st century skills: Research and applications . Dordrecht: Springer.

Wilson, M., Scalise, K., & Gochyyev, P. (2018b). Intersecting learning analytics and measurement science in the context of ICT literacy assessment. In E. Care, P. Griffin, & M. Wilson (Eds.), Assessment and teaching of 21st century skills: Research and applications . Dordrecht: Springer.

Download references

Author information

Authors and affiliations.

Brookings Institution, Washington, DC, USA

Esther Care

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Esther Care .

Editor information

Editors and affiliations.

Brookings Institution, Washington DC, Washington, USA

Melbourne Graduate School of Education, University of Melbourne, Parkville, Victoria, Australia

Patrick Griffin

University of California, Berkeley, California, USA

Mark Wilson

Rights and permissions

Reprints and permissions

Copyright information

© 2018 Springer International Publishing AG

About this chapter

Care, E. (2018). Twenty-First Century Skills: From Theory to Action. In: Care, E., Griffin, P., Wilson, M. (eds) Assessment and Teaching of 21st Century Skills. Educational Assessment in an Information Age. Springer, Cham. https://doi.org/10.1007/978-3-319-65368-6_1

Download citation

DOI : https://doi.org/10.1007/978-3-319-65368-6_1

Published : 09 November 2017

Publisher Name : Springer, Cham

Print ISBN : 978-3-319-65366-2

Online ISBN : 978-3-319-65368-6

eBook Packages : Education Education (R0)

Share this chapter

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Publish with us

Policies and ethics

• Find a journal
• Track your research

Warning: The NCBI web site requires JavaScript to function. more...

An official website of the United States government

The .gov means it's official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you're on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings
• Browse Titles

NCBI Bookshelf. A service of the National Library of Medicine, National Institutes of Health.

National Research Council (US) Committee on the Assessment of 21st Century Skills. Assessing 21st Century Skills: Summary of a Workshop. Washington (DC): National Academies Press (US); 2011.

Assessing 21st Century Skills: Summary of a Workshop.

• Hardcopy Version at National Academies Press

3 Assessing Interpersonal Skills

The second cluster of skills—broadly termed interpersonal skills—are those required for relating to other people. These sorts of skills have long been recognized as important for success in school and the workplace, said Stephen Fiore, professor at the University of Central Florida, who presented findings from a paper about these skills and how they might be assessed (Salas, Bedwell, and Fiore, 2011). 1 Advice offered by Dale Carnegie in the 1930s to those who wanted to “win friends and influence people,” for example, included the following: be a good listener; don’t criticize, condemn, or complain; and try to see things from the other person’s point of view. These are the same sorts of skills found on lists of 21st century skills today. For example, the Partnership for 21st Century Skills includes numerous interpersonal capacities, such as working creatively with others, communicating clearly, and collaborating with others, among the skills students should learn as they progress from preschool through postsecondary study (see Box 3-1 for the definitions of the relevant skills in the organization’s P-21 Framework).

Interpersonal Capacities in the Partnership for 21st Century Skills Framework. Develop, implement, and communicate new ideas to others effectively Be open and responsive to new and diverse perspectives; incorporate group input and feedback into the work (more...)

It seems clear that these are important skills, yet definitive labels and definitions for the interpersonal skills important for success in schooling and work remain elusive: They have been called social or people skills, social competencies, soft skills, social self-efficacy, and social intelligence, Fiore said (see, e.g., Ferris, Witt, and Hochwarter, 2001 ; Hochwarter et al., 2006 ; Klein et al., 2006 ; Riggio, 1986 ; Schneider, Ackerman, and Kanfer, 1996 ; Sherer et al., 1982 ; Sternberg, 1985 ; Thorndike, 1920 ). The previous National Research Council (NRC) workshop report that offered a preliminary definition of 21st century skills described one broad category of interpersonal skills ( National Research Council, 2010 , p. 3):

• Complex communication/social skills: Skills in processing and interpreting both verbal and nonverbal information from others in order to respond appropriately. A skilled communicator is able to select key pieces of a complex idea to express in words, sounds, and images, in order to build shared understanding ( Levy and Murnane, 2004 ). Skilled communicators negotiate positive outcomes with customers, subordinates, and superiors through social perceptiveness, persuasion, negotiation, instructing, and service orientation ( Peterson et al., 1999 ).

These and other available definitions are not necessarily at odds, but in Fiore’s view, the lack of a single, clear definition reflects a lack of theoretical clarity about what they are, which in turn has hampered progress toward developing assessments of them. Nevertheless, appreciation for the importance of these skills—not just in business settings, but in scientific and technical collaboration, and in both K-12 and postsecondary education settings—has been growing. Researchers have documented benefits these skills confer, Fiore noted. For example, Goleman (1998) found they were twice as important to job performance as general cognitive ability. Sonnentag and Lange (2002) found understanding of cooperation strategies related to higher performance among engineering and software development teams, and Nash and colleagues (2003) showed that collaboration skills were key to successful interdisciplinary research among scientists.

• WHAT ARE INTERPERSONAL SKILLS?

The multiplicity of names for interpersonal skills and ways of conceiving of them reflects the fact that these skills have attitudinal, behavioral, and cognitive components, Fiore explained. It is useful to consider 21st century skills in basic categories (e.g., cognitive, interpersonal, and intrapersonal), but it is still true that interpersonal skills draw on many capacities, such as knowledge of social customs and the capacity to solve problems associated with social expectations and interactions. Successful interpersonal behavior involves a continuous correction of social performance based on the reactions of others, and, as Richard Murnane had noted earlier, these are cognitively complex tasks. They also require self-regulation and other capacities that fall into the intrapersonal category (discussed in Chapter 4 ). Interpersonal skills could also be described as a form of “social intelligence,” specifically social perception and social cognition that involve processes such as attention and decoding. Accurate assessment, Fiore explained, may need to address these various facets separately.

The research on interpersonal skills has covered these facets, as researchers who attempted to synthesize it have shown. Fiore described the findings of a study ( Klein, DeRouin, and Salas, 2006 ) that presented a taxonomy of interpersonal skills based on a comprehensive review of the literature. The authors found a variety of ways of measuring and categorizing such skills, as well as ways to link them both to outcomes and to personality traits and other factors that affect them. They concluded that interpersonal effectiveness requires various sorts of competence that derive from experience, instinct, and learning about specific social contexts. They put forward their own definition of interpersonal skills as “goal-directed behaviors, including communication and relationship-building competencies, employed in interpersonal interaction episodes characterized by complex perceptual and cognitive processes, dynamic verbal and non verbal interaction exchanges, diverse roles, motivations, and expectancies” (p. 81).

They also developed a model of interpersonal performance, shown in Figure 3-1 , that illustrates the interactions among the influences, such as personality traits, previous life experiences, and the characteristics of the situation; the basic communication and relationship-building skills the individual uses in the situation; and outcomes for the individual, the group, and the organization. To flesh out this model, the researchers distilled sets of skills for each area, as shown in Table 3-1 .

Model of interpersonal performance. NOTE: Big Five personality traits = openness, conscientiousness, extraversion, agreeableness, and neuroticism; EI = emotional intelligence; IPS = interpersonal skills. SOURCE: Stephen Fiore’s presentation. Klein, (more...)

Taxonomy of Interpersonal Skills.

Fiore explained that because these frameworks focus on behaviors intended to attain particular social goals and draw on both attitudes and cognitive processes, they provide an avenue for exploring what goes into the development of effective interpersonal skills in an individual. They also allow for measurement of specific actions in a way that could be used in selection decisions, performance appraisals, or training. More specifically, Figure 3-1 sets up a way of thinking about these skills in the contexts in which they are used. The implication for assessment is that one would need to conduct the measurement in a suitable, realistic context in order to be able to examine the attitudes, cognitive processes, and behaviors that constitute social skills.

• ASSESSMENT APPROACHES AND ISSUES

One way to assess these skills, Fiore explained, is to look separately at the different components (attitudinal, behavioral, and cognitive). For example, as the model in Figure 3-1 indicates, previous life experiences, such as the opportunities an individual has had to engage in successful and unsuccessful social interactions, can be assessed through reports (e.g., personal statements from applicants or letters of recommendation from prior employers). If such narratives are written in response to specific questions about types of interactions, they may provide indications of the degree to which an applicant has particular skills. However, it is likely to be difficult to distinguish clearly between specific social skills and personality traits, knowledge, and cognitive processes. Moreover, Fiore added, such narratives report on past experience and may not accurately portray how one would behave or respond in future experiences.

The research on teamwork (or collaboration)—a much narrower concept than interpersonal skills—has used questionnaires that ask people to rate themselves and also ask for peer ratings of others on dimensions such as communication, leadership, and self-management. For example, Kantrowitz (2005) collected self-report data on two scales: performance standards for various behaviors, and comparison to others in the subjects’ working groups. Loughry, Ohland, and Moore (2007) asked members of work teams in science and technical contexts to rate one another on five general categories: contribution to the team’s work; interaction with teammates; contribution to keeping the team on track; expectations for quality; and possession of relevant knowledge, skills, and abilities.

Another approach, Fiore noted, is to use situational judgment tests (SJTs), which are multiple-choice assessments of possible reactions to hypothetical teamwork situations to assess capacities for conflict resolution, communication, and coordination, as Stevens and Campion (1999) have done. The researchers were able to demonstrate relationships between these results and both peers’ and supervisors’ ratings and to ratings of job performance. They were also highly correlated to employee aptitude test results.

Yet another approach is direct observation of team interactions. By observing directly, researchers can avoid the potential lack of reliability inherent in self- and peer reports, and can also observe the circumstances in which behaviors occur. For example, Taggar and Brown (2001) developed a set of scales related to conflict resolution, collaborative problem solving, and communication on which people could be rated.

Though each of these approaches involve ways of distinguishing specific aspects of behavior, it is still true, Fiore observed, that there is overlap among the constructs—skills or characteristics—to be measured. In his view, it is worth asking whether it is useful to be “reductionist” in parsing these skills. Perhaps more useful, he suggested, might be to look holistically at the interactions among the facets that contribute to these skills, though means of assessing in that way have yet to be determined. He enumerated some of the key challenges in assessing interpersonal skills.

The first concerns the precision, or degree of granularity, with which interpersonal expertise can be measured. Cognitive scientists have provided models of the progression from novice to expert in more concrete skill areas, he noted. In K-12 education contexts, assessment developers have looked for ways to delineate expectations for particular stages that students typically go through as their knowledge and understanding grow more sophisticated. Hoffman (1998) has suggested the value of a similar continuum for interpersonal skills. Inspired by the craft guilds common in Europe during the Middle Ages, Hoffman proposed that assessment developers use the guidelines for novices, journeymen, and master craftsmen, for example, as the basis for operational definitions of developing social expertise. If such a continuum were developed, Fiore noted, it should make it possible to empirically examine questions about whether adults can develop and improve in response to training or other interventions.

Another issue is the importance of the context in which assessments of interpersonal skills are administered. By definition, these skills entail some sort of interaction with other people, but much current testing is done in an individualized way that makes it difficult to standardize. Sophisticated technology, such as computer simulations, or even simpler technology can allow for assessment of people’s interactions in a standardized scenario. For example, Smith-Jentsch and colleagues (1996) developed a simulation of an emergency room waiting room, in which test takers interacted with a video of actors following a script, while others have developed computer avatars that can interact in the context of scripted events. When well executed, Fiore explained, such simulations may be able to elicit emotional responses, allowing for assessment of people’s self-regulatory capacities and other so-called soft skills.

Workshop participants noted the complexity of trying to take the context into account in assessment. For example, one noted both that behaviors may make sense only in light of previous experiences in a particular environment, and that individuals may display very different social skills in one setting (perhaps one in which they are very comfortable) than another (in which they are not comfortable). Another noted that the clinical psychology literature would likely offer productive insights on such issues.

The potential for technologically sophisticated assessments also highlights the evolving nature of social interaction and custom. Generations who have grown up interacting via cell phone, social networking, and tweeting may have different views of social norms than their parents had. For example, Fiore noted, a telephone call demands a response, and many younger people therefore view a call as more intrusive and potentially rude than a text message, which one can respond to at his or her convenience. The challenge for researchers is both to collect data on new kinds of interactions and to consider new ways to link the content of interactions to the mode of communication, in order to follow changes in what constitutes skill at interpersonal interaction. The existing definitions and taxonomies of interpersonal skills, he explained, were developed in the context of interactions that primarily occur face to face, but new technologies foster interactions that do not occur face to face or in a single time window.

In closing, Fiore returned to the conceptual slippage in the terms used to describe interpersonal skills. Noting that the etymological origins of both “cooperation” and “collaboration” point to a shared sense of working together, he explained that the word “coordination” has a different meaning, even though these three terms are often used as if they were synonymous. The word “coordination” captures instead the concepts of ordering and arranging—a key aspect of teamwork. These distinctions, he observed, are a useful reminder that examining the interactions among different facets of interpersonal skills requires clarity about each facet.

• ASSESSMENT EXAMPLES

The workshop included examples of four different types of assessments of interpersonal skills intended for different educational and selection purposes—an online portfolio assessment designed for high school students; an online assessment for community college students; a situational judgment test used to select students for medical school in Belgium; and a collection of assessment center approaches used for employee selection, promotion, and training purposes.

The first example was the portfolio assessment used by the Envision High School in Oakland, California, to assess critical thinking, collaboration, communication, and creativity. At Envision Schools, a project-based learning approach is used that emphasizes the development of deeper learning skills, integration of arts and technology into core subjects, and real-world experience in workplaces. 2 The focus of the curriculum is to prepare students for college, especially those who would be the first in their family to attend college. All students are required to assemble a portfolio in order to graduate. Bob Lenz, cofounder of Envision High School, discussed this online portfolio assessment.

The second example was an online, scenario-based assessment used for community college students in science, technology, engineering, and mathematics (STEM) programs. The focus of the program is on developing students’ social/communication skills as well as their technical skills. Louise Yarnall, senior research scientist with SRI, made this presentation.

Filip Lievens, professor of psychology at Ghent University in Belgium, described the third example, a situational judgment test designed to assess candidates’ skill in responding to health-related situations that require interpersonal skills. The test is used for high-stakes purposes.

The final presentation was made by Lynn Gracin Collins, chief scientist for SH&A/Fenestra, who discussed a variety of strategies for assessing interpersonal skills in employment settings. She focused on performance-based assessments, most of which involve role-playing activities.

Online Portfolio Assessment of High School Students 3

Bob Lenz described the experience of incorporating in the curriculum and assessing several key interpersonal skills in an urban high school environment. Envision Schools is a program created with corporate and foundation funding to serve disadvantaged high school students. The program consists of four high schools in the San Francisco Bay area that together serve 1,350 primarily low-income students. Sixty-five percent qualify for free or reduced-price lunch, and 70 percent are expected to be the first in their families to graduate from college. Most of the students, Lenz explained, enter the Envision schools at approximately a sixth-grade level in most areas. When they begin the Envision program, most have exceedingly negative feelings about school; as Lenz put it they “hate school and distrust adults.” The program’s mission is not only to address this sentiment about schools, but also to accelerate the students’ academic skills so that they can get into college and to develop the other skills they will need to succeed in life.

Lenz explained that tracking students’ progress after they graduate is an important tool for shaping the school’s approach to instruction. The first classes graduated from the Envision schools 2 years ago. Lenz reported that all of their students meet the requirements to attend a 4-year college in California (as opposed to 37 percent of public high school students statewide), and 94 percent of their graduates enrolled in 2- or 4-year colleges after graduation. At the time of the presentation, most of these students (95 percent) had re-enrolled for the second year of college. Lenz believes the program’s focus on assessment, particularly of 21st century skills, has been key to this success.

The program emphasizes what they call the “three Rs”: rigor, relevance, and relationships. Project-based assignments, group activities, and workplace projects are all activities that incorporate learning of interpersonal skills such as leadership, Lenz explained. Students are also asked to assess themselves regularly. Researchers from the Stanford Center for Assessment, Learning, and Equity (SCALE) assisted the Envision staff in developing a College Success Assessment System that is embedded in the curriculum. Students develop portfolios with which they can demonstrate their learning in academic content as well as 21st century skill areas. The students are engaged in three goals: mastery knowledge, application of knowledge, and metacognition.

The components of the portfolio, which is presented at the end of 12th grade, include

• A student-written introduction to the contents
• Examples of “mastery-level” student work (assessed and certified by teachers prior to the presentation)
• Reflective summaries of work completed in five core content areas
• An artifact of and a written reflection on the workplace learning project
• A 21st century skills assessment

Students are also expected to defend their portfolios, and faculty are given professional development to guide the students in this process. Eventually, Lenz explained, the entire portfolio will be archived online.

Lenz showed examples of several student portfolios to demonstrate the ways in which 21st century skills, including interpersonal ones, are woven into both the curriculum and the assessments. In his view, teaching skills such as leadership and collaboration, together with the academic content, and holding the students to high expectations that incorporate these sorts of skills, is the best way to prepare the students to succeed in college, where there may be fewer faculty supports.

STEM Workforce Training Assessments 4

Louise Yarnall turned the conversation to assessment in a community college setting, where the technicians critical to many STEM fields are trained. She noted the most common approach to training for these workers is to engage them in hands-on practice with the technologies they are likely to encounter. This approach builds knowledge of basic technical procedures, but she finds that it does little to develop higher-order cognitive skills or the social skills graduates need to thrive in the workplace.

Yarnall and a colleague have outlined three categories of primary skills that technology employers seek in new hires ( Yarnall and Ostrander, in press ):

Social-Technical

• Translating client needs into technical specifications
• Researching technical information to meet client needs
• Justifying or defending technical approach to client
• Reaching consensus on work team
• Polling work team to determine ideas
• Using tools, languages, and principles of domain
• Generating a product that meets specific technical criteria
• Interpreting problems using principles of domain

In her view, new strategies are needed to incorporate these skills into the community college curriculum. To build students’ technical skills and knowledge, she argued, faculty need to focus more on higher-order thinking and application of knowledge, to press students to demonstrate their competence, and to practice. Cooperative learning opportunities are key to developing social skills and knowledge. For the skills that are both social and technical, students need practice with reflection and feedback opportunities, modeling and scaffolding of desirable approaches, opportunities to see both correct and incorrect examples, and inquiry-based instructional practices.

She described a project she and colleagues, in collaboration with community college faculty, developed that was designed to incorporate this thinking, called the Scenario-Based Learning Project (see Box 3-2 ). This team developed eight workplace scenarios—workplace challenges that were complex enough to require a team response. The students are given a considerable amount of material with which to work. In order to succeed, they would need to figure out how to approach the problem, what they needed, and how to divide up the effort. Students are also asked to reflect on the results of the effort and make presentations about the solutions they have devised. The project begins with a letter from the workplace manager (the instructor plays this role and also provides feedback throughout the process) describing the problem and deliverables that need to be produced. For example, one task asked a team to produce a website for a bicycle club that would need multiple pages and links.

Sample Constructs, Evidence of Learning, and Assessment Task Features for Scenario-Based Learning Projects. Ability to document system requirements using a simplified use case format; ability to address user needs in specifying system requirements. Presented (more...)

Yarnall noted they encountered a lot of resistance to this approach. Community college students are free to drop a class if they do not like the instructor’s approach, and because many instructors are adjunct faculty, their positions are at risk if their classes are unpopular. Scenario-based learning can be risky, she explained, because it can be demanding, but at the same time students sometimes feel unsure that they are learning enough. Instructors also sometimes feel unprepared to manage the teams, give appropriate feedback, and track their students’ progress.

Furthermore, Yarnall continued, while many of the instructors did enjoy developing the projects, the need to incorporate assessment tools into the projects was the least popular aspect of the program. Traditional assessments in these settings tended to measure recall of isolated facts and technical procedures, and often failed to track the development or application of more complex cognitive skills and professional behaviors, Yarnall explained. She and her colleagues proposed some new approaches, based on the theoretical framework known as evidence-centered design. 5 Their goal was to guide the faculty in designing tasks that would elicit the full range of knowledge and skills they wanted to measure, and they turned to what are called reflection frameworks that had been used in other contexts to elicit complex sets of skills ( Herman, Aschbacher, and Winters, 1992 ).

They settled on an interview format, which they called Evidence-Centered Assessment Reflection, to begin to identify the specific skills required in each field, to identify the assessment features that could produce evidence of specific kinds of learning, and then to begin developing specific prompts, stimuli, performance descriptions, and scoring rubrics for the learning outcomes they wanted to measure. The next step was to determine how the assessments would be delivered and how they would be validated. Assessment developers call this process a domain analysis—its purpose was to draw from the instructors a conceptual map of what they were teaching and particularly how social and social-technical skills fit into those domains.

Based on these frameworks, the team developed assessments that asked students, for example, to write justifications for the tools and procedures they intended to use for a particular purpose; rate their teammates’ ability to listen, appreciate different points of view, or reach a consensus; or generate a list of questions they would ask a client to better understand his or her needs. They used what Yarnall described as “coarse, three-level rubrics” to make the scoring easy to implement with sometimes-reluctant faculty, and have generally averaged 79 percent or above in inter-rater agreement.

Yarnall closed with some suggestions for how their experience might be useful for a K-12 context. She noted the process encouraged thinking about how students might apply particular knowledge and skills, and how one might distinguish between high- and low-quality applications. Specifically, the developers were guided to consider what it would look like for a student to use the knowledge or skills successfully—what qualities would stand out and what sorts of products or knowledge would demonstrate a particular level of understanding or awareness.

Assessing Medical Students’ Interpersonal Skills 6

Filip Lievens described a project conducted at Ghent University in Belgium, in which he and colleagues developed a measure of interpersonal skills in a high-stakes context: medical school admissions. The project began with a request from the Belgian government, in 1997, for a measure of these skills that could be used not only to measure the current capacities of physicians, but also to predict the capacities of candidates and thus be useful for selection. Lievens noted the challenge was compounded by the fact the government was motivated by some negative publicity about the selection process for medical school.

One logical approach would have been to use personality testing, often conducted through in-person interviews, but that would have been very difficult to implement with the large numbers of candidates involved, Lievens explained. A paper on another selection procedure, called “low-fidelity simulation” ( Motowidlo et al., 1990 ), suggested an alternative. This approach is also known as a situational judgment test, mentioned above, in which candidates select from a set of possible responses to a situation that is described in writing or presented using video. It is based on the proposition that procedural knowledge of the advantages and disadvantages of possible courses of action can be measured, and that the results would be predictive of later behaviors, even if the instrument does not measure the complex facets that go into such choices. A sample item from the Belgian assessment, including a transcription of the scenario and the possible responses, is shown in Box 3-3 . In the early stages of the project, the team used videotaped scenarios, but more recently they have experimented with presenting them through other means, including in written format.

Sample Item from the Situational Judgment Test Used for Admissions to Medical School in Belgium. Situation: Patient: So, this physiotherapy is really going to help me?

Lievens noted a few differences between medical education in Belgium and the United States that influenced decisions about the assessment. In Belgium, prospective doctors must pass an admissions exam at age 18 to be accepted for medical school, which begins at the level that for Americans is the less structured 4-year undergraduate program. The government-run exam is given twice a year to approximately 4,000 students in total, and it has a 30 percent pass rate. Once accepted for medical school, students may choose the university at which they will study—the school must accept all of the students who select it.

The assessment’s other components include 40 items covering knowledge of chemistry, physics, mathematics, and biology and 50 items covering general cognitive ability (verbal, numerical, and figural reasoning). The two interpersonal skills addressed—in 30 items—are building and maintaining relationships and exchanging information.

Lievens described several challenges in the development of the interpersonal component. First, it was not possible to pilot test any items because of a policy that students could not be asked to complete items that did not count toward their scores. In response to both fast-growing numbers of candidates as well as technical glitches with video presentations, the developers decided to present all of the prompts in a paper-and-pencil format. A more serious problem was feedback they received questioning whether each of the test questions had only one correct answer. To address this, the developers introduced a system for determining correct answers through consensus among a group of experts.

Because of the high stakes for this test, they have also encountered problems with maintaining the security of the test items. For instance, Lievens reported that items have appeared for sale on eBay, and they have had problems with students who took the test multiple times simply to learn the content. Developing alternate test forms was one strategy for addressing this problem.

Lievens and his colleagues have conducted a study of the predictive validity of the test in which they collected data on four cohorts of students (a total of 4,538) who took the test and entered medical school ( Lievens and Sackett, 2011 ). They examined GPA and internship performance data for 519 students in the initial group who completed the 7 years required for the full medical curriculum as well as job performance data for 104 students who later became physicians. As might be expected, Lievens observed, the cognitive component of the test was a strong predictor, particularly for the first years of the 7-year course, whereas the interpersonal portion was not useful for predicting GPA (see Figure 3-2 ). However, Figure 3-3 shows this component of the test was much better at predicting the students’ later performance in internships and in their first 9 years as practicing physicians.

Correlations between cognitive and interpersonal components (situational judgment test, or SJT) of the medical school admission test and medical school GPA. SOURCE: Filip Lievens’ presentation. Used with permission.

Correlations between the cognitive and interpersonal components (situational judgment test, or SJT) of the medical school admission test and internship/job performance. SOURCE: Filip Lievens’ presentation. Used with permission.

Lievens also reported the results of a study of the comparability of alternate forms of the test. The researchers compared results for three approaches to developing alternate forms. The approaches differed in the extent to which the characteristics of the situation presented in the items were held constant across the forms. The correlations between scores on the alternate forms ranged from .34 to .68, with the higher correlation occurring for the approach that maintained the most similarities in the characteristics of the items across the forms. The exact details of this study are too complex to present here, and the reader is referred to the full report ( Lievens and Sackett, 2007 ) for a more complete description.

Lievens summarized a few points he has observed about the addition of the interpersonal skills component to the admissions test:

• While cognitive assessments are better at predicting GPA, the assessments of interpersonal skills were superior at predicting performance in internships and on the job.
• Applicants respond favorably to the interpersonal component of the test—Lievens did not claim this component is the reason but noted a sharp increase in the test-taking population.
• Success rates for admitted students have also improved. The percentage of students who successfully passed the requirements for the first academic year increased from 30 percent, prior to having the exam in place, to 80 percent after the exam was installed. While not making a causal claim, Lievens noted that the increased pass rate may be due to the fact that universities have also changed their curricula to place more emphasis on interpersonal skills, especially in the first year.

Assessment Centers 8

Lynn Gracin Collins began by explaining what an assessment center is. She noted the International Congress on Assessment Center Methods describes an assessment center as follows 9 :

a standardized evaluation of behavior based on multiple inputs. Several trained observers and techniques are used. Judgments about behavior are made, in major part, from specifically developed assessment simulations. These judgments are pooled in a meeting among the assessors or by a statistical integration process. In an integration discussion, comprehensive accounts of behavior—and often ratings of it—are pooled. The discussion results in evaluations of the assessees’ performance on the dimensions or other variables that the assessment center is designed to measure.

She emphasized that key aspects of an assessment center are that they are standardized, based on multiple types of input, involve trained observers, and use simulations. Assessment centers had their first industrial application in the United States about 50 years ago at AT&T. Collins said they are widely favored within the business community because, while they have guidelines to ensure they are carried out appropriately, they are also flexible enough to accommodate a variety of purposes. Assessment centers have the potential to provide a wealth of information about how someone performs a task. An important difference with other approaches is that the focus is not on “what would you do” or “what did you do”; instead, the approach involves watching someone actually perform the tasks. They are commonly used for the purpose of (1) selection and promotion, (2) identification of training and development needs, and (3) skill enhancement through simulations.

Collins said participants and management see them as a realistic job preview, and when used in a selection context, prospective employees actually experience what the job would entail. In that regard, Collins commented it is not uncommon for candidates—during the assessment—to “fold up their materials and say if this is what the job is, I don’t want it.” Thus, the tasks themselves can be instructive, useful for experiential learning, and an important selection device.

Some examples of the skills assessed include the following:

• Interpersonal : communication, influencing others, learning from interactions, leadership, teamwork, fostering relationships, conflict management
• Cognitive : problem solving, decision making, innovation, creativity, planning and organizing
• Intrapersonal : adaptability, drive, tolerance for stress, motivation, conscientiousness

To provide a sense of the steps involved in developing assessment center tasks, Collins laid out the general plan for a recent assessment they developed called the Technology Enhanced Assessment Center (TEAC). The steps are shown in Box 3-4 .

Steps involved in Developing the Technology Enhanced Assessment Center. SOURCE: Adapted from presentation by Lynn Gracin Collins. Used with permission.

Assessment centers make use of a variety of types of tasks to simulate the actual work environment. One that Collins described is called an “inbox exercise,” which consists of a virtual desktop showing received e-mail messages (some of which are marked “high priority”), voice messages, and a calendar that includes some appointments for that day. The candidate is observed and tracked as he or she proceeds to deal with the tasks presented through the inbox. The scheduled appointments on the calendar are used for conducting role-playing tasks in which the candidate has to participate in a simulated work interaction. This may involve a phone call, and the assessor/observer plays the role of the person being called. With the scheduled role-plays, the candidate may receive some information about the nature of the appointment in advance so that he or she can prepare for the appointment. There are typically some unscheduled role-playing tasks as well, in order to observe the candidate’s on-the-spot performance. In some instances, the candidate may also be expected to make a presentation. Assessors observe every activity the candidate performs.

Everything the candidate does at the virtual desktop is visible to the assessor(s) in real time, although in a “behind the scenes” manner that is blind to the candidate. The assessor can follow everything the candidate does, including what they do with every message in the inbox, any responses they make, and any entries they make on the calendar.

Following the inbox exercise, all of the observers/assessors complete evaluation forms. The forms are shared, and the ratings are discussed during a debriefing session at which the assessors come to consensus about the candidate. Time is also reserved to provide feedback to the candidate and to identify areas of strengths and weaknesses.

Collins reported that a good deal of information has been collected about the psychometric qualities of assessment centers. She characterized their reliabilities as adequate, with test-retest reliability coefficients in the .70 range. She said a wide range of inter-rater reliabilities have been reported, generally ranging from .50 to .94. The higher inter-rater reliabilities are associated with assessments in which the assessors/raters are well trained and have access to training materials that clearly explain the exercises, the constructs, and the scoring guidelines. Providing behavioral summary scales, which describe the actual behaviors associated with each score level, also help the assessors more accurately interpret the scoring guide.

She also noted considerable information is available about the validity of assessment centers. The most popular validation strategy is to examine evidence of content validity, which means the exercises actually measure the skills and competencies that they are intended to measure. A few studies have examined evidence of criterion-related validity, looking at the relationship between performance on the assessment center exercises and job performance. She reported validities of .41 to .48 for a recent study conducted by her firm ( SH&A/Fenestra, 2007 ) and .43 for a study by Byham (2010) . Her review of the research indicates that assessment center results show incremental validity over personality tests, cognitive tests, and interviews.

One advantage of assessment center methods is they appear not to have adverse impact on minority groups. Collins said research documents that they tend to be unbiased in predictions of job performance. Further, they are viewed by participants as being fairer than other forms of assessment, and they have received positive support from the courts and the Equal Employment Opportunity Commission (EEOC).

Assessment centers can be expensive and time intensive, which is one of the challenges associated with using them. An assessment center in a traditional paradigm (as opposed to a high-tech paradigm) can cost between $2,500 and $10,000 per person. The features that affect cost are the number of assessors, the number of exercises, the length of the assessment, the type of report, and the type of feedback process. They can be logistically difficult to coordinate, depending on whether they use a traditional paradigm in which people need to be brought to a single location as opposed to a technology paradigm where much can be handled remotely and virtually. The typical assessment at a center lasts a full day, which means they are resource intensive and can be difficult to scale up to accommodate a large number of test takers.

Lievens mentioned but did not show data indicating (1) that the predictive validity of the interpersonal items for later performance was actually greater than the predictive validity of the cognitive items for GPA, and (2) that women perform slightly better than men on the interpersonal items.

See http://www7 ​.national-academies ​.org/bota/21st ​_Century_Workshop_Salas_Fiore_Paper ​.pdf [August 2011].

See http://www ​.envisionschools.org/site/ [August 2011] for additional information about Envision Schools.

Lenz’s presentation is available at http://www7 ​.national-academies ​.org/bota/21st ​_Century_Workshop_Lenz.pdf [August 2011].

Yarnall’s presentation is available at http://www7 ​.national-academies ​.org/bota/21st ​_Century_Workshop_Yarnall.pdf [August 2011].

See Mislevy and Risconscente (2006) for an explanation of evidence-centered design.

Lievens’ presentation is available at http://www7 ​.national-academies ​.org/bota/21st ​_Century_Workshop_Lievens.pdf [August 2011].

Collins’ presentation is available at http://www7 ​.national-academies ​.org/bota/21st ​_Century_Workshop_Collins.pdf [August 2011].

See http://www ​.assessmentcenters ​.org/articles/whatisassess1.asp [July 2011].

• Cite this Page National Research Council (US) Committee on the Assessment of 21st Century Skills. Assessing 21st Century Skills: Summary of a Workshop. Washington (DC): National Academies Press (US); 2011. 3, Assessing Interpersonal Skills.
• PDF version of this title (1003K)

In this Page

Other titles in this collection.

• The National Academies Collection: Reports funded by National Institutes of Health

Recent Activity

• Assessing Interpersonal Skills - Assessing 21st Century Skills Assessing Interpersonal Skills - Assessing 21st Century Skills

Your browsing activity is empty.

Activity recording is turned off.

Turn recording back on

Connect with NLM

National Library of Medicine 8600 Rockville Pike Bethesda, MD 20894

Web Policies FOIA HHS Vulnerability Disclosure

Help Accessibility Careers

Numbers, Facts and Trends Shaping Your World

Read our research on:

Full Topic List

Regions & Countries

• Publications
• Our Methods
• Short Reads
• Tools & Resources

Read Our Research On:

• The Future of Jobs and Jobs Training
• Theme 2: Learners must cultivate 21st‑century skills, capabilities and attributes

Table of Contents

• About this canvassing of experts
• Theme 1: The training ecosystem will evolve, with a mix of innovation in all education formats
• Theme 3: New credentialing systems will arise as self‑directed learning expands
• Theme 4: Training and learning systems will not meet 21st‑century needs by 2026
• Theme 5: Jobs? What jobs? Technological forces will fundamentally change work and the economic landscape
• Acknowledgments

Will training for the skills likely to be most important in the jobs of the future work be effective in large-scale settings by 2026? Respondents in this canvassing overwhelmingly said yes, anticipating improvements in such education will continue. However, when respondents answered the question, “Which of these skills can be taught effectively via online systems?” most generally listed a number of “hard skills” such as fact-based knowledge or step-by-step processes such as programming or calculation – the types of skills that analysts say machines are taking over at an alarming pace right now. And then, when asked, “What are the most important skills needed to succeed in the workplace of the future?” while some respondents mentioned lessons that might be taught in a large-scale setting (such as understanding how to partner with AI systems or how use fast-evolving digital tools) most concentrated on the need for “soft skills” best developed organically, mentioning attributes such as adaptability, empathy, persistence, problem-solving, conflict resolution, collaboration and people skills, and critical thinking.

Tough-to-teach intangibles such as emotional intelligence, curiosity, creativity, adaptability, resilience and critical thinking will be most highly valued

Learning will, in itself, become important. The skill to continue to learn will be important in all jobs Anonymous respondent

[how to use]

Overall, as these respondents foresee a big re-sorting of workplace roles for machines and humans, they expect that the jobs-related training systems of the future will often focus on adding or upgrading the particular capabilities humans can cultivate that machines might not be able to match.

An anonymous respondent ’s terse description of top future skills was echoed by many dozens of others in this study: “Learning will, in itself, become important. The skill to continue to learn will be important in all jobs.”

Susan M ernit , CEO and co-founder at Hack the Hood, explained, “At Hack the Hood, the tech-inclusion nonprofit I lead, the most valuable skill we teach low-income young people of color, ages 16-25, is that they have the ability and the discipline to learn harder and harder things – the most critical skill for the emerging workplace. Research shows that for our cohorts a blend of online and real-world learning is an effective mix.”

George McKee , a retiree, predicted, “As always, the most important skills will be the ability to learn and organize new things and to discriminate sense from nonsense. Public schools will continue to fall behind in their ability to foster these skills in large populations.”

Meryl Krieger , career specialist at Indiana University, Bloomington’s Jacobs School, replied, “The most important skills in the workforce of the future are 1) transferrable skills and 2) training in how to contextualize and actually transfer them. These are really hard to teach at scale, but then the workforce of the future is something we are barely coming to have the dimmest perceptions about.”

Jessica Vitak , an assistant professor at the University of Maryland, observed that the most-needed skills are capabilities that have always had value, writing, “As much as people like to imagine the future being heavily reliant on robots and high-tech gadgets, I don’t see too much of the workforce shifting dramatically in terms of the skills required to complete tasks.”

Many of the skills of the future are hybrid skills – requiring expertise or fluency across some of our traditional domains. Trevor Hughes

Many other participants in this study said highly valued strengths of human character will be necessary to partner with technology in jobs of the future. An anonymous respondent wrote, “The increasing reach of data, automation and eventually AI will force those jobs that remain to require even greater human touch.”

Susan Price , a digital architect at Continuum Analytics, expanded on that point, explaining, “People will continue to prefer and increasingly value human nurses, teachers, writers, artists, counselors, ethicists and philosophers. This shift has been apparent over the past 20 years or so. As we have come to prefer ATMs over tellers and travel apps over travel agents, our patronage of other ‘human contact’ specialists such as counselors and therapists, personal trainers, manicurists, and massage therapists has increased. Example: People skills in user interface and experience design will be increasingly in demand, but will greatly benefit from artificial intelligence and machine learning for usability evaluations and testing. Another example: The role of truck drivers will need to evolve as they are replaced with self-driving transports. There will remain the need for humans to manage transportation tracking and auditing, perform problem-solving, and occupy stakeholder contact roles such as sales and customer support communication.”

Trevor Hughes , CEO at the International Association of Privacy Professionals, replied, “Training will indeed be an important part of preparing the workforce for our digital future, but it won’t be easy. Many of the skills of the future are hybrid skills – requiring expertise or fluency across some of our traditional domains. Take privacy as an example. Any digital economy professional needs to understand privacy and how it creates risk for organizations. But that means grasping law and policy, business management, and technology. Modern professionals will need to bridge all of these fields.”

An anonymous respondent replied, “The two trends with the most hype right now are AI and VR. Let’s assume that these technologies will have a large impact on the nature of the future work. The workforce of the future (that is not completely displaced by this tech) then needs the skills to utilize these technologies. Some broad skills I anticipate are interacting with machine learning systems, reasoning with underlying algorithms and embedded judgments, being comfortable delegating tactical decisions to those algorithms, etc.”

Michael Rogers, author and futurist at Practical Futurist, said, “In a rapidly changing work environment populated by many intelligent machines, we will need to train people from an early age in communication skills, problem-solving, collaboration and basic scientific literacy. Without those basics in place, occupational training is insufficient.”

The anonymous director of evaluation and research at a university ranked in the top 10 in the U.S. wrote, “Sure, Lynda.com and Udacity and others that can provide skills, just like the corporate training programs we use now. … But those skills won’t be the same as an education – as the habits of mind and social interleavings that make for the types of problem definition, interdisciplinary perspectives, and incisive thought that will be most needed – deep engagement with the stuff of distinctly human capabilities.”

Justin Reich , executive director at the MIT Teaching Systems Lab, observed, “The most important skills for the future will be the kinds of things that computers cannot readily do, places where human workers have a comparative advantage over computers. Two important domains of human comparative advantage are ill-structured problem solving and complex, persuasive communication. (Frank Levy and Richard Murnane’s ‘ Dancing with Robots ’ offers a nice summary of the research informing this position.) Ironically, computers are most effective at teaching and assessing routine tasks, the kinds of things that we no longer need human beings to do. Large-scale learning, which generally depends on automated assessment, is most effective at teaching the kinds of skills and routine tasks that no longer command a living wage in the labor market.”

An anonymous CEO for a nonprofit technology network argued that some “soft” skills can be taught, observing, “Many research reports have demonstrated that one of the most important skills in our developing workforce is reasoning and complex problem solving. The internet enables us to teach and practice these skills in a unique and appropriate way by connecting and engaging people across geographies, backgrounds, ages, etc.” And an anonymous professor at the University of California, Berkeley said, “I do think there will be a lot more online training in the future – and it will actually be more successful at teaching things that are not directly translatable to jobs (humanities subjects, such as art history, media studies, etc.) – the things that television documentaries are already good at teaching. I’m not sure that great writing skills or public speaking/presentation skills will be taught in this format.”

Alf Rehn , professor and chair of management and organization at Åbo Akademi University in Turku, Finland, responded, “The key thing to realize about skills and the future is that there is no one set of skills that we can identify as core or important. The future of skills is going to be one of continuous change and renewal, and any one special skill we can identify now will almost certainly be outdated in not too long. Creativity and critical thinking will be as important in the future as it is today, but beyond this we should be very careful not to arrogantly assume too much. And this is precisely why new programs, online and off, will be so crucial. Innovative, faster and more agile training systems will not only be helpful, they’ll be critical.”

An anonymous self-described “chief problem solver” said the world needs problem solvers, writing, “Huge portions of the human condition can be effectively learned through one-to-many learning environments enabled through the internet. Many cannot. … If you take a look at the prevalence of strong problem-solving skills in our society now versus 20 years ago, you’ll notice that an overwhelming majority are now quite specialized in their particular areas of interest/work, but on average have less ability than their counterparts 20 years ago to adequately handle new/incongruous/conflicting information or tasks. Instead of figuring it out and thereby training up our ingenuity-focused skills, we now tend to simply Google someone else’s answer. While this is ‘efficient’ in terms of getting to an adequate solution rapidly, it means that … people are not able to handle new inputs, be flexible, or actually puzzle out new problems.”

Many participants mentioned the general categories of communication and people skills. An anonymous respondent summed it up, writing, “No matter what kind of hard skills one comes to the workplace with, at the end of the day things always seem to boil down to people and communication challenges.”

Micah Altman , director of research at MIT Libraries, wrote, “Given the increased rate of technical change and the regular disruptions this creates in established industries, the most important skills for workforces in developed countries are those that support adaptability and which enable workers to engage with new technologies (and especially information and communication technologies) and to effectively collaborate in different organizational structures.”

No matter what kind of hard skills one comes to the workplace with, at the end of the day things always seem to boil down to people and communication challenges. Anonymous respondent

An anonymous technology analyst for Cisco Systems commented, “The gig economy takes over, and micro-skill training will come to the fore. Debate is a most important skill that can be taught online, emphasizing the importance of preparation.”

[there will be]

An anonymous respondent observed, “The job of the future is the one that combines technical, operational, managerial and entrepreneurial skills.”

Axel Bruns , professor in the Digital Media Research Center at Queensland University of Technology, wrote, “Over the past decade there has been a substantial growth in generic digital literacies training, and this is now being replaced or enhanced by literacies training in specific areas and for particular purposes (social media literacy for communication professionals, data literacy for journalists, etc., to name just two particularly obvious fields). There has also been the emergence of a range of specialist positions that address the cutting edge of such literacies – under job titles such as data scientist or computational journalist, for instance. Across the creative industries, and beyond, the possession of such skills will increasingly serve as a differentiator between job applicants, and within organisational hierarchies in the workplace. Those who possess these skills are also more likely to branch out beyond their core disciplines and industries, as many such skills are inherently interdisciplinary and enable the worker to engage in a wider range of activities. Beyond generic digital literacies, some of the key areas I see as important are: 1) platform-specific literacies, e.g., social media literacies; 2) data science, i.e., the ability to gather, process, combine, and analyse ‘big data’ from a range of sources; 3) data visualisation. Until the accreditation schemes for workers with these skills are standardised, which eventually they will be, we will continue to see leading workers in these areas … be able to enter the workplace on the basis of their demonstrated expertise and track record rather than on the basis of formal accreditation. While there are many MOOCs and other online courses now purporting to teach these skills, it is important to point out that there is a substantial qualitative component to these skills – somewhat paradoxically, perhaps, especially where they deal with ‘big data’: the engagement with such large datasets is less about simply generating robust quantitative metrics and more about developing a qualitative understanding of what such metrics actually mean . Such an understanding is difficult to teach through semi-automated online courseware; direct teacher/learner interaction remains crucial here.”

An anonymous vice president of product at a new startup commented, “In a grand folly of correlation being mistaken for causation, we’re trying to pipeline all kids into college to try to juice their earnings, while steering kids away from practical technical skills like manufacturing tech that might be a better fit, opting instead to saddle them with student loans for a degree they won’t finish from a school that no employer will respect.”

The consultant said one more skill that could be most critical: “An important skill for the workforce of the future is an ability to cultivate a strong network, so if your job disappears you’re able to quickly find a new role.”

[to choose]

Practical experiential learning via apprenticeships and mentoring will advance

Several experts wrote about the likelihood that apprenticeship programs will be refashioned offline and online via evolving application of human knowledge and technology tools. An anonymous security engineer at Square commented, “Never before in history has it been so easy for anyone to learn to become anything they want to be, and that will only continue to improve.”

Connecting the virtual to the physical will change everything. Will Kent

Cory Salveson , learning systems and analytics lead at RSM US, predicted, “There will be a big market for this: more self-directed or coached/mentored, project-based, online learning options that coexist with traditional brick-and-mortar university degree credentialing to make the labor market more agile, whether it wants to be or not.”

Will Kent , e-resources librarian at Loyola University-Chicago, replied, “Connecting the virtual to the physical will change everything. Anyone can learn anything online now. With the right kind of career or social positioning/privilege/luck/connections, users can sidestep traditional degree processes. For those in industries that still demand degrees as currency, the requirements for degrees will change, continuing education will become more embedded in the workplace or new types of evaluation will become more popular. Deliverable-based time constraints rather than 9-5, asynchronous offices/projects will be commonplace, and employers will have to make time for employees to self-educate or else they will fall behind. New credentialing systems will complement, not compete with, older iterations. One will not be favored above the other in practice (i.e., if you can do your work, no one will question how you learned what you learned).”

D. Yvette Wohn , assistant professor of information systems at the New Jersey Institute of Technology, wrote, “Knowledge can be acquired through massive online means, but skills will still require a small-group, personalized approach with much individual feedback. In the future, the technology will be advanced such that the modality – online or offline – is not the issue; rather, it is the size and intimacy of the learning environment that will matter. Formalized apprenticeships that require both technical skills and interpersonal interaction will become more important. As more people get degrees, university degrees will matter less, but that does not mean that higher education does not have its place. Schools that are able to provide a more holistic learning experience that does not focus on a specific skill but is able to provide students with an interdisciplinary and social experience will become more valuable.”

John B. Keller , director of e-learning at the Metropolitan School District of Warren Township, Indiana, wrote, “Online training will continue to improve, and … any skills or knowledge updating that can reasonably be delivered online will be. That said, there will still be a need in many areas for verifiable performance of complex skills and behaviors that may not be possible to be accomplished algorithmically. Skills demanded in the future will include analysis of big data sets, interpretation of trends within historic contexts, clear and effective intercultural communication, design and systems thinking, as well as the ability to advance and advocate for distinctly human contributions to progress and the advance of culture. As more and more skills are broken down into repeatable processes, they will be handed off to technology and video as key transfer platforms. The demand for skills that cannot be easily transferred via online systems will ensure that experience, mentorship, coaching, apprenticeship and demonstrated proficiency all have prominent roles to play against a backdrop of online learning.”

An anonymous respondent wrote, “Online classes can teach prerequisite knowledge that can prepare workers for further hands-on training or apprenticeship.”

[I include]

Some respondents believe mentoring does not have to be in person – in the traditional face-to-face sense – to be one-to-one. In fact, they say the online world has already multiplied the number of available mentors in every subject.

Valerie Bock , VCB Consulting, former Technical Services Lead at Q2 Learning, responded, “To develop proficiency, we seem to need exposure to war stories of others who were there when the usual rules didn’t apply. MIT’s … EdX platform has a code checker built in, which means well-structured classes can be created with automatically graded exercises supplemented by discussion forums where students can ask their questions and move past places where they are stuck. These courses actually provide the coaching learners need to become skillful. So yeah, coding is probably a skill that can be taught and credentialed effectively via a self-directed online course. In the meantime, a lot of coders learn their craft informally, by examining code written by others and asking questions about it. To me, the most promising application of the internet is the way it increases the number of potential mentors. Global organizations are already leveraging the asynchronous properties of online venues to put their subject matter experts in touch with mentees half the world away, spanning time and distance obstacles. … People use the internet every day, informally, to learn bits and pieces that help them be more effective in the work (paid and unpaid) they do, sometimes by accessing content, but often by contacting other people. The value added to human welfare by parenting forums, elder care discussions, recipe exchanges, addiction recovery communities and even stain removal resources is deeply underestimated.”

Ed Dodds , digital strategist at Conmergence, added, “The global startup ecosystem and makerspace ecosystem will both be intersecting and growing in parallel. … More intentional formal mentorship networks (guilds) are likely to proliferate.”

“What would help,” added an anonymous respondent , “is improving people’s efficiency at providing supports to others. Sort of Slack on steroids.”

Sign up for our weekly newsletter

Fresh data delivery Saturday mornings

Sign up for The Briefing

Weekly updates on the world of news & information

• Business & Workplace
• Digital Divide
• Education & Learning Online
• Future of the Internet (Project)
• Future of Work

A look at small businesses in the U.S.

A look at black-owned businesses in the u.s., 2023 saw some of the biggest, hardest-fought labor disputes in recent decades, do you tip more or less often than the average american, diversity, equity and inclusion in the workplace, most popular.

901 E St. NW, Suite 300 Washington, DC 20004 USA (+1) 202-419-4300 | Main (+1) 202-857-8562 | Fax (+1) 202-419-4372 |  Media Inquiries

Research Topics

• Email Newsletters

ABOUT PEW RESEARCH CENTER  Pew Research Center is a nonpartisan fact tank that informs the public about the issues, attitudes and trends shaping the world. It conducts public opinion polling, demographic research, media content analysis and other empirical social science research. Pew Research Center does not take policy positions. It is a subsidiary of  The Pew Charitable Trusts .

© 2024 Pew Research Center

  University of Dar es Salaam Library Journal Journal / University of Dar es Salaam Library Journal / Vol. 19 No. 1 (2024) / Articles (function() { function async_load(){ var s = document.createElement('script'); s.type = 'text/javascript'; s.async = true; var theUrl = 'https://www.journalquality.info/journalquality/ratings/2408-www-ajol-info-udslj'; s.src = theUrl + ( theUrl.indexOf("?") >= 0 ? "&" : "?") + 'ref=' + encodeURIComponent(window.location.href); var embedder = document.getElementById('jpps-embedder-ajol-udslj'); embedder.parentNode.insertBefore(s, embedder); } if (window.attachEvent) window.attachEvent('onload', async_load); else window.addEventListener('load', async_load, false); })();  

Article sidebar.

Article Details

Copyright for articles published in this journal is retained by the journal.

The content is licensed unader a  Creative Commons  Attribution-NonCommercial-NoDerivs (CC BY-NC-ND) License.

Main Article Content

Teachers’ perception of digital literacy skills as a tool for 21st century teaching in nigeria, rachael ojima agarry, moses opeyemi babalola, priscilla ayomide jacob.

The study examined teachers’ perception of digital literacy skills as a relevant tool for teaching in the 21 st Century in Ilorin South Local Government Area of Kwara State. The descriptive research design was adopted. One research question and four research hypotheses were generated. The sampling technique was simple random sampling, selecting 200 teachers as respondents. The instrument used for data collection was a self-constructed questionnaire. Its reliability was determined using Cronbach’s alpha with 0.85 co-efficient. Descriptive and inferential statistical tools were employed in answering and testing the generated research question and the hypotheses respectively, using mean and standard deviation, independent t-test and ANOVA. The findings showed that teachers perceived digital literacy skills as a relevant tool for teaching in the 21st Century; there is a significant difference in the perception of teachers towards digital literacy skills for teaching in the 21 st century based on gender and age while there is no significant difference in the perception of teachers towards digital literacy skills for teaching in the 21st Century based on educational qualification and years of teaching experience in Ilorin South. Recommendations were made that school managements should further encourage teachers’ digital literacy skills by organizing or sponsoring them to attend training on how to integrate e-learning tools into their classroom instructions; all age groups and both genders especially female teachers should be encouraged to acquire digital literacy skills among others.

AJOL is a Non Profit Organisation that cannot function without donations. AJOL and the millions of African and international researchers who rely on our free services are deeply grateful for your contribution. AJOL is annually audited and was also independently assessed in 2019 by E&Y.

Your donation is guaranteed to directly contribute to Africans sharing their research output with a global readership.

• For annual AJOL Supporter contributions, please view our Supporters page.

Journal Identifiers

21st Century Site Supervisors for 2024-2025

Job posting for 21st century site supervisors for 2024-2025 at brockton public schools.

Apply for this job

Receive alerts for other 21st Century Site Supervisors for 2024-2025 job openings

Report this Job

Sign up to receive alerts about other jobs that are on the 21st Century Site Supervisors for 2024-2025 career path.

Click the checkbox next to the jobs that you are interested in.

Sign up to receive alerts about other jobs with skills like those required for the 21st Century Site Supervisors for 2024-2025 .

Benefits Program Administration Skill

• Leave of Absence Administrator III Income Estimation: $83,693 - $108,836
• Benefits Analyst III Income Estimation: $89,068 - $110,471

Bug/Defect Analysis Skill

• Mobile Applications Developer Income Estimation: $99,603 - $128,597
• Midrange Programmer IV Income Estimation: $101,469 - $136,426

Job openings at Brockton Public Schools

Not the job you're looking for here are some other 21st century site supervisors for 2024-2025 jobs in the brockton, ma area that may be a better fit., we don't have any other 21st century site supervisors for 2024-2025 jobs in the brockton, ma area right now..

Home Health Aide

21ST CENTURY MANAGEMENT SERVICES , Canton, MA

Service Associate / Shift Supervisors - 021524

The UPS Store Franklin #0738 , Franklin, MA