The Story of a Learning College: The Evolution of Outcomes Assessment at LaGuardia

Print Friendly

LaGuardia Community College’s approach to outcomes assessment  has become a national model for using ePortfolios as a basis for assessment of authentic student learning. LaGuardia has made tangible progress in advancing student learning, shaping curricula, and creating a notable culture of faculty-driven assessment over the past ten years using a deliberate, developmental, evolving process to change the college culture around assessment to ensure and deepen learning across the institution. The use of ePortfolios in the assessment process was first proposed in 2001 as a way to gain a more comprehensive understanding of what students learned during the course of their tenure at the institution. Most recently, LaGuardia’s pervasive culture of assessment was highlighted by LaGuardia’s accrediting body (Middle States Commission on Higher Education). Our method focuses on utilizing ePortfolio for collection and demonstrates the value of engaged faculty input into the outcomes assessment design to continually close the assessment loop.  

Setting the Stage

LaGuardia’s assessment effort is multilayered, targeting student learning in the program/major and institution-wide. The process engages students, faculty, and staff in a program and institutional examination of student learning. Faculty anchor both processes, with ePortfolio as the central collection mechanism for artifacts which are then reviewed anonymously against carefully developed rubrics.

Program-wide Assessment:

Outcome_assessment_1_editEvery five years, programs/majors at LaGuardia undergo Periodic Program Review (PPR). As part of this review, faculty teams from each department score student work that has been collected for key courses in the ePortfolio Assessment Database. The PPR process asks faculty to assess student work against programmatic competencies, developed by each program in accordance with best practices and standards in their disciplines and any applicable external accrediting bodies. Teams also review student work in established core competencies using institutional rubrics. As faculty in a program/major complete the PPR process, they can apply for an ePortfolio-Assessment Mini-Grant from the College’s Center for Teaching and Learning. The Mini-Grants support department/program/major-based professional development activities that help faculty to integrate recommendations highlighted by a PPR to promote program-wide pedagogical and curricular improvement. This process helps to close the loop in the assessment cycle, moving program faculty from inquiry, analysis, and reflection to the integration of new approaches designed to facilitate faculty development and improve student learning across courses and over time.

College-wide Assessment:

In 2011, the first Benchmark Assessment teams were brought together to read student work demonstrating six of the college’s seven core competencies: critical thinking, writing, and reading (critical literacy); quantitative literacy; research and information literacy; and oral communication. The goal was to enrich the assessment data by looking at general education competencies outside of individual majors, and also to augment the findings from PPRs. The PPR process provides a rich source of assessment data for the college, but it does not offer a comprehensive overview of general education because it is focused on student learning within a program. The Benchmark Assessment process generates rich, rewarding, and productive interdisciplinary conversations around the college’s core competencies, as faculty from varying disciplines articulate their understandings of student competencies. The assessment readings provide important evidence of student learning (see graph below), and prove to be an enormously valuable occasion for shared faculty exploration into the general education competencies and how those competencies play out in different courses and degree programs. Based on the initial success of these readings, they have become a routine practice each academic year.

Developmental Story

LaGuardia’s accrediting body, Middle States, selected the college’s 2012 self-study to serve as one of the model reports sent to participating institutions, but this honor reflects a sustained and still-evolving process that has taken over a decade.

The period from 2001 to 2005 served as foundational years for putting in place key elements of LaGuardia’s assessment plan. In these years, the college first piloted ePortfolio, developed campus-specific rubrics around the core competencies adapted from the AAC&U VALUE rubrics, and began a deliberate process of professional development activity to help faculty consider how the core competencies manifested in their own courses. This emphasis on competencies, combined with the reflective and expressive features of student-built ePortfolios, helped to build engagement and improve student success. It also supported the growth of an assessment culture through the college’s ePortfolio system, which serves a dual function by allowing for the collection and analysis of student artifacts in a database for assessment against programmatic and core competencies.

This student work is the foundation for the college’s direct confirmation of student learning. Spurred by focused outreach aimed at faculty, LaGuardia had collected 3,465 artifacts through the 2007-08 academic year, a figure that grew to 23,176 by 2010-11 and is now approaching 40,000. Not only is the body of collected work vast, but what it reveals is quite positive. A recent assessment of student learning outcomes in general education competencies across all majors (see graph below) revealed that LaGuardia students are making real progress with an average increase of 0.87 across all rubrics. Although such student learning gains are impressive and statistically significant, the college is far from satisfied with the results and continues to make efforts for further improvement. The following graph, showing the results of the 2011 benchmark reading of over 3,000 student artifacts, compared students with 25 or fewer credits with students holding 45 or more credits. The graph is excerpted from LaGuardia’s 2012 accreditation self-study:

middlestates-graph

Selected staff play a key support role in this process, and upper level academic leaders are supportive and attentive, but faculty are truly the key actors in outcomes assessment at LaGuardia, designing the assignments, gathering and reviewing the artifacts, discussing the implications of their findings, and implementing evidence-based plans for improvement. One hallmark of LaGuardia’s general education assessment program is the commitment of faculty and administrators to using rubrics developed by teams of faculty to look at authentic student work—work that is created by students, in actual courses, connected to their majors. LaGuardia continually asks its faculty: What do we want students to learn, why, and how can we measure their learning?

This process also positions faculty to take what they have learned from assessment and to make corresponding pedagogical and curricular changes—―closing the loop—with appropriate implementation based on outcomes assessment findings. For example, the Physical Therapist Assistant, A.A.S. degree program began by examining the core competencies of critical literacy and communication. PTA faculty had learned from reviewing their students’ artifacts that they were generally lacking in these areas. After subsequent reexamination of both individual assignments and the sequence of courses in their major, faculty considered ways that they might better enable students to develop these skills. The program’s faculty discovered during this curriculum mapping activity that two main courses could be redesigned to more fully address these competencies. This inquiry and integration of their curriculum was detailed in the department’s recent Periodic Program Review (PPR), and the program will reassess the impact of the changes during its next PPR in 2015-16.

LaGuardia’s Center for Teaching and Learning (CTL) has provided key support for the collection of assessment artifacts for over a decade. Initially, the CTL was not directly involved in the outcomes assessment process, and focused its ePortfolio work primarily on the kinds of instructional pedagogies that would directly benefit students. However, as faculty development on the use of ePortfolio to enhance student learning inevitably overlapped with the goals of outcomes assessment, the role of the CTL in assessment became more pronounced and the two aims came into greater balance. During this time, the CTL leadership has also steered a substantial investment of resources to the effort through grant writing and management of institutional resources. The post-PPR Assessment Mini-Grant Program is the key vehicle for CTL support of outcomes assessment. During the 2013-14 academic year, mini-grants are supporting work in Math basic skills, Natural Sciences, Physical Therapy Assistant (PTA), and Urban Studies programs. ePortfolio is a key dimension of this “loop-closing” curricular work in the PTA and Natural Sciences areas.

LaGuardia has been recognized by the National Institute for Learning Outcomes Assessment (NILOA) as a nationwide “best practice” institution for the degree to which it has woven inquiry, reflection, and integration for outcomes assessment into our institutional fabric. See the NILOA report here. Indeed, the principles of inquiry, reflection, and integration are central to LaGuardia’s assessment philosophy. At the college, assessment is about:

  • Inquiry: Asking key questions as they relate to student engagement and learning, faculty development, and the involvement of faculty in the assessment process.
  • Reflection: Supporting faculty as they examine artifacts of learning, reflect on the teaching and learning process, and consider changes in pedagogy and curriculum needed to close the gap between what students already know and what they need to learn.
  • Integration: Designing and implementing experiences that advance student learning and prompt students to look beyond a single course, a major, or a discipline to develop a holistic perspective on their educations.

LaGuardia’s commitment to ePortfolio, reflected in many years of cultivating ePortfolio pedagogy, has created a strong, widely accepted base for building these principles into outcomes assessment.

LaGuardia has made significant progress in building a campus culture which values outcomes assessment. Assessment work is by its nature cyclical, and is in many ways complicated by the realities of working in a resource-poor, urban community college where faculty teach a nine-course load and where policies from the central university administration sometimes hinder local campus efforts. Moreover, while ePortfolio is a key instrument used to facilitate this work, we still face the challenge of moving beyond the single-competency model and toward assessing ePortfolios holistically for integrative learning. The recommendation to redesign a First-Year Experience built on a new credit-bearing First-Year Seminar and grounded in the competencies of problem-solving, inquiry, and integration (cited by LaGuardia’s First-Year Experience Task Force) supports the use of ePortfolio in the first year to connect curricular with co-curricular and advisement processes, and lays a critical and strong foundation to assess students’ longitudinal growth. This is a key step towards assessing full ePortfolios for integrative learning, and reflects one part of the next institutional chapter in the connected stories of ePortfolio pedagogy and outcomes assessment. The college is deeply grateful to its colleagues from Virginia Tech for demonstrating the viability of this innovative approach and generously helping to develop our plans.

GUIDING PRINCIPLES

Our faculty and staff have taken the following steps around ePortfolio based Outcomes Assessment in relation to Inquiry, Reflection and Integration.

Inquiry:

  • Designing and implementing activities or assignments that will generate artifacts of student learning for outcomes assessment
  • Facilitating the collection of artifacts
  • Working with other faculty and staff to examine artifacts of student learning and rate them again rubrics for outcomes assessment

Reflection:

  • Discussing the insights generated by artifact review with other faculty and staff and considering the implications for courses and programs
  • Developing recommendations for change, based on the review of artifacts
  • Sharing findings and recommendations with other segments of the college

Integration:

  • Planning/guiding the implementation of recommendations for change in curriculum and pedagogy
  • Taking part in professional development related to implementation of recommendations for change
  • Implementing new curriculum and pedagogy based on the process of assessment and recommendations for improvement

Challenges:

  • Closing the Loop – moving from assessment to change in pedagogy and practice
  • Building faculty/staff understanding of the ways more authentic and holistic Outcomes Assessment can effectively support efforts to improve student learning
  • Helping faculty/staff understand how ePortfolio can support more authentic and holistic approaches to Outcomes Assessment
  • Overcoming faculty/staff anxiety about Outcomes Assessment
  • Developing assignments aligned with designated competencies
  • Using Outcomes Assessment to build a campus culture focused on student learning general
  • Engaging faculty/staff in the process of assessing artifacts of student learning
  • Identifying the competencies and developing the rubrics needed to evaluate artifacts of student learning for Outcomes Assessment
  • Adapting the ePortfolio platform to support our campus’ approach to outcomes assessment
  • Gathering a meaningful collection of artifacts or ePortfolios to use for Outcomes Assessment
  • Gaining the support of key campus stakeholders for the use of ePortfolio for Outcomes Assessment

What’s Next?

The next chapter in LaGuardia’s outcomes assessment efforts will build on another major foundation as well. In the summer of 2013, and in response to our accreditor’s recommendations as well as to university- and campus-based initiatives, the college convened a task force to develop new General Education competencies and a new plan for Gen Ed assessment. The task force engaged a broad range of faculty and staff in an extended conversation, guided by Provost Paul Arcario’s two-fold charge: to avoid simply adding more competencies to the existing ones, and to rethink, synthesize, and look to articulate competencies which will challenge the whole campus community to move student learning to a higher level.

The result of this “Learning Matters” exercise was an extraordinarily rich campus-wide conversation and a reconceived General Education structure, thoroughly vetted by all campus stakeholders and approved by college governance. The new structure features three core competencies: Inquiry and Problem-Solving, Global Learning, and Integrative Learning. These will be carefully mapped to curricula, integrated into course work, and assessed utilizing rubrics that faculty will develop for the purpose. The competencies will be demonstrated by students in one of three communicative abilities: Written, Oral, and Digital. Faculty in programs will determine which abilities students will use to demonstrate each competency, and when they will do so during their course of study.

The new General Education competencies promise to strengthen a LaGuardia education by emphasizing and solidifying underlying principles which have guided our ePortfolio work and offer powerful cross-disciplinary and interdisciplinary learning opportunities. Inquiry and Problem-Solving represent, respectively, a process and the product of sophisticated intellectual work. To that end, the competency will enable our faculty to guide students toward posing informed questions, finding and evaluating relevant information, and applying their research to a meaningful problem. Global Learning will present opportunities to explore and reflect on values, ethics, and diversity across a similarly broad range of contexts, whether intellectual, social, political, or professional. Finally, structuring opportunities for Integrative Learning will enable students to forge connections across disciplines, between academic learning and life experience, and between the classroom and co-curricular activities. And not least, an integrative orientation to General Education will encourage the same kind of connections across time, allowing students to see, and to reflect upon, their own progress.

LaGuardia has long experience in assessing oral and written communication, and these will remain central to our General Education structure. But the Learning Matters conversation, as well as a Middle States recommendation to enhance LaGuardia students’ technological literacy, engendered a new emphasis on digital communication as an essential 21st century ability. The key argument, advanced by more than one Learning Matters participant, focused on students becoming creators of digital content and not merely consumers, and the critical importance of learning to be fluent and responsible digital authors in a world where such skills now sit at the center of both culture and economy. The prospect of linking this communicative ability to inquiry and problem-solving activities has already energized the thinking of LaGuardia faculty.

The new General Education competencies, like the first-year initiatives discussed above, represent the LaGuardia present and a rapidly emerging future. Their focus on broadly defined higher-order skills and abilities will allow us to build on the good work already accomplished, while reflecting our aspirations, as articulated in the College Mission Statement: “LaGuardia Community College’s mission is to education and graduate one of the most diverse student populations in the country to become critical thinkers and socially responsible citizens who help shape a rapidly evolving society.” Our outcomes assessment processes, updated to include the new competencies, will help us to meet these goals.

The following timeline reflects the recent evolution of LaGuardia’s outcomes assessment efforts:

TImeline

Read More:

  • LaGuardia’s Self-Study Report on the Assessment of Student Learning for 2012 Middle States Accreditation Review
  • Assessment appendices for LaGuardia’s Self-Study Report on the Assessment of Student Learning for 2012 Middle States Accreditation Review (beginning on page 129)
  • Brief Assessment Version for C2L Meeting on LaGuardia’s evolving assessment story.
  • VALUE Ingration
  • LaGuardia’s Assessment today: “Closing the Loop: How We Better Serve Our Students through a Comprehensive Assessment Process” by Paul Arcario, Bret Eynon, Marisa Klages, and Bernard A. Polnariev
  • Assessment at LaGuardia: Outcomes Assessment Website at LaGuardia (including core competencies, history, rubrics, and feedback loop)
  • Initial Report from the first Benchmark Assessment Reading
  • Assessment Presentation for IUPUI Assessment Conference IUPUI10 (2010)
Skip to toolbar