Holistic Evaluation of ePortfolios through Rubrics: A Professional Development Exercise

Print Friendly

This activity guides faculty through a multi-stage process of collaboratively brainstorming, creating, testing, and revising rubrics for holistic evaluation of student ePortfolios.

Description

holistic_evaluation_1_622x415This activity aims to draw faculty attention to the integrative qualities of LaGuardia student ePortfolios and to stimulate pedagogies that make integration routine for students.

The steps of the activity are as follows:

Day 1

Preliminary: A PowerPoint presentation on the value and mechanics of rubrics presented by one of the seminar leaders

Step 1: Constructing your own ePortfolio rubric

Working collaboratively with your partners, please discuss and then develop a single rubric for holistic evaluation of ePortfolios that could potentially be implemented in your courses. You might first think about the content included in and competencies evinced by students’ ePortfolios. Then you will need to determine together the scale (descriptive? point-based?), the dimensions (categories) for the evaluation, and the criteria descriptions (i.e., what separates the highest-possible score from the next-highest, and so on). Please use the paper and markers provided to display your rubric for presentation later.

Step 2: Brief reports on group rubrics (10 minutes/each)

Day 2

Step 1: Scoring

Working individually, review the ePortfolio belonging to LaGuardia student XXXXXXXX [note: the subject ePortfolio belongs to a currently enrolled student and is shared publicly within the college’s network] and then score it using the evaluation rubric developed by your group. When you have finished, score the same ePortfolio a second time using the evaluation rubric developed by the group immediately following your own. The purpose of this exercise is to test the effectiveness of each rubric, so take note of any areas where the two rubrics might warrant revision or additional work. (20 min.)
Step 2: Group norming & evaluation

With the members of your original group, compare the scores you assigned to the student ePortfolio first using your own group’s rubric, then the other rubric you used. Feel free to devote some time to discussing the merits of the ePortfolio itself and to where your scores do or do not align, but be sure that your conversation eventually turns to each respective rubric’s effectiveness as an evaluation tool. Appoint a member of each group to record your findings. (40 min.)
Step 3: Reports

We will reconvene as a whole group to share your findings. (15 min.)

Homework (completed outside of seminar time)
Step 4: Revision

Following today’s meeting, we will post images of your collaborative ePortfolio rubric drafts to the seminar eP hub. With your fellow group members, locate and then complete (if you have not already done so to your satisfaction) your rubric. Then, using Office or Google Docs applications, or an online rubric generator, like iRubric, produce a digital version of your rubric for presentation on your individual ePortfolios. Each group member should post the same rubric to his or her own eP.

Step 5: Reflection

With the finished digital version of your rubric, post a reflection to your seminar ePortfolio in which you comment on the process of building it, the changes you made between drafts, and what you learned.

Type of Professional Development

This practice is a discrete activity that spans across two monthly seminar meetings in the context of a full-year PD seminar. The Connected Learning: ePortfolio & Integrative Pedagogy seminar has staged versions of this activity twice before, with a third iteration scheduled for the current cohort in spring 2013. While the activity’s core–the collaborative building of eP rubrics–has remained the same, the seminar leaders have experimented with extending the activity to allow more time for discussion and revision.

Framework

All three PD concepts surface through this activity. Most obviously, it guides faculty through a collective inquiry into ePortfolio effectiveness as a demonstration of student learning. Integration itself is the focus of the activity, as faculty evaluate the integrative value of student work presented through their ePortfolios and strategize ways to teach integrative thinking more effectively. Finally, the activity affords space for faculty to step back from the work and reflect upon their own teaching.

Participants and Seminar Leadership

The Connected Learning seminar has averaged approximately 15 enrolled faculty per year–roughly 2/3 of whom are full-time, with the other 1/3 part-time lecturers–since the inception of this activity. Participants represent a cross-section of the college’s twelve academic departments; no more than three faculty from any single department have participated in the seminar in a given year.

The seminar’s three-member leadership team consists of two full-time faculty and one full-time staff from the Center for Teaching and Learning, who meet regularly to plan and design PD activities for the seminar. Two of the three also sit on LaGuardia’s C2L campus team.

Successes

To date, 27 faculty have participated in this activity (12 in 2010-11; 15 in 2011-12), with another 13 scheduled to participate in spring 2013.

While teaching with ePortfolio during a targeted implementation semester is a requirement of theConnected Learning seminar, we cannot mandate that faculty use all of the materials they develop through the seminar, so direct follow-up using the rubrics developed is likely partial at best. Likewise, because this activity represents only a portion of a year-long PD seminar, there is no way to isolate its impact on faculty beyond what they report anecdotally (that said, anecdotal reports about the value of the activity have been quite positive). However, the rubrics faculty have developed and the resultant discussion have helped to move the college closer to piloting holistic assessment of ePortfolios, which was one of the recommendations that emerged out of our recent Middle States reaccreditation process.

Challenges

Most directly, this activity forces faculty to confront their shared responsibility for teaching stendents to think integratively. Faculty are typically able to point out where LaGuardia students’ ePortfolios fall short of true integrative learning, but they do not always recognize what they themselves can do to make it possible, or that integrative learning should be a curricular priority for everyone.

The activity also disabuses narrow, disciplinary thinking about ePortfolio by partnering faculty from disparate disciplines. Because they are asked to develop rubrics that assess holistic student learning, faculty cannot retreat to the familiar conventions and uses of ePortfolio within their disciplines or professional fields.

Connections

Resources

Sample faculty rubrics (first drafts):

Practice Profile

TECHNOLOGY
This particular ePortfolio-related Professional Development activity:
Primarily focused on pedagogy

OUTCOMES ASSESSMENT
The linkage between this particular ePortfolio-related Professional Development activity and Outcomes Assessment is the following:
Considerable linkage to Outcomes Assessment

EVIDENCE
The type of evidence we collect to document the effectiveness of this particular ePortfolio-related Professional Development activity is the following:
See #22 on the limitations of evaluating this PD activity apart from the longer seminar of which it is a part

FUNDING
This particular ePortfolio-related Professional Development activity is funded through:
Grant-Funding

ADJUNCT FACULTY
The following represents the percentage of adjunct faculty participants in this particular e-Portfolio related Professional Development activity:
26-50% of participants are adjuncts

Skip to toolbar