Curricular Initiatives: College of Arts and Sciences
The College of Arts and Sciences is committed to a program of annual assessment to improve student learning. A yearly cycle of effective assessment of student learning outcomes is an iterative developmental process. The following examples represent academic programs at various stages of development of their annual program of student assessment.
BS, BA, and Dual Degree Programs (PT, PA, AT)
Faculty and instructors of the Biology Department conducted several assessments based on program learning goals and student learning outcomes during the 2012-2013 school year. Faculty used a rubric to directly assess undergraduate students' ability to think critically during the biological sciences poster presentations at the annual Petersheim Academic Exposition. Undergraduate students with direct research training experience demonstrated noticeably greater critical thinking ability compared to students presenting their poster to satisfy the Senior Seminar course requirement. Also, the department developed two new surveys for indirect assessments of student outcomes. A satisfaction survey of current graduating students yielded an overall majority of satisfied/very satisfied students. An alumni satisfaction survey was also developed and evaluated preliminarily with a small number of alumni. The department will continue collecting the alumni satisfactory data in the coming years. To assess outcomes of dual-degree students the department tracked the percent of students successfully moving forward to the PT, PA, and AT graduate programs at Seton Hall. The majority of dual degree students obtained the principles and concepts of biology and cognate sciences and developed the required skills to move forward to PT, PA and AT graduate program.
MS in Biology/Microbiology and PhD in Molecular Bioscience (MOBS Program)
Faculty and instructors of the Biology Department conducted several assessments based on graduate programs learning goals and student learning outcomes during the 2012-2013 school year. Doctoral students in the MOBS program have certain "landmarks" of progress to follow throughout the course of their program including completion of lab rotations, preliminary exams, candidacy exams, paper publication, and thesis completion. An evaluation of the progress attained by students has led to some modifications in the program and PhD guidelines to account for recognized difficulties students have faced. Critical Thinking was the common student learning outcome chosen for direct assessment this year. To assess MS and MOBS students' ability to think critically in the field of biological sciences, faculty used a rubric to assess the poster presentations of 18 graduate students during the biological sciences symposium at the annual Petersheim Academic Exposition. The presentation included background information, hypotheses, results, and conclusions. The students' poster presentations earned an overall average of 85% with the strongest performance on organization and the weakest performance in the results. Indirect assessment of student outcomes were also initiated with the development of new satisfaction surveys that will be fully implement in the coming years.
The Department of Chemistry and Biochemistry used standardized subject tests from the American Chemical Society to assess undergraduate students' knowledge in General Chemistry, Organic Chemistry, Biochemistry, Physical Chemistry, Analytical Chemistry, and Inorganic Chemistry which correlated with specific courses in the program. Students graduating with a B.S. in Chemistry scored well within the range of the national scores and fell within the 30-40 percentile. An important measure of outcomes for graduate students in the MS and PHD programs includes publications in co-authored peer-reviewed publications and presentations. The department is currently analyzing this data over several years to assess trends in outcomes.
Faculty and instructors of the Department of the Communication and the Arts initiated a student learning outcome assessment based on program learning goals. Program learning goals for students in the Art History program included ability to develop an appropriate vocabulary of course material, understand and recognize objects of art and visual culture, understand prerequisites and modalities of artistic production and training, understand the merit of studying art and visual culture, recognize the way in which art and visual culture relates to experience, develop meaning and interpretation of art, gain a global perspective on art, and develop researching, writing, and presentation skills on art. For this year's assessment students completing an Art History capstone course or a 3-credit independent study were chosen. Faculty assessed written papers in relation to proficiency in discipline-specific research, critical analysis, and writing, as well as student presentations to the faculty. In addition, graduating seniors wrote a formal analysis of selection works of art and visual culture which was then examined in relation to the program learning goals. The results are being evaluated to further develop reliable direct measures of assessment and to determine the effectiveness of the in-class experience on student learning.
Assessment of student learning outcomes for students in the Theatre program focus on students' ability to analyze and interpret plays and theatrical events, reach an audience effectively, function safely and effectively using contemporary theatre technology, use the skills and techniques needed in research, express the results of research and critical judgment in performance and other modes of communication, relate theory to theatrical literature and performance, and respond as a critically informed member of the theatre audience. Direct measures of these skills are being developed to include traditional measures such as course examinations, papers, and grades, as well as measures to evaluate journals and production books, mentoring and performance at internships.
B.A. Art and Design
Assessment of student learning outcomes for students graduating with a B.A. in Art and Design included student presentations during their Senior Practicum Seminar that were followed by a critique of their graphic portfolios and presentation skills. The assessments were made by 10 reviewers including university and outside professional reviewers. Performance was measured with an instrument quantifying a student's strengths and weakness by assigning valences and adding additional comments. Both design skills as well as presentation skills were assessed. The collected student scores were used to support the reviewers' comments and verbal critiques. Overall, the reviewers were impressed. They applauded the diversity of the work, summarizing the event as being "idea-forward". The majority of students demonstrated an excitement for their work and a confidence in themselves. No overall weaknesses were identified. Individuals with lower scores were encouraged to embrace the identified problems cited by the reviewers and build their confidence further by working independently on additional projects.
B.A. in English
The Department of the English has been engaged in a yearly cycle of student assessment for several years. Goals for students graduating with a B.A. in English include: providing all students with linguistic and literary competency, providing majors with a broad knowledge and critical understanding of literatures in English, and developing student ability to think analytically, read close, and write well. Twelve student learning outcomes (SLOs) have been identified that address these goals, organized into 4 categories: Content, Rhetoric & Style, Research Methodology & Critical Acumen, and Mechanics. For the last two years the department has assessed the SLOs by evaluating student portfolios. The portfolios are scored by four faculty using a common rubric including the four categories. An effort was made to evaluate inter-rater reliability. Although improvement in inter-rater correlations improved this year, the reason for improvement was not clear. Benchmark essays from these initial assessments will be used for norming. No significant difference in total score performance were observed when the scores of successive years were compared. In addition to the total rating scores, the number of positive and negative reviewer comments was coded for each SLO. Analysis of the specific outcomes indicated that certain outcomes appear to receive greater emphasis in the portfolios than others. One category, Mechanics, received comments disproportionately in the negative. Comments in all other categories were more positive than negative. The results of the assessment report are shared at the department's annual retreat. At the 2013 retreat the decision was made to have a workshop in the fall for faculty teaching upper-level literature classes on best practices for having students use research materials: identifying good sources, understanding them, and using them effectively in papers.
M.A in English
Student learning outcomes for students graduating with a M.A. in English included developing students' abilities as graduate level readers, writers, and researchers, introducing students to traditional and contemporary modes of literary theory, assisting students' development as scholars and writers, expanding students' knowledge of schools of theory, historical periods, and literary genres, and developing students' abilities to do complex close reading. The program assessed students using a Comprehensive Exam, testing their knowledge, as well as their writing and analytical ability.
B.A. in History
The student learning outcomes that were assessed in History Majors (B.A.) for 2012-2013 included: demonstrating "historical fluency", recognizing the significance of available primary and secondary source material, and demonstrating knowledge of how and why societies have changed over time. The department implemented a longitudinal study to compare student proficiency in historical research, writing, and analysis based on papers submitted in HIST 2180 (Intro to Historical Research) and HIST 5199 (Senior Seminar). Within these courses the department analyzed research, argument, historiography, writing mechanics, and citation. Data revealed solid improvement among history majors in their acquisition of "historical fluency".
Russian and East European Studies (REES)
Faculty and instructors of the Department of History conducted a program assessment based on program learning goals in the 2012-2013 school year. The Russian and East European Studies program aims to provide rigorous and comprehensive liberal arts training, preparing students for continuing engagement with Russia and Eastern Europe throughout their careers. To assess this target, students had to submit a research project on a relevant topic.
During the 2012-2013 academic school year, faculty and instructors of the Department of the Languages, Literatures, and Cultures conducted a student learning outcome assessment based on program learning goals.
Faculty of the Italian program used a research paper on Italian civilization to assess the student learning outcome for students to write about content knowledge in a high quality, organized, clear, and original manner.
Student learning outcomes for students in the Russian program included the ability to read and write in Cyrillic in all tenses, speak and understand spoken Russian, identify major literary and historical figures of the Russian language, read and understand Russian media, and value the history of the former Soviet Union. Students were assessed on a report written in Cyrillic after visiting a Russian speaking community in addition to completion of the National Russian Language Competition test.
Student learning outcomes for students minoring in German included students' ability to speak basic German, talk and write about any topic so that it's understandable to native speakers of German language, read and understand German media, understand German spoken by native speakers, and value and understand the role of Germany in the EU. Students were assessed through essays, final exams, and presentations.
One student learning outcome for the classical studies program included giving students a deeper knowledge of ancient languages and civilizations. All Latin students were assessed through completion of a National Latin Exam or equivalent, while all Greek students completed the College Greek Exam.
Student learning outcomes for the Asian studies program included students' ability to gain good proficiency in Chinese or Japanese, have advanced backgrounds and learning about histories and cultures of Asian studies, and conduct independent research related to Asia or Asian matters. Students were assessed through the compilation a portfolio.
Student learning outcomes for the students in the Chinese program included motivating students to learn the Chinese language and culture, engaging students to develop real communicative skills, and helping students become life-long learners of Chinese. Students were assessed using the annual Chinese Speech Contest as a part of the Petersheim Academic Expo. Additionally, students completed the HSK test.
Graduate Program in Asian Studies
Student learning outcomes for the students in the Area Studies Track of the Graduate Asian Studies program included student proficiency in Chinese or Japanese, development of advanced backgrounds and knowledge of histories and cultures of Asian societies, and demonstration of conducting independent research related to Asian matters. Student learning outcomes for students in the Chinese teaching track included student achievement of a near-native proficiency in English, knowledge of Chinese civilization, history, linguistics, and literatures, and understanding of theories and practices in teaching Chinese. Students were assessed through the compilation of a portfolio.
Latin American and Latino/Latina Studies (LALS)
One student learning outcome for students in the LALS program was students' ability to learn to critically examine the representations of Latin American and/or Latinos/Latinas with respect to their race, ethnicity, class and culture in works of history, art, religion and literature. This student learning outcome was assessed though the evaluation of Individual Guided Research projects that were included in students' e-portfolio.
Student learning outcomes for the B.S. in Mathematics are based on the standards of the NCTM (National Council of Teachers of Mathematics) as well as CUPM (Collegiate Undergraduate Programs in Mathematics). During the 2012-2013 school year two of the eleven student learning outcomes were chosen for assessment:
SLO 2. Knowledge of Reasoning and Proof -Students reason, construct, and evaluate mathematical arguments and develop as appreciation for mathematical rigor and inquiry.
SLO 3. Knowledge of Mathematical Communication. - Students communicate their mathematical thinking orally and in writing to peers, faculty and others.
The SLOs were assessed in two courses required for the major (MATH 3912-Junior Seminar and MATH 3515-Analysis I) and one course required for Departmental Honors (MATH 4912-Senior Project). For the junior and senior seminar courses poster and oral presentations, respectively, were assessed at the Petersheim Academic Exposition with a common rubric. For the Analysis I course, the faculty were furnished with a sample of students' final exams and student performance on this exam was discussed at the annual faculty retreat. Department of Mathematics and Computer Science faculty concluded that the student learning objectives are being addressed in an exemplary fashion and that student performance of the two standards was entirely satisfactory. While high-achieving students performed as expected, the department was particularly impressed by the improvement in the students who had not previously distinguished themselves. A paper written by the MATH 3912 class was accepted and published by Graph Theory Notes of New York LXV, 27-32 (2013).
Student learning outcomes for the Master of Healthcare Administration (MHA) included students' ability to lead and manage in the healthcare environment, critically think in a complex and competitive environment, develop politics and community in the healthcare environment, communicate, and display professionalism and ethics. The assessment plan for the MHA student competencies included the development of a MHA Competency Self-Assessment Survey, the inclusion of the competency domains in the regular course evaluation form, and the continued use of the American College of Healthcare Executives' (ACHE) Leadership Competency self-assessment as a comparison benchmark. During the 2012-2013 school year the department chose to develop a solid baseline so that the department could then target those learning domains that showed gaps in student learning outcomes. As part of the final set of Graduation Exit surveys, students completed the MHA Competency Self-assessment Survey that asks them to identify which of the six competency domains were of most value to them. Students reported that they were strongest on the "Communication within a Diverse Healthcare Environment" competence and most weak on the "Political and Community Development within the Healthcare Environment" competence. At least 94% of students reported themselves as either competent, proficient, or expert in all six of the MHA Learning Domains.
Student learning outcomes for students in the Philosophy program included student identification and commenting on the thesis, argument, and conclusions presented in a text, development of an argument or disagreement with conclusions presented in the original article, and articulation of the evidence and reasons for one's position. Faculty focused on indirect measures of assessment for the 2012-2013 academic year by conducting and reviewing senior surveys. Going forward, the department Curriculum Committee will provide more guidelines for writing a philosophy paper.
For the 2012-2013 academic year the Department of Physics identified one student learning outcome for assessment of the undergraduate Physics program. The student learning outcome "Implementation of theoretical concepts of physics to analyze scientific problems" was assessed in the subject matter of geometrical optics. The Department conducted pre- and post-tests in three courses covering this subject: PHYS 1702 General Physics II, PHYS 1706 Principle of Physics II, and PHYS 3217 Modern Optics.
The Political Science Department administers an exit survey to its majors as they complete Senior Seminar. This survey asks about student's learning experiences in the political science curriculum and their preparedness as they approach graduation. We will continue to use this survey instrument, but we will modify certain questions to improve its usefulness. The department also developed a new assessment instrument ,an (ungraded) pre- and post-test that asks students to respond to an op-ed piece related to a political science topic. The test consists of four questions:
- What are the key points the writer makes ?
- What evidence does the writer use to support his/her argument?
- What debate (or debates) within political science scholarship are relevant to the article?
- What political science terminology is used (or could be used) to discuss the topic?
The rubric for evaluating this test will include the four questions above, and will add an additional (5 th) evaluation component focusing on the student's writing: mechanics, logic, organization and clarity. These five test questions track the assessment outcomes listed on our department website. The instrument is administered to incoming first-year students, and to seniors. Comparing the two results will allow us to measure what students have learned over the course of completing the major.
Psychology B.A. & B.S & Honors
The Psychology Department adopted the undergraduate learning goals and related outcomes recommended by the American Psychological Association as the Student Learning Outcomes (SLOs) for the BA, BS, BA/BS-Honors Psychology programs. The department includes all Goals and Outcomes on the course syllabi with the specific SLOs that are covered by a course explicitly marked. The syllabi were used to identify the SLOs addressed by psychology courses and to create a curriculum map. A review of the curriculum map confirmed that all learning goals are covered in the curriculum, although to different degrees. The most frequently covered goal was Goal 3 - Critical Thinking Skills in Psychology (82.4%); the least covered was Goal 4- Application of Psychology (39.5%). During the 2012-2013 academic year the department decided to directly measure critical thinking skills in four target courses with a common rubric. The rubric was used to score student essays assigned in the courses. At the annual department assessment meeting the faculty discussed the appropriateness of the rubric as a direct measure of critical thinking and reviewed the students' performance. The faculty concluded that two of the six categories of the rubric items did not directly address critical thinking and skeptical enquiry and should be deleted. It was clear from the results that students are not proficient in critical thinking skills despite the fact that critical thinking was the most frequently covered goal in the curriculum map. Interestingly, the results of the senior survey indicated that compared to all other goals the critical thinking goal was identified as "thoroughly addressed" by the fewest number of students (the department administers an annual senior survey as an indirect measure of student attitudes and experiences). It was concluded that students must be given explicit and repeated instructions to engage in critical thinking. The next step is to identify consistent "places" throughout the curriculum that might lend themselves to explicit "teaching" of critical thinking skills. The department also initiated a pilot project to evaluate the feasibility of an ePortfolio requirement for future assessment activities.The purpose of the pilot was to determine 1) the ease with which students can learn to use WordPress to post artifacts, 2) the interest in, and facility with, going beyond the basic features of the WordPress ePortfolio template, and 3) student's motivation to develop the ePortfolio beyond the basic course requirements.
M.S. in Experimental Psychology
During the 2012-2013 academic year the Psychology Department decided to assess the presentation skills of graduate students in the MS Program in Experimental Psychology. Student's oral presentations were evaluated in two graduate courses and during the Petersheim Academic Exposition. At the 2013 Academic Exposition 12 students gave oral presentations, 8 graduate students and 4 undergraduate Psychology Honors students. The full-time faculty in attendance at the symposium (n = 5) rated the student presenters with the Oral Presentation rubric that was provided by the University Assessment Center. The same rubric was used to assess oral presentations in the two graduate courses. The results were compiled and discussed at the Psychology Department's annual assessment meeting. The main goal of this initial assessment was to evaluate the usefulness of the rubric for assessing student oral presentation skills and to determine how students performed in their oral presentations. Because some faculty gave consistently lower scores than others the raw scores were standardized for each faculty rater by converting them into Z-scores. In general, there was consistency in how faculty rated the performance of most students. For example, the same 6 students were identified as at or above average by all faculty and 2 students were consistently identified as the lowest performers. However there was disagreement with some of the remaining students. Some items in the rubric were edited in an attempt to improve inter-rater reliability for next year's assessment activities. Overall the students did well with their presentations with graduate students scoring slightly higher than undergraduate honors students, although this difference was not statistically significant. The department also developed a graduate student survey (using SurveyMonkey) for indirect measures of student experiences and attitude. Generally, the M.S. program received very positive reviews by the students.
Student learning outcomes for students a program in the Department of Religion included student demonstration of 1) knowledge of Christian and non-Christian religions, 2) understanding of contemporary debates that effect human flourishing in relation to and in light of religious texts, 3) an ability to interpret contemporary issues in the light of the theological and ethical commitments, and 4) an ability to offer an analysis of topics in religion. Faculty reviewed and discussed course artifacts brought to a department assessment meeting. Each faculty member shared examples of excellent, good, and poor artifacts (exams, presentation reports, and papers). The most prominent student learning outcomes covered in the artifacts were outcomes #1 and #4. The department covers outcome #1 (knowledge base) quite well and the departmental artifacts show this. Even at the poorer levels, students show a basic knowledge base. Student ability to apply moral principles to a particular contemporary issue (outcome #3) in a nuanced and intelligent manner was mixed. Faculty decided to experiment with alternatives to the research paper (e.g., intensive analysis of faculty-chosen material rather than extensive research). The goal is to support class learning/material with more pedagogically useful assignments than, to this point, the research paper has proven to be.
Sociology & Anthropology
Student learning outcomes for students in the Sociology & Anthropology program included student familiarity with sociological and anthropological theories, exposure to a range of substantive concentrations within the two disciplines, training in the logic of research methodology, and development of the capacity to analyze contemporary issues and controversies. Two exit interviews were conducted in the senior seminar course to assess the sociology and anthropology programs.
The Social Work program has been engaged in several years of extensive assessment of program level goals and student learning outcomes for accreditation requirements of the Council of Social Work Education. Student learning outcomes have been assessed in numerous courses and field work. Assessments were conducted by classroom faculty as well as field instructors and included evaluation of specific assignments and performance in the field. The observation of improved student scores for Senior Field II compared to Senior Field I, for example, demonstrates an efficacious paradigm of incremental, integrated, and cumulative preparation for professional practice. In the 2013 "Reaffirmation of Accreditation Progress Report" that was submitted to the Council of Social Work Education the authors of the report summarized their assessment activities during the accreditation process, "…the data affirmed how diligent our faculty and students are. We all not only learned from this process that will continue, but have internalized it."