Introduction
Social and cultural trends, economic forces, population changes, new and emerging technologies, and issues of public policy will have powerful and lasting effects on the ability of colleges and universities to fulfill the demands of their mission and the expectations of their students and constituencies. (ACPA & NASPA, 2004)
The accountability movement is an active part of management in American higher education. As a result, student affairs divisions have been forced to justify not only their existence as a unit but also their seat at the table of university decision making. Divisions across the country have had to devise unique methods for “telling their story” to the university community. In many cases the challenge is to not only show support for the academic mission of the institution but to also demonstrate responsible stewardship of university resources. Maintaining a culture of evidence built upon consistent data collection will provide a foundation for the division to maintain and evaluate its key activities, while assessing its contribution to the learning enterprise of the institution.
An Integrated Process
An effective assessment process assists departments and divisions in not only telling their story but increasing the accuracy and effectiveness of their decision making processes. With the development of a systematic assessment process, data becomes more reliable. In addition, the tracking of trends related to the key activities of the division will help in the assessment of the program’s vitality. Making decisions based on valid data can assist student affairs divisions in securing and allocating resources effectively. A systematic process will help the division have influence in university policies and decisions.
Effective data collection during the assessment process will help the division develop benchmarking data that can be used in the assessment of programs to comparable institutions and divisions. Again, that data will help support decisions in relation to national trends and the development of the division. Reliable data should be collected across time and with comparable institutions.
Initiating an assessment process around the principles of strategic management ensures that the departments of the division are involved in a day to day process of achieving the goals and meeting the factors that lead to the success of the division. There is distinction between strategic management and strategic planning. A management process needs to be in place before a division or institution can enter a strategic planning process. The management process provides the systematic data collection that feeds the development of the vision for a strategic planning process. The strategic planning process takes into the long term success of the division. Strategic planning has a place in the management of a division but the two processes are not mutually exclusive. The strategic plan of a division looks at the long term viability of the program and the division’s strategic position in the decision making process of the institution.
The DePaul University Process
In effort to address the growing need for accountability at the university and community level, the Student Affairs Division at DePaul University developed an assessment process that addressed good practices in assessment in higher education and the use of strategic management principles. The DePaul process measured the environment, the activities and the outcomes associated with those activities for the division.
The environmental measures allowed the division to understand its position in the institution, and with peer institutions, relative to the mission and vision it has established. The establishment of the core values of the division served as the environmental scan for the division.
The goal of the process was to provide a framework to translate the mission and goals of the division into operational terms, or key activities. The process included identifying key activities across the division and cascading those activities throughout the departments of the division. In addition to the key activities, assessment questions were developed to compliment the assessment process taking place within the colleges and institutes of the university. The assessment questions were tied to the learning outcomes developed for each activity. This process was performed uniformly at both the department and division level.
The key activities were the essential functions of the department and division that lead to fulfillment of the division’s mission. It was critical for each department of the division to communicate their key activities and the measures that demonstrated progress towards fulfilling those activities.
The key activities of each department were measured and collected into an annual report submitted to the senior leadership of the division. The division developed a key activity report that was informed by the reports submitted by the departments. The key activities measured cost, magnitude, and satisfaction associated with each activity. Cost measures included but were not limited to: cost per student; cost per staff; overhead; and all expenditure used in support of this activity.
Magnitude measured the impact of the activity on the university community: This number can be stated as sum, average or ratio. Satisfaction measured the ability of the activity to meet not only the expectations of the participants in the activity, but also satisfaction of the learning outcomes with the activity. Each activity should be grounded in a set of learning outcomes. The learning outcomes ensured that the programs were striving towards support of the learning enterprise of the institution.
Once the key activities are identified, each department and the division decided on an assessment question to answer for the year. The assessment question was derived from the learning outcomes developed for the key activities. Answering one assessment question was appropriate for this process. The process had to be manageable. A deterrent to progress is attempting to measure more than is possible. Assessment goals that are too grand will eventually lead the failure of the assessment process. A vast majority of the culture of evidence was captured in the measurement of the key activities. Each year the department can choose to assess a different aspect of their program or continue answering the assessment question from the previous year if it is warranted.
All data sources were considered when measuring the environment, the key activities and answering the annual assessment question. Departments developed a systematic way to collect the data necessary to support the key activities of the depart. The data came from sources such as intake forms, budget documents, and survey data collected at the university and division level. Qualitative methods such as focus groups or professional observation were also used in the collection of data. This process resulted in the development of an annual report of key activities by each department, a student affairs fact book, and richer data to support the division during the resource allocation process.
The key activities represented the everyday management processes for the department. The learning assessment represented the systematic inquiry into opportunities to help the office grow and develop. In addition, the assessment process lead to the development of a clear data source for the departments and the division, the development of publication to celebrate the accomplishments of the division, and the establishment of the student affairs assessment committee. Through this process data became consistent and reliable. The division will be able to conduct studies with peer institutions and divisions. The establishment of a culture of evidence made the development of new priorities and planning more efficient. Also, systems have been built to support the division’s assessment efforts. Figure 1. The division of student affairs integrated assessment process.
Conclusion
Development of an integrated assessment process can assist a divisions with continued positive change while creating a culture of evidence to support the decision making process. To have outcome assessment without the data to measure the key activities of the program is an incomplete assessment process. The measurement of the environment will be made easier once a consistent and reliable data structure has been developed. With an integrated management process the division will have a holistic understanding of its efforts to fulfill its mission and vision. This process requires the support of various offices within the institution. Participation from across the institution is essential. In particular, offices of institutional research and offices of academic assessment can be valuable resources for divisions beginning this process. The DePaul University process demonstrates that an integrated assessment process is manageable and attainable.