Master of Arts Degree in Counseling & Guidance

Comprehensive Program Evaluation

February 16, 2005

A comprehensive program evaluation of the MA Counseling Program was conducted as required by CACREP Section VI C. 1-5. The evaluation includes a review of: the mission, objectives, and student learning outcomes; characteristics of program applicants and application process; curriculum offerings; and recent surveys of graduates, alumni, and employers of alumni.

Mission, Objectives and Learning Outcomes

The review of the mission, objectives, and learning outcomes is ongoing. It involves input from the faculty, students, and personnel from cooperating agencies. The Mission Statement. In accordance with CACREP Section II "A" 1-4: the MA counseling Program mission statement was developed to bring the program into focus and describe the program's intent and purpose (please see Appendix "A" for copy of departmental and program mission statements). The program mission statement describes the types of students served, geographic orientation, and priorities of the faculty and was the basis for the development of program objectives and curriculum. For example, the program mission statement addresses several issues such as the multiple settings mental health counselors serve (e.g., "professional counselors will be of greater service to an ever-changing society if they are able to offer high quality counseling services to a wide variety of clientele in a wide variety of settings, including adults and children in community agencies and schools").

The Counseling and Educational (CEP) faculty met during the 2005 Spring retreat and concluded that the program mission statement continues to accurately reflect the intent and purpose of the MA Counseling Program.

Program Objectives and Outcomes.

During the Fall semester of 2004 the CEP faculty reviewed their course syllabi to ensure that they met CACREP standards. Faculty indicated where in their syllabi standards were met and this information was forwarded to the MA Committee for review. The MA Committee is currently requesting more detailed information on syllabi regarding how the program objectives and CACREP standards are being addressed. This process is ongoing and will continue into the Spring semester 2005.

Program objectives for the MA Counseling Program have been identified and are available in the student handbook (please see attached "B"). Program objectives were reviewed during the Spring semester 2005 by the CEP faculty during the full faculty retreat, the MA Counseling committee, and the advisory board. It was decided that the program objectives needed to be changed to more directly reflect the 8 CACREP core areas and the CACREP standards associated with those core areas. In addition, the MA Committee planned to conduct a syllabi review (during Spring semester 2005) to ensure that program objectives and CACREP standards would be stated in a manner that could be assessed.

Characteristics of Program Applicants and Application process

Admission criteria and application information are stated in the MA Counseling Application Materials Handbook. Fourteen students were admitted in April 2004- half of whom were full time and half ¾ time. One student declined just before classes started in the summer leaving 7 full time and 6- ¾time students. The admission deadline was changed from April 1st to March 1st to allow more time to process applicants.

The MA Counseling Program strives to maintain diversity. There are currently 37 students in the program who self-identified themselves in terms of the following: 12 Hispanic/Latino, 14 Caucasian, 1 "other", and 2 international. Eight students are male and 29 are female. Recruitment efforts are being enhanced via various activities such as updating the CEP Web Cite regarding the MA Counseling Program and participating in the Graduate Professional School annual event at NMSU.

Review of Curricular Offerings

Review of curricular offerings is ongoing by the MA Counseling Committee to ensure that courses are offered consistent with program objectives and CACREP standards. The following curriculum changes occurred since the Mid Cycle Report of 2002. During the 2003-2004 academic year, the curricular offerings were reviewed by the MA Counseling Committee resulting in the following changes: Two courses- CEP 550-counseling theory and CEP 504- counseling skills were combined into one course- CEP 550 counseling theory and technique. The course includes 2 hours of lecture/discussion of counseling theories and techniques and a separate 2 hour lab during which students engage in supervised practice of techniques. The courses were combined to allow unified coverage of theories and the techniques that are based on those theories. In this format, students learn about a theory, learn about a technique based on that theory, see that technique performed, and then try the technique in a supervised role play.

Combining the courses also enabled the program to add a course- CEP 558 child and adolescent counseling. This course was perceived to have clear value in both school and mental health settings. Feedback from students engaged in advanced practicum in schools suggested the need for additional course work on counseling children.

Surveys of Recent Graduates, Alumni, and Employers of Alumni

Surveys of recent graduates, alumni, and employers of alumni occurs every three years. An overview of these surveys is as follows.

Survey of Recent graduates. During the past three years, recent graduates were surveyed after they successfully passed their orals. A report regarding this survey was written and disseminated to the CEP faculty during the Fall semester of 2004 (please see attached "C"). Numerical results of the survey were reported using a 5 point Likert scale (1 low; 3 moderate; and 5 excellent). Results of the survey suggested:

Admission criteria was clear and reasonable
MA Counseling annual orientation session was valuable
Ratings for advisement were between moderate and excellent (3.18 to 4.0)
Evaluation of MA Counseling courses were all between moderate and excellent when combining and averaging the two cohort groups.
Value of the courses in preparing them to be a counselor received a rating of 4.0.
The written exam received a rating of 3.54 and the oral exam was rated at 4.33. Survey of Past Graduates (Alumni). A survey was distributed in June of 2004 to graduates from the last four years (please see attached "D"). Ratings were based on a 5 point Likert scale (1= inadequate; 3= adequate; and 5= excellent). A summary of the survey is as follows.
Alumni were surveyed regarding how much time they were spending doing work-related activities. Results showed: individual counseling to be the dominant activity at 22.83%. Other major activities included: administration, couples/family counseling, and group work, with the remainder split among various activities. No time was associated with assessment, and research garnered only 1.67% of the time.
Alumni were ask to rate their current skills. Highest ratings were: legal/ethical decision making (4.0); individual counseling (4.0); and addictions (4.0). Appraisal received the rated lowest at 2.83.
Alumni also provided ratings of the CEP curriculum. All courses received 3 or above ratings with the exception of appraisal theory and technique (2.71) and professional issues in mental health counseling (2.86).
Alumni rated advisement a (2.83).

Survey of Supervisors/Employers of Past Graduates. The supervisors/employers of CEP MA alumni were administered a survey between May and August of 2004 (please see attached "E"). The survey was distributed to supervisors of CEP MA Counseling graduates from the past four years. Five supervisors responded. A five point Likert scale was utilized (1=inadequate; 3 adequate; 5=excellent). Results of the survey were as follows.

Supervisors assessed what percentage of time alumni spent on work-related tasks. Supervisors reported that CEP graduates spent over half of their time performing case management, individual counseling, and other activities. The remainder of their time involved couples/family counseling, diagnosis, group work, and research. In addition, it was noted that the least amount of time was spent consulting and teaching.

Supervisors also rated CEP graduates in terms of skills they were trained to perform (skills rated by at least 3 respondents were included). Ratings of alumni counseling skills were between adequate and excellent. Particularly high ratings included: child/adolescent counseling (4.3); professional role of mental health counselor (4.1); awareness of client's worldview (4.0); and culturally appropriate intervention strategies (4.0). Perhaps most important, the overall rating of skills for alumni was rated high (4.3). Lower than average were found for group counseling (2.8) and career counseling (2.8).

Program Modifications

Information from the comprehensive program evaluation was used to generate strengths and concerns in the program. These findings have implications for program modification which will be shared with the CEP faculty, students, and personnel from the community during the Spring semester 2005. Strengths:

Recent graduates contended the admission process was clear and reasonable. In addition, the annual new student orientation session was valuable.
Recent graduates and alumni gave overall positive rating to the value of their courses in preparing them to be counselors.
Alumni gave their highest ratings for training in legal-ethical decision making, individual counseling, and addictions counseling.
Employers rated alumni's counseling skills between acceptable and exceptional.

Concerns:

Concerns related primarily to the percent of time MA graduates spend in work related activities. In terms of time spent doing counselor related activities, past graduates rated research as only 1.67% of their time at work with no time spent on assessment. Graduates also gave a rating of only 3% of their time spent on case management. Employers of alumni noted that CEP alumni spent the least amount of their time doing consultation and teaching. These results suggest the following programmatic modifications.

Courses need to promote an understanding of the relevance of research (e.g., the interrelationship between theory, research, and practice). In addition, curriculum should address how research can be conducted to enhance practice (e.g., empirically supported treatments in managed care).
More training in consultation in school and non-school settings.
Students need to have a better understanding of assessment in clinical practice.
More emphasis needs to be made regarding case management in course work.

Click here for a print-ready version of the Complete report including appendices. PDF Files require Adobe Acrobat Reader. Get the free version of Acrobat Reader by clicking on icon.