CBT Review and Evaluation

Competency based Training, like any training, should evolve dynamically. The drive for continuous improvement will only be possible if training is reviewed, measured and evaluated. Adjustments can be made  based on the results of the evaluation.

In this lesson you will learn how to evaluate training session.

Training Evaluation

The  evaluation of training refers to any attempt to obtain information or feedback on the effects of a  training program and to assess  the  value  of the training in the  light of that information. The  primary purpose of evaluation is to improve training by determining which  training  processes  achieved their objectives.

It is important after you deliver a training session that you review your delivery and look at what went well and what might be improved.

In your review, the following components shall be included:

a. Knowledge – refers to what we know. It is the cognitive domain  of  human behavior

– the acquisition of knowledge is measured  through  the  pre- test and post test. A pre-test posttest analysis is a very effective and objective method of analysis.

b. Skills – refers to what we do correctly and accurately. It is also the psychomotor domain of human behavior.

the acquisition of skills is measured through the performance test. But since we do not  have  a  grading system, rating the performance of trainees for purposes of program evaluation will not be as effective. An evaluation of how the training enhanced their skills shall be used  to analyze the effectivity of the training program.

c. Attitude – refers to what we feel. It is also known as  the  affective  domain of human behavior.

positive attitude towards work, co-workers and the environment is a very important component of training that needs to be evaluated. A rating system on how the training program developed their positive work attitude maybe  helpful in evaluating the effectiveness the training program.

d. Program Implementation – the approach of implementing the training program.

– CBT implementation review and evaluation will be most effective if the methods on how the following categories of program implementation are analyzed:

d.1 Program Design and implementation

d.2 Course Content

d.3 Training Methodology

d.4 Program Administration and Management

d.5 Trainer’s Competence

We will be using pre-test/post test method to test for knowledge and Questionnaire will be used to gather data for skill, attitude and program implementation.

Feedback

In order to review your training you will need to gather feedback. It is useful to gather this feedback at the end of each session so that you can think about whether you need to make any changes to your next session. 

Feedback can be obtained from participants, other people like other trainers or supervisor and yourself.

Sometimes you need to report feedback in order to help management make decisions about whether to continue the program as it was, change it or stop it. Feedback can be gathered in a number of ways. The  table lists  the most commonly used methods, their advantages and disadvantages.

Method	Advantages	Disadvantages
Evaluation sheets	Cheap and easy  to  use Identifies areas for improvement	Cannot identify skills acquired Gives limited information
Interviews                with trainees or supervisor	Provides useful information Explores problems in depth	Time consuming Effectiveness      depends     on     the skills of the interviewer
Supervisor’s observation	Can use checklist Provides expert opinion	May influence performance Time consuming
Evaluation workshops	Provides useful in-depth information Provides       opportunity        for       open discussion	Time consuming, costly Results difficult to report Need an experienced facilitator

CBT Evaluation Tools

The table below shows some information gathering tools:

Information Required	Evaluation Tool
1. seek expression of satisfaction with the training process from the trainees	Rating scale Written comments
2. examine his/her own perceptions of the success of the training process	Rating scale (self-evaluation) Written comments from trainees
3. examine success of the training program in the light of the subject criteria	Focus      group      discussion       on     specific criteria
4. examine success of the training program in light of the area of the objective criteria	Passing rates in assessments Average       training        time       to       achieve satisfactory level performance
5. review success of training program in meeting enterprise goals	Rating        scale        on        applicability         of competencies acquired Written comments Impact of evaluation
6. examine own performance	Evaluation of preparation, timeliness, thoroughness,              attitude             towards trainees, closing training session.

Rating Sheets

In preparing a rating sheet to evaluate competency based training, a trainer must recall the characteristics of an ideal competency based training (CBT). Group the characteristics into the following aspects, preparation, delivery, support system and facilities. For ease of analysis a recommended rating scale should be used such as the Likert scale.

A sample evaluation tool and an analysis follow in the succeeding pages. The rating sheet will however depend on what the trainer need to know or verify about his performance as a trainer or the appropriateness of the methodologies that are used.

Focus Group Discussion

To use a focus group discussion as a form of evaluation, you will have to list down discussion questions, which focuses on the areas you want to evaluate. Examples or questions for discussion are the following:

1. What are the competencies you have acquired  from  the  training that are very useful?
2. Do you think the training prepared you with the competencies required in the workplace?
3. What do you think are the weaknesses and strength of the CBT program?
4. What improvements can you suggest for the program?

Once questions are prepared you can organize a meeting where all respondents will be present to give their opinions and come up with a concession.

Document review

This is done to gather information relating to the average training time of learners to acquire the desired level of competency, to identify the success rates in national and institutional assessment, and to summarize the  written comments of participants, supervisor and other stakeholders. Available data such as the individual records of trainees, national assessment results as in RWAC and results of focus group discussion or satisfaction surveys and other related researches that covers a  certain period of time.

Self Evaluation

An important part of the evaluation process is self evaluation. Self evaluation can take place during your training session as you reflect on how things are going. This will help you make any instant changes to your plan in response to the situation.

Self evaluation can also take place after training session when you ask yourself in-depth questions about whether there are things to be changed or problems to be solved.

The following strategies can be used for evaluating your own training and help you improve:

1. Take a video of the CBT and look for ways to improve yourself
2. Ask for feedback from colleagues and other trainers
3. Compare      your     training       methods     and      techniques      with     other trainers.

A template below is used as instruments for self evaluation

During the session did I	YES	NO
Establish an atmosphere of trust?
Encourage all trainees to speak and contribute ideas?
Assist the less articulate to express their ideas?
Consider all contributions to be worthwhile?
Remain aware of non verbal communication?
Praise/appreciate effort?
Summarize key points?
Used varied activities and task
Provide opportunities for practice?
Achieve the stated learning objectives?

Questions to answer

1. Were there any parts of the session which did not run as expected? Why?
2. Did any unexpected problem arise?
3. Were the session outcomes achieved? If not why?
4. Should anything be changed for the next session?