Ways of Gathering Evidence

After knowing the forms of evidences, you will be introduced to the different ways of gathering evidences. This lesson will equip you with the different ways of gathering evidences and will give the tips on how to use  this evidences.

Ways of Gathering Evidence Observation

Observation

Observation is a key form of evidence gathering in the actual workplace. When the picture of competence is developed within a workplace context it is easy to find opportunities for observing work performance. However, there are situations in which it might not be possible or it may be inappropriate to use observation. For example:

• work activities that occur irregularly, take place over an  extended period of time or are difficult to observe (this may include work activities such as dealing with an emergency situation, planning a process or managing a team task)
• work activities that involve issues of privacy and confidentiality, such as counselling, providing feedback to team members on work performance or dealing with confidential or private client information
• situations where it may be culturally inappropriate to observe the activity

In these situations alternative forms of evidence will be required, such as structured activities, third party feedback.

Observation should aim to collect evidence of all components of competency, including key competencies, dimensions of competency, safety and quality.

When conducting an observation, the assessor should:

• use observation as an opportunity to observe practical skills as well as attitudes, underpinning knowledge, key competencies and dimensions of competency
• include a range of work activities

• supplement observation with questioning or an interview, to draw out the way in which underpinning knowledge, key competencies and the dimensions of competency are being applied in practice
• consider other forms of supporting evidence which show how the candidate has prepared or planned for the task
• remember that the candidate might feel nervous while being observed
• use a checklist to record that the candidate has performed all required tasks.

Demonstration

For logistical reasons such as safety, expense or time, it is not always possible for assessors to observe a process during real work time. Therefore, it may be necessary for the candidate to demonstrate the process at another time or location. For example, it may not be possible for the candidate to demonstrate machine shutdown procedures during normal work hours. So the assessor may ask the candidate  to  demonstrate this skill out of working hours or at another site, such as an Accredited Competency Assessment Center (ACAC), using similar machinery.

Demonstrations can be used in a wide range of situations, such as:

• showing how to use specific functions of a computer package
• showing safe lifting procedures for different types of loads
• showing how to use an equipment in a range of situations
• giving a verbal presentation to a group.

In implementing a demonstration the assessor should:

• check that the demonstration complies with the information in the Training Regulation,
• including the elements and performance criteria, range statement and evidence guide
• equipment used is safe and reflects the type of equipment currently used in the industry
• conduct the demonstration at a site which reflects conditions in the workplace (Accredited Competency Assessment Center)
• set realistic expectations, for example, time limits and quality specifications
• use materials and  applications that  are  relevant to the workplace and the unit of competency

Third party reporting

There are some situations in which workplace evidence is required but it is not possible for the assessor to directly observe the candidate at work. For example, the work activities may be at a remote location,  involve issues of privacy or confidentiality, only occur rarely or may be conducted at a location where safety is compromised if another person is present.

Situations in which an assessor may not be able to observe the candidate include:

• while it may be possible to see a security guard carry out the mechanical operations required to use a gun, it is unlikely that the assessor will be able to observe the officer using the weapon appropriately in real life situations
• in some situations assessments have to be carried out in  distance mode because it is not possible for the assessor to visit the workplace to observe the candidate.

In each of these situations, the evidence will need to be collected by an observer or a third party, who may be the security officer or the work supervisor.

The assessor should select the best person to  observe  the  candidate and prepare materials which help the observer to provide relevant feedback on the candidate’s performance that is directly related to the relevant unit(s) of competency.

It is important to remember that the assessment decision is made by the assessor, not the third party observer.

Questioning

• Questioning is an evidence gathering technique that is widely used in competency based assessment. Questions may be asked orally or in a written format. Answers to questions
• provide evidence of underpinning knowledge, application of skills and the capacity of the candidate to transfer knowledge and skill to different contexts.

Questioning may be used to determine:

• why the candidate does particular activities or tasks at work
• the candidate’s responsibilities at work
• the candidate’s understanding of legislative and safety requirements that impact on activities or tasks at work
• the candidate’s understanding of workplace procedures
• whether the candidate is able to transfer skills to other contexts and situations

• what the candidate would do if something different or unusual happened or if something went wrong.

Oral questioning

• Oral questioning involves the assessor asking the candidate a number of questions about real, simulated or hypothetical situations. The questions may be preset. Alternatively, assessors may develop their own set of questions to suit the context of the assessment and the requirements of the competency standard.
• Oral questioning complements real time/real work observations and structured activities as it provides a means of probing the candidates understanding of the work. Oral questioning: is a valuable tool for collecting evidence of underpinning knowledge and its application across a wide range of contexts
• can be used where written questioning might not be appropriate due to literacy and language problems (in particular, to confirm the candidate’s understanding of the question and to probe for further information if the candidate’s response is not sufficient)
• can be standardised to enhance validity and  reliability
• provides immediate information about the candidate’s knowledge and understanding to assist in determining what other evidence may need to be collected.

Although questioning is a valuable tool for collecting evidence, assessors need to be aware that the correct questioning techniques need  to be used to avoid asking leading questions, mistakenly answering questions or giving  clues to the required responses. Assessors also need  to be aware that some candidates may not be able to visualise or conceptualise hypothetical situations and that questioning may favour those with good communication skills. To ensure that oral questioning is effective, the questions need to be well planned and structured. The following tips will provide assessors with guidelines for structuring and asking questions.

Hints for effective questioning

The assessor should:

• keep questions short and focused on one key concept
• ensure that questions are formal and structured
• test the questions to check that they are not ambiguous
• use open-ended questions such as ‘what if…?’ and ‘why…?’, rather than closed questions
• keep questions clear and straightforward and ask one at a time
• link the questions to work experience

• use words that the candidate will understand
• look at the candidate when asking questions
• ensure that the candidate understands the questions
• ask the candidate to clarify an answer if the assessor does not understand the response
• confirm the response by saying the answer back in the candidate’s own words
• document responses on a checklist or recording sheet
• time questions so that the candidate is not interrupted while carrying out a task that requires full concentration
• encourage a conversational approach when appropriate, to put the candidate at ease and also to make the  questioning  flow with the task
• use questions or statements as prompts for keeping focused on the purpose of the questions and the kind of evidence being collected
• keep questions flexible and adjust language to a suitable level
• listen  carefully  to  the  answers  for  opportunities  to  find unexpected evidence

Recording responses

It is important for the assessor to have a record of the questions asked and an indication of the suitability of the responses given by the candidate. If the candidate’s response is insufficient the assessor should record why on the recording sheet or checklist. This provides information that can be used later, if necessary, to explain to the candidate where skills and/or underpinning knowledge need to be developed to achieve the required competence.

Written test

Written test can be useful for assessing underpinning knowledge and for supplementing evidence gathered through real time/real  work and structured activities.

Written test can be asked under test situations or as part of a structured activity.

Assessing written questions

An assessment guide, containing acceptable responses  to  questions, should be developed for all written test given to the candidate, whether the questions are set under test conditions or not. It is good practice to write the expected response(s) to a question as the question is being written. Developing responses to questions helps to ensure that the answers are assessed objectively and consistently, increasing the  reliability and validity of the assessment.

Portfolios

What is a portfolio?

A portfolio contains individual pieces of evidence demonstrating work outputs that have been collected by the candidate. The items are usually produced over a period of time and come from different sources.   A well-constructed portfolio incorporates a selection of evidence that is clearly benchmarked against the relevant unit(s) of competency and indicates consistent performance of work activities in accordance with workplace standards.

As indicated below, portfolios may be organised and presented in a variety of forms:

Portfolio evidence

Portfolios can include a wide variety of evidence. In many cases evidence will come from the candidate’s day to day work. However, the portfolio can also include evidence from other work situations, previously certificated learning and other activities such as recreational pursuits. This evidence may derive from either current performance or past achievements.

The evidence presented in a portfolio must be valid. That is, it must relate to the elements, performance criteria, range statement and  evidence guide of the relevant unit(s) of competency. It must be sufficient and current, demonstrating that the candidate is competent  across the full range of activities described in the relevant unit(s).

Portfolio materials

Form of evidence	Application
Transcripts of records, certificates	Provides evidence of prior education or training completed on or off-the-job.
Assessment feedback or completed checklists	Provides evidence of on-the-job performance, observers’ comments and future action as a result of assessment.
Written	Provides evidence of the candidate’s job

statements or references	performance, responsibilities, achievements and skills levels.
Job descriptions	Provides evidence of prior work experience
Work journals	Provides evidence of tasks, activities or other achievements accomplished by the candidate in the course of days or weeks at the workplace or in a community/volunteer role.
Work samples, for example reports, letters, designs	Provides evidence of the candidate’s ability to do part or whole of work tasks or processes
Finished products, for example tools completed	Provides evidence of candidate’s ability to produce a product or service.
Product descriptions or specifications	Provides evidence that the candidate is aware of the inputs, outputs or standards required to produce a product or provide a service.
Statutory declarations	Provides evidence that work samples are the candidate’s own work.

Portfolios may contain both direct and indirect evidence. A portfolio may include work samples produced by the candidate, performance evidence such as photographs and video tapes and reports  describing what the candidate has been observed doing in the past. It may also contain written evidence describing what the candidate would do in response to emergencies, breakdowns or other contingencies (supplied by the candidate).

There are no set rules about the proportion of direct evidence to indirect evidence in a portfolio. The importance attached to any given item of evidence varies according to its context, age and whether or not it is supported by other evidence.

Evaluating a portfolio

The provision of workplace documents, work samples and other records in a portfolio does not mean that the candidate is competent.  This is the responsibility of the assessor. In evaluating a portfolio the assessor should:

• seek verification of the role of the candidate from others who have witnessed the events documented
  • interpret the currency of the material and the way that the candidate applies the skills in the present context

• obtain explanations of the material from the candidate to clarify the context and relevance to the unit(s) of competency involved (this may be achieved by interviewing the candidate)
  • evaluate the contents of the portfolio using a holistic approach (rather than taking an atomistic approach by checking  each  item in the portfolio against the unit requirements, assessors should consider evidence as a whole or in large integrated  pieces. The benefits of this approach are that it is less time- consuming and allows assessors to concentrate on those areas where there are perceived gaps in the evidence).