Youth Program Quality Assessment External Assessor

External Assessor

pqa practiceThis workshop gives participants practice using the Youth PQA and increases their accuracy in observing and scoring. The bulk of the workshop consists of practice video scoring.  Participants who successfully achieve acceptable levels of reliability (80% at the item level) through a video reliability check become “Endorsed Assessors” and are able to collect “research quality data”.

Who should attend?

Anyone who will be conducting external assessments as part of a quality assessment and improvement process should attend this workshop. This workshop is also recommended for coaches or network advisors that will be working with sites to complete the self assessment process or to implement improvement plans.

All participants must complete Youth PQA Basics (live or online) prior to attending the External Assessor workshop.  Additionally, it is highly recommended they complete at least one practice observation at a local program. Both of these will help prepare participants for the intensive video practice included in this workshop .

Objectives:

To become reliable in using and scoring the YPQA.

 

Participant Prerequisites:

Although we do not expect that any participant will be an accomplished assessor before taking the training, the following prerequisites will provide an orientation to the Youth or School-Age PQA tools. In addition, the tasks below will give participants familiarity with the items so that more ground can be covered during the live training through discussion of nuance in video examples and written scenarios. In preparation for the External Assessor Reliability Training, all participants should:

  • Complete Youth or School-Age PQA Basics (live 1-day or online 1-3 hour) training prior to attending the External Assessor Reliability training.
  • Complete the pre-training homework assignment (estimated at up to 2 hours).
  • Additionally, it is highly recommended they complete at least one practice observation at a local program.

Training Participation Expectations:

In order to provide the best possible workshop experience and the greatest opportunity for achieving reliability, the following expectations should be met.

  • This is a 2-day (16-hour) training.  It is recommended that the trainings run from 9am-5pm on consecutive days. Exact times can be adjusted if necessary. Participants are expected to attend BOTH days of training from 9am – 5pm. 
  • There is a 1-hour homework assignment after the first day of training.

External Assessor Refresher

This workshop gives participants practice using the Youth PQA and increases their accuracy in observing and scoring. The bulk of the workshop consists of practice video scoring. Participants who successfully achieve acceptable levels of reliability (80% at the item level) through a video reliability check receive “Re-endorsement” and are able to collect “research quality data”. 

Who should attend?

All participants must have previously completed the External Assessor workshop and received a letter of endorsement. It is strongly encouraged that participants review the YPQA Handbook and do a practice observation, especially if they have not collected data in a while, prior to this workshop.

Objectives:

To renew reliablity in using and scoring the YPQA.

 

Participant Prerequisites:

In preparation for the External Assessor Reliability Refresher Training, all participants should:

  • Have successfully completed the External Assessor Reliability Training within 1 calendar year.
  • Complete the pre-training homework assignment (estimated at up to 2 hours).
  • Additionally, it is highly recommended they complete at least one practice observation at a local program within a month before the training.

Training Participation Expectations:

In order to provide the best possible workshop experience and the greatest opportunity for achieving reliability, the following expectations should be met.

  • This is a minimum 1-day (8-hour) training. It is recommended that the trainings run from 9am-5pm . Exact times can be adjusted if necessary.
  • Participants are expected to either return on a second day to take a reliability video OR to complete the reliability check online within 2 weeks of the live training.