Catherine Hernandez
Assessment Plan
Instructional System Design
Dr Aimee Klimczak
April 16, 2006

Participants:

I have six participants for my trial instruction and assessment. There are two “veteran programmers” who are students in the lab environment, and very well versed with all of the programming environments provided in our lab settings. Jasmine and Nathan will be effective “expert” users who can provide valuable insights to the completeness of our course content. They will also be able to provide information and suggestions about the delivery process and how we can maintain interest among advanced users. I have two seasoned technical support staff who can offer a “middle ground” point of view about our instruction. Jason and Jack are not avid programmers, but have completed the undergraduate course of study and have a good understanding of the curriculum and our programming environments. Lastly, I have two testers who are not seasoned programmers but know how the technical support environment works. Stephen and Carlos are familiar with the applications we support, but are not in the CS programming track; rather they are MIS majors from the College of Business with a firm grounding in technology other than programming.

Process of Instruction:

The pilot test will be held in our Instructional Lab. This room is equipped with a computer for each student running all the applications and software needed for the curriculum and course work required in our undergraduate programs. There is also an instructor’s computer, with the same software configuration, connected to a projector. All machines have internet connections and appropriate browsers.

In simple terms, instruction will be focused around locating and launching the applications on the systems; learning how to import sample code from instructor websites; then compiling, running and saving the output of the programs. These activities will be modeled and discussed in the lab and then practiced in hands-on experimentation by the participants.

The training will be held during our regular staff meetings which frequently have training components. The structure of how each session will be portioned is still being determined, but preliminary timing shows us that we will require a separate session for each of the three platforms and one additional, introductory session describing file types, courses affected and download options and procedures. With a schedule like this, all modules of this training can be completed in one month, if there are training sessions once a week for four weeks. 

Instructional Materials:

Participants will be provided brochures which describe the processes being discussed. These brochures are the materials which we provide to students in the labs, and so are important learning materials for the PD participants on two counts: first, so that they can learn how to interact with the systems and second, so they are familiar with the tools being provided to our users and how those tools can be used to help students with their work. These materials are still underdevelopment at this time. 

We will work with and provide participants with materials located on the web describing JAVA programming generally and these programming environments, specifically. 
	http://www.eclipse.org/documentation/main.html
	http://www.netbeans.org/kb/50/index.html

The Instructional Lab itself is a resource, as it is the exact environment our students will be using, and is the environment my staff be working in with our users. Working directly on this equipment with these applications is the most direct method I can conceive of to teach this topic. 

Assessment Tools and Procedures:

The helpdesk staff being trained happen to be peers to the students they are helping. This environment lends itself well to Peer Assessments. Each term, students are asked to complete a survey regarding how well the computer lab, staff and services met their needs. One method to assess how well my staff have learned and implemented the new learning is to include two or three questions in the lab survey at the end of each term. This will allow my staff to be assessed by their peers, and will be a recurring assessment which will follow training staff will have at the beginning of each semester. Sample questions might include:

1.	How effective are lab assistants in helping you navigate through JAVA programming environments like Eclipse and NetBeans?
Very effective
			Somewhat effective
			Not very effective
			I have not used these applications

2.	How helpful do you find the reference materials and resources in the lab and on the lab website?
Very effective
			Somewhat effective
			Not very effective
			I have not used these resources

3.	How effective are the lab assistants in answering your questions about “run”, “compile” and “saving” JAVA programs with Eclipse and NetBeans?
Very effective
			Somewhat effective
			Not very effective
			I have not used these applications

The second way I plan to assess my staff is with a Performance Assessment which will occur on-the-job. This is an observational, informal method of assessment. I spend much of my time managing and observing my staff during their work. In order to assess the effectiveness of this training, I will spend time in the lab during peak usage at the beginning and middle of the semester, when projects are due and students are learning the environment. I will observe each staff member and make notes about how well they are able to answer students’ questions about the programming environment. A method I frequently use with my staff is to make observational notes and e-mail them to myself, filing them away for their next evaluation period. This keeps all the details I observe about their performance at hand and makes incorporating that information into other documents (evaluations, letters of recommendation, etc.) very simple.

The last method of assessment I plan to include will be a Simulation exercise during the training session. After exploring several examples of using the programming environments, a hands-on experiment in which my staff must use the applications will clearly expose who understands the programs and who needs more review and training.

In order to determine how well the training was received by the participants, we will administer and “attitude questionnaire.” A draft of this instrument follows:

On a scale of 1 (worst) to 5 (best) please rate the TRAINING you received: 
1. This training was timely and relevant to my daily work
2. The content met my expectations 
3. The provided materials were high-quality
4. The provided materials were useful 
5. The time commitment was appropriate

On a scale of 1 (worst) to 5 (best) please rate your INSTRUCTORS: 
1. Instructors were effective
2. Instructors covered concepts clearly and concisely
3. Instructors appeared to be well practiced and confident 
4. Instruction was at a pace suitable to the audience
5. Instructor left room for questions & answered each thoroughly  

Media Analysis:

Media has been covered effectively in the areas of Materials and Implementation Process. 

References:

I am working with one of my senior staff, who is proficient with our programming environments, to develop the core of the material to be covered. Tom Langston is sought by many students in our open undergraduate lab to assist with questions about the tools and process of computer programming. It was Tom Langston who brought to our attention the need to have better trained staff and more detailed student resources.

My guidelines for the “attitude questionnaire” were based heavily on the following example found at WiderNet.org:
http://www.widernet.org/TechTraining/TechTraining01/TT01Evals.htm

Additionally, as noted above, we are using the online documentation for each programming environment as a resource for our materials. We will provide these references to our students and staff so that they can delve deeper in to the material as they are interested and able. 

Appendices:

No specific additional documents or instruments have been developed which have not been presented above.