Response to Request for Proposal for the Determining Instructional Purposes (DIP) Program

Submitted To: Far West Laboratories for Educational Research and Development

Submitted By: Jennifer L. Harris EdTech 505: Evaluation of Educational Technology Fall 2012

Introduction The Harris Evaluation Company (HEC) located in Boise, Idaho, submits the following proposal to Far West Laboratories for the evaluation of the Determining Instructional Purposes (DIP) program. The evaluation will consist of trial sites and several data collecting strategies. These will help HEC provide recommendations for and information to Far West Laboratories which will help FWL determine if valuable resources should be committed to the marketing and sales of the Determining Instructional Purposes product. This evaluation is a two fold process where HEC will use data to determine the effectiveness of the Coordinators Handbook before moving forward with the full scale evaluation of the Determining Instructional Purposes program. Program Description Determining Instructional Purposes (DIP) is a training program created by Far West Laboratories for the training of school administrators and graduate students in 3 areas of administration skills. These three skills are supposed to be part of the planning for any effective school program. The DIP program is divided into three units which can be used together or separately depending on individual and site needs. The resources provided include a Coordinators Handbook and the following units: 1-Setting Goals, 2Analyzing Problems, and 3-Deriving Objectives. Modules within units include reading materials, individual activities, small group activities, and feedback. Activities usually apply to a hypothetical situation in a mock school district. The DIP training units require a coordinator to oversee the instruction of materials. While prior knowledge of the subject is not a necessary part of being the coordinator, this leader will need to have worked through any of the units being used in the training. The coordinator will use the Coordinators Handbook, which is part of the training package, when planning and leading activities found in each module. The materials are in the form of print text and range from 155 pages to 259 pages. Each DIP unit requires about 10-18 hours of instruction. The price of individual units is $8.95, while the entire set sells for $24.95. The separate Coordinators Handbook, necessary for any unit, sells for $4.50. Evaluation Method In Far West Laboratories request for an evaluation of the DIP program, FWL provided statements related to the purposes for seeking an evaluation. The primary goal of this evaluation will be to help Far West Laborites make decisions pertaining to the commitment of resources applied towards the marketing and sales of the Determining Instructional Purposes program. The evaluation will also help FWL target school administrators and personnel by providing them with information related to the program. Several data sources will be used as part of the process of evaluating the marketability of Determining Instructional Purposes: Coordinators Pre-Survey and Interview (after Coordinators have the material, but before presenting instruction) Coordinators Post-Survey and Interview (after presenting instruction) Administrators Interview

Sample Staff Survey Prior to the start of the evaluation, HEC will need to meet with FWL, the major stakeholder to discuss specific marketing information which HEC can use develop data collection tools. The future use of these marketing resources can have a major impact on the personnel, funds, and credibility of FWL. Harris Evaluation Group proposes that a test trial of the product be carried out in at least three real world situations with at least 25 participants in each situation. Since the Program Coordinator will be a major factor in the success of the presentation of DIP, it is suggested that a pre-interview and survey be directed especially towards that person. HEC recommends that FWL avoid sending in trainers, and allow individual sites to choose their coordinators. The results of interviews and surveys from program coordinators will be detrimental in determining if the rest of the program will be successful with participants. The survey will address Coordinators comfort level after reading the material, length of time spent on comprehension, and usability. The interview will focus on what the Coordinator feels is most applicable to real world situations and determine if any of the information fails to be successfully presented. HEC makes this suggestion with the idea that Coordinator success will affect participant opinions and success in the program. HEC suggests decisions be made based on the Coordinator interview and survey findings before the trial moves to full scale instruction at the trial sites. After completing the interviews and surveys with the Program Coordinator, it will be important for HEC to meet with FWL developers to discuss results. Decisions will need to be made regarding any recommendations that may result from data, and modifications to the Coordinator Handbook may be suggested. If this occurs, and FWL makes the decision to continue their pursuit of the DIP evaluation, HEC suggests the evaluation start from the beginning in order to determine the usability of the new Coordinators Handbook. Once training has occurred, data will be collected using survey and evaluation tools. These tools will provide the qualitative data necessary for determining the marketability of the DIP program. Program Coordinators and the administrators, who were seeking this type of professional development tool, will be interviewed to find out what aspects of the DIP materials are user friendly, applicable, and if DIP met expectations. They will also be surveyed to determine what areas of the program they would suggest to other administrators, educators, and personnel, in terms of materials and cost-effectiveness. These surveys and interviews will be used to help the Harris Evaluation Company and Far West Laboratories determine if resources should be used and/or continued to be allocated towards Determining Instructional Purposes. The information regarding user friendliness, materials, and cost-effectiveness will aid in providing information to the potential buyers as requested by FWL. Data regarding the usability and materials, as well as administrator opinions about success could be used to help FWL determine if the product is marketable. HEC also suggests that an interview with the administrator who is seeking professional development tools be used to determine individual needs and situations for marketability and sales. This interview will also aid in the collection of data related to FWLs secondary goal of providing buyers with information regarding the program. The purpose of this interview is to outline what administrators and other personnel were looking for in the use of the program and determine if DIP met the needs of participants.

Finally, a group of sample staff members for each site will be surveyed. The purpose of this data collection tool is to compile qualitative data on the DIP experience. These surveys will provide information pertaining to marketing and sales points concerning: Language Level Higher level thinking Relationship between activities and actual situations Application to district needs Recommendations Tentative Schedule The schedule below was created with the assumption that Far West Laboratories would be willing to establish a trial program with three sites within the state so that information related to the use and marketability of the DIP product can be evaluated. The schedule also assumes that the trial site coordinators will have the Coordinators Handbook two months in advance (November 12th, 2012) to read and prepare, allowing them to speak intelligently in the surveys and interviews provided by the evaluation team. Task Meet with FWL staff, discuss proposal and make modifications Develop Coordinator Interviews and Surveys Submit drafts of surveys and interviews, modify if necessary. Train interns Collect and summarize data from Coordinator Interviews and Surveys Meet with FWL staff to discuss results, make decisions based on Coordinators Handbook Survey and Interview. Possible decisions to:
1. Continue evaluation as is 2. Revamp Coordinators Handbook 3. End evaluation 4. Other outcome Allotted for sites implement DIP program

Responsibility Deadline HEC & FWL January 11th, 2013 (1 week) HEC February 1st (3 weeks) HEC & FWL February 22nd (3 weeks) HEC HEC & FWL March 22nd (4 weeks) April 5th (2 Weeks)

Sites

September 6th (6 months)* April 26th (3 weeks)* May 10th (2 weeks)* September 20th

Develop post surveys and interviews HEC Meet with FWL staff - revise post surveys and HEC & FWL interviews Collect data: HEC Surveys and Interviews:

Coordinator Post Survey & Interview Administrator Interview

-Within 2 weeks of training completion according to site training schedules (6 weeks)* November 1st (6 weeks)*

Sampled Staff Survey Compile Data and Create Evaluation Report HEC

*indicates overlaps in time Project Personnel Dr. Turanga Leela, is tenured professor at the esteemed Mars University. She has previously taught courses in Evaluation Tools, Evaluation Methodologies, and Writing the Evaluation Report. Additionally, Dr. Leela has been selected to be a member of the ESA (Evaluating School Administration) Project commissioned by the mayor of New-New York. Dr. Leela has a Ph.D. in Sociology from the Futurama Conservatory. Dr. Leela will be responsible developing all data collection tools. Dr. Leela will also interpret data and write the evaluation report.. In addition to the personnel mentioned above, HEC will also provide two graduate student interns, from Mars University, who will be trained by Dr. Leela in the collection and administration of surveys and interviews. A secretary will also be provided for use during the data collection phases. Budget Far West Laboratories has set a maximum budget of $50,000 for this evaluation project. The table below shows a breakdown of these costs for the proposal provided by Harris Evaluation Company. Please note that the travel and per diem costs reflect the use of three sites in the evaluation project and assumes that all sites will be evaluated within the state of Idaho. The maximum budget is not attainable if sites that are part of the evaluation process are located outside state lines, or Far West Laboratories wishes to use more than three sites for the evaluation of the DIP program. Additionally, the budget maximum will be exceeded if the Coordinators Handbook has to be reevaluated. Proposed Budget Personnel: Professional Salaries: Dr. Tarunga Leela: 80 days at $400/day Intern 1: 50 days at $96/day Intern 2: 50 days at $96/day

$32,000 $4800 $4800

Project Secretary: 25 days at $120/day

$3000 $44,600 $44,600

Travel and per Diem: 150 mi/day for 30 days at $.30/mi Miscellaneous mileage: 2500 miles at $.30/mi Three 2 day site visits by Dr. Leela at $200/visit

$1350 $750 $600 $2700 $2700

Communications: Telephone and Internet ($120/month for 10 months) $1200 Postage $500 $1700 Supplies, Materials, and Photocopying Photocopying Supplies and Materials $1700

$500 $500 $1000 TOTAL BUDGET $1000 $50,000