Basics of Educational Evaluation

Abstract

It is usually helpful to define evaluation objects according to an input, process, output and context framework (see Fig. 2.1). This basic framework will be used throughout this book as a model to categorize different kind of educational content. The model can be defined at different levels of aggregation. The central box can be viewed as the national education system, a particular program an individual school, classroom and even student. It is important to note that the model is dynamic in the sense that it views education as a production function: educational inputs are transformed to educational outputs. Typical inputs are material and financial resources, process characteristics are organizational and instructional structures and process, outputs of schooling are, for example, scores on achievement tests and relevant context aspects are, for example, attainment targets set by a higher administrative level. Each of these abstract entities, inputs, processes, outputs and contextual conditions may be evaluated on its own. Accordingly a distinction can be made between input, process, output and context evaluation; of these four, process and output evaluation are most commonly encountered. In the case of input evaluation the actual financial resources of a national system, program or school may be described and judged according to the level that is thought to be necessarily in order context outputs inputs Process or throughput school level classroom level to keep the system running. Process indicators may be assessed by comparing them to generally accepted ideas on educational good practice, while outputs can be judged according to pre-fixed attainment levels, margins of variability that are deemed acceptable and by comparing them to other relevant situations. Indicators of the context of, for example, a school may be judged with respect to their being considered as favorable or unfavorable to a proper functioning of the school. The terms evaluation criterion and evaluation standard are often confused. The criterion is the dimension on which the evaluative interpretations are ultimately made. For example, a math test can be used as the criterion in an educational evaluation. The standard refers to two things: the criterion (in the sense just defined in the above) and a norm on the basis of which it can be decided whether a “success” or a “failure” has been achieved. Cutting scores defined on a particular achievement test provide an example of a standard. In this case, the example of a cutting score, the standard is absolute. An alternative is the use of comparative standard; for example the statistical significance of difference in mean scores of a treated and a control group. 2.2.2 Measurement of criteria and antecedent conditions Measuring outcomes In many educational evaluations attainment, in the sense of scores on achievement tests in particular subject-matter areas, is the central criterion. In such situations achievement tests are used as the operationalization of educational goals or objectives. When such objectives are tied to subject-matter areas the process of operationalization and test construction is quite straightforward. The basic steps of this process are as follows: x precise statement of the general educational goal, e.g. pupils should master all aspects of numeracy at the level of the final grade of primary school; x delineation of components, e.g. fractions, mental arithmetic, decimal system, etc.; x further specification of subject-matter elements and required skills for each component, e.g. multiplication, addition and division of fractions; x ordering of subject-matter elements and types of problems/questions/items according to difficulty-level; x formulation of specific questions, items, problems; x scaling of items and questions, i.e. determining on the basis of try-outs and by means of specific psychometric analyses whether sets of items are homogeneous and can be placed on a one-dimensional continuum or scale; x further analyses concerning the reliability and validity of the tests (in other words, the provisionally scaled sets of items).

Cite this work

No public copy is stored for this record, so only the metadata is cited.

Stored only when both papers are in the corpus — 67 citations exist upstream.