Field Operations Unit

“Field operations” consist essentially in the preparations for a survey. The particular tasks that field operations involve are described below through the example of academic performance studies in Germany.

To ensure that such surveys are conducted in a comparable manner in all participating schools, the tests must be administered according to a standardized procedure. For this reason the surveys are, as a rule, conducted by outside test administrators.  

“Field operations” at IEA DPC include the following specific responsibilities:

In consultation with the scientific directors or with the clients, a timetable is developed that encompasses every stage of the project, up to and including the testing that takes place in the schools.

Although the stages required prior to testing are very similar for different tests, the time available for pre-test preparations varies from one study to another. Each stage must therefore be planned in great detail, but with adequate flexibility.

Conducting an academic performance study normally requires the approval of the appropriate educational authority of each German federal state. This means that approval must be obtained, not only for every background questionnaire used in the course of the study, but also for the study procedures and the written information intended for all study participants (pupils, parents and teachers). As a condition for ministerial approval, all of the documents mentioned above must also be submitted for review as to compliance with data protection law.

Once the ministries of education at the German federal state level have notified the schools that they have been selected for a particular study, the schools are informed, in a letter issued jointly by the study’s scientific directors and IEA DPC, of the content and objectives of the study and of the steps to be taken in preparation for the tests. To ensure communication runs as smoothly as possible, each school must designate a responsible contact person, referred to as a school coordinator. All communications with regard to further steps in preparation for the tests are to be conducted between IEA DPC and this school coordinator.

Standardized testing requires that participating schools and test administrators abide by specific guidelines as laid down for each study in the relevant instruction manuals for school coordinators or test administrators. These manuals are prepared by IEA DPC. For international academic performance studies that are also conducted in Germany (e. g., PISA and TIMSS), the manuals are to be translated and adapted to national circumstances and the specific circumstances of each German federal state.

Depending on the study design as conceived by the study's scientific directors, IEA DPC selects, by means of random sampling, either entire classes or individual pupils within an age cohort for participation in the study. So-called class lists or pupil lists, in which the schools list all of their classes or all of their pupils within an age cohort (though not by name), provide the basis for the samples. In very large studies (for example, PISA-E), IEA DPC provides the schools with the appropriate software. IEA DPC also sends the schools a written report of the class or pupil sampling results.

For every study conducted in the schools by outside test administrators, IEA DPC is able to draw on a pool of experienced test administrators in all German federal states that now numbers some 1,200 people. If additional test administrators are required, IEA DPC recruits teachers-in-training or psychology students who are in their final semesters at German universities. Those who are employed as test administrators are required to attend several hours of training conducted by IEA DPC staff. In the course of preparations for the tests, the schools inform IEA DPC of their preferred test dates. IEA DPC then assigns to each school the required test administrators, who then make telephone contact with the schools on their own to make further arrangements.

Once IEA DPC has received the test instruments from the scientific directors, the test booklets and questionnaires are checked, formatted as needed, and finalized for printing. Because test instruments in Germany cannot bear the names of persons participating in a study, they are, in preparation for printing, provided with identification numbers and barcodes that make it possible to merge data from different instruments pertaining to a single person (such as test booklets and pupil questionnaires).

In order to keep the test problems secret before the test, the test materials for the schools are sent, not to the schools themselves, but to the outside test administrators, who do not take the test instruments into the schools until the day of the test. Printing and distribution of test materials — even for studies that are carried out in parallel — must accordingly be organized in such a way that the test instruments for a specific school are received on time by the test administrator responsible for that school.

Svenja Bundt
IEA Data Processing and Research Center
Mexikoring 37
22297 Hamburg
Germany

Phone: +49 40 48 500 601
Fax: +49 40 48 500 608
Email: mailiea-dpc.de