Computerized Overview of a
Complex Field Process

Ruth Sheshinski

Data Processing Division

Tel. 972-2-655 3481

FAX 972-2-655 3531

e-mail: sheshins@cbs.cov.il

Introduction

When planning complex field activity, combining the employment of thousands of
workers spread over a wide geographical area, and involving different operational
procedures and many managerial levels, planners must also provide for constant
back-up support and for recovery procedures from any failure which may occur in
the fieldwork. Similarly, the planner must find a way to enforce the planned methods
and ensure that all instructions and procedures, without exception, are understood
and complied with.

The census' geographical breakdown parallels the organizational division of the
fieldwork. Regions were divided into sub-regions, which were sub-divided into
sections, and sections were divided into Enumeration Areas (EA), with one person in
charge of each geographical unit. Census material comes in from the field from a
wide range of very different persons, and it would be difficult to detect the
influence of any subjective element on the census' 'sterile' by-the-book field
procedures. The source of the problem is the fact that there is no option of
reversibility in the case of changes occurring in enumerator and supervisor work
procedures, since enumerators and supervisors are the only ones in personal
contact with respondents and who physically handle census questionnaires. The
objective of computerizing fieldwork is to aid in the creation of rigid, standardized
procedures which would lessen the subjective element.

1. Target of Computerization: to construct a
"parachute" as well as a "harness" on fieldwork
activity

Fieldwork supervision is designed, on the one hand, to give the fieldworker a large
"parachute" to convey the feeling that his working environment is supportive of him
at all levels, and on the other hand, to prevent deviations from pre-planned
methodologies or changes in procedures. The principle objective of computerizing
fieldwork is to enable the enumerator to achieve good coverage of all addresses --
all dwellings, all households in those dwellings, and all individuals belonging to the
census population and to pick-up questionnaires of good quality. Supervision is a
secondary objective that complements the main coverage objective by preventing
deviations from procedures tested in early trials.

Operationally, supervision, especially computerized checks, of enumerator's work
leads to improved quality of the material and forms, collected by the enumerator,
better coverage and completeness.

The enumerator's main tool of work is the Enumerator's Report Book (ERB),
which the enumerator uses, to all intents and purposes, to conduct a "census of
addresses and dwellings", at the questionnaire drop-off stage. A systematic
canvassing of buildings and dwellings with an indicator which sort of questionnaire
(long form or short form) was left at the dwelling, are written on the Visitation
Record Pages (VRP) in the ERB. This information serves as the basis for efficient
collection of questionnaires. At the later Pick-UP stage, essential values, as the size
of the household, the type of the dwelling, are added to the VRB. The ERB
constitutes, therefore, a field worker's work summary record.

A properly maintained ERB indicates from which addresses a questionnaire should
and should not be obtained. Once the census is concluded, preliminary estimates of
population, households enumerated, households that refused to be enumerated,
inhabited closed dwellings, and uninhabited vacant dwellings are available. If ERB
records are orderly, uniform and internally consistent, then the work of imputation
of households and individuals to dwellings that provided no data, or only partial
data, can be based on such records. From the administrative point of view, a well-
maintained ERB also simplifies the manpower management system and payments for
enumeration.

Computerization of the Enumerator's Report Book, which is, in essence, the
"heart" of "fieldwork computerization", facilitates execution of the tasks and achieves
the objectives mentioned above. It enabled the regional offices to receive daily
progress reports, which were then transmitted via communication (modem) to census
Headquarters, so that early problem detection and immediate response was possible.

The problems detected were: slow pace of work, Enumeration Areas whose
enumeration was especially difficult, unsuitable enumerators, the need for dismissal
and replacement of workers, etc. the Headquarters could look into the detailed field
work from the top to the last enumeration area.

ERB computerization permitted immediate calculation of enumerator output, i.e.
completed questionnaires or explanations of why completed questionnaires had not
been obtained, such that at the end of the canvassing of the Enumeration Area, the
enumerator signed his name on the Final Production Report to confirm his output
and production. In the 1983 census, the process of calculating output and wages
payments took a great deal of time, entailing numerous law-suits and years of
litigation. In the 1995 census, this process was concluded in less than thirty days
and less than ten complaints out of circa ten thousand workers.

The computerized system had a built in model of evaluating the rank of difficulty
which the enumerator encountered, and the endorsement was done according to these
ranks, with the full knowledge of the enumerator.

Other calculations that could be generated immediately were preliminary estimates of
population size, number of households, and number of dwellings.

2. Control Methods for Verification of
Coverage and Quality

2.1 Logic Checks, Cross-checking, Quality Control

ERB data were keyed in every two or three days, logic-checked by computer
program, and also checked manually by the supervisors. This process compelled
enumerators and supervisors to comply with instructions and procedures. The
computer checks resulted in a series of reports that enabled supervisors to check
up on every stage of the enumerators' work.

Examples of logic checks at the drop-off stage:

a. Problems detected as a result of checking Visitation Record Pages (VRP) were
recorded in the report in detail. For example: In each EA, the assignment of
sample digits ensured that 20% of dwellings would be given the long forms. If the
enumerator recorded that he had given a short form to a household, but the random
digits designated that this dwelling should be enumerated with a long form, the report
would state the error message: "Sample dwelling received short form". The
enumerator would be told by his supervisor to return to the address, apologize, for
the error and to replace the form. In this way, it was assured that an enumerator
could not choose which households would be enumerated using the long form.

b. In order to improve coverage and to verify completeness, each enumerator was
issued detailed maps of his EA and instructions designed to make non-compliance
extremely difficult. At the end of the drop-off stage, when the enumerator affirmed
that he had finished canvassing all the addresses and dwellings in his EA and had
dropped off questionnaires, the file of the enumeration's area addresses was
compared with the Geographical Information System's (GIS) file of the EA . The
report on this check supplied the supervisor with a listing of all addresses in that
EA map, generated by GIS, which did not appear in the address file captured from
the Visitation Record Pages. The report revealed information to the supervisor
which guided him to send the enumerator to all those addresses in order to complete
the drop-off procedure. Inconsistencies were not always caused by enumerator
errors. At times, the geographical information was not accurate.

Example of a logic check at the pick-up stage:

Computer checks at the questionnaire pick-up stage detected inconsistencies in ERB
records and problems with the values of variables that were recorded, both on the
questionnaires' First Page and in the ERB. Two of the principle variables common
to both the questionnaires and the ERB are type of household (regular, multiple
households in one dwelling, refusals, closed dwellings, etc.) and number of
individuals in household. It was the supervisor's responsibility to ensure that the
value recorded in the ERB was also the value recorded on the questionnaire. In
practice, this also generated an indirect quality-check on the completion and quality
of these fields in the questionnaire.

2.2 Absorption of Field Information and Routing of Information to
the Decision Making Levels

The computer-generated real-time reports equipped supervisors with an efficient
monitoring and control tool. The cumulative data permitted the generation of interim
summary and final summary reports for the various levels of management, depending
on position and authorization. The reports to supervisors related to individual EAs;
EA summaries gave an updated picture of the section; and section summaries
permitted the sub-regional director (SRD) to check progress, as far as both outputs
and quality of material concerned are concerned, over a sub-region. EA and
summary data were transmitted to the Regional-Office and from there to census
Headquarters.

Heuristic difficulties and problems which derived from certain behaviors of sub-
populations, that were not always compatible with the rigid procedure dictated by a
computerized process forced the intervention of decision-makers to meet requests
from the field to release requirements.

Computerization did not allow unrestrained liberties to be taken. It had strict
standards of commencement and closure and precisely laid-down rules of procedure.
Even those modifications of procedure that were necessary, were carried out within
the framework of rules and standards for the keying-in and checking of ERB
data.

The following are three examples of procedural modifications and their integration
into the computerized system:


1.	In some locations, the population, which refused to be enumerated, threatened enumerators, and even used violence, forced the system to gather the information indirectly, from centrally held records and from the localities' official institutions. This data, which were of lower quality, were keyed in and checked by regular testing procedures.
2.	In the enumeration procedure for another 'refuser' population, it was necessary to skip over the pick-up stage and the pre-clean-up stage. As soon as all buildings and dwellings had been recorded, enumerators were allowed to proceed to clean-up and EA closing, and complete refusal questionnaires (empty questionnaires, recording only the address, the classification of the dwelling as a refusal, and the number of inhabitants in the dwelling, without any descriptive data).
3.	In remote localities, problems of transportation difficulties, distance and security reasons made it impossible for enumerators to appear at the Sub- Regional-Office every two days. Headquarters allowed a process whereby the supervisor would collect the ERBs from the field, bring them to the Sub- Regional-Office for keying-in and logic-checking and return them to the enumerators the next day for resumption of work. In some of these localities, it was even agreed to combine the census and household enumeration stages if self-completion was not possible (in which case enumerators would complete the questionnaires).

In the above three cases, despite modifications and adjustment of the procedure to
field conditions, computerization obliged enumerators and supervisors to confine the
methods to the same fieldwork mold in which material is keyed in and checked.

The computer also helped define the variability in working conditions from EA to
EA. Since enumeration conditions could not be uniform in all EAs, differential
enumerator effort was required to obtain the same number of questionnaires.

Enumeration in difficult EA's, where there were difficult physical conditions or
where the population was unwilling to participate in the census, entitled the
enumerator to extra payment; the decision about which EA's should be classified
'difficult' was not free of subjective judgment.

We constructed a methodology, which used a battery of questions (a structured
questionnaire), to characterize any given EA on three criteria: geography of the
EA, objectively difficult conditions (dogs that bite, dark stairwells, language
difficulties, etc.), and respondent behavior (refusals, violence, lack of cooperation
on ideological or other grounds).

During preparation of EAs prior to the entry of the enumerators to the field,
supervisors filled out the above-mentioned questionnaire, and later, during the pick-
up stage, interviewed their enumerators on the subject. In an attempt to keep the
questionnaire answers objective, the completed questionnaires were transferred to
the Regional-Office for data capture and ranking, in order to score every EA.
These scores were transferred to Headquarters. There, according to a model,
designed with the use of simulation techniques and based on the normal statistical
distribution, a grade of difficulty was determined for every EA. About 85% were
Regular enumeration, 10% - Special, 5% - difficult. These decisions were made with
budgetary limitations, which were known in advance, being taken into account. The
model permitted manual intervention by the Census Director (or his representative) in
order to manually change the status of the EA characterization in cases in which an
appeal of the score was accepted.

This computerized model and the ability to adapt enumerator payment to EA difficulty
reduced much of the resentment in the field.

Another channel of information flow passed through the Help Desk (HD). An EA that
was mired in insuperable difficulties was taken over in its entirety by one of the
HD support staff, who overrode the local computer and solved the problem. Most
such problems occurred during the transition in the fieldwork from the pick-up stage
to the clean-up and closing stage. Since we did not permit a return to the previous
stage, HD intervention was required to release the locked stage in cases where the
pick-up stage had to be revisited in the file.

2.3 Non-computer-based Checks

The computer is elegant, speedy and effective, but the human brain and human
communications are priceless. The process is, therefore, supported by continuous
interaction between supervisor (and, occasionally, the Sub-Regional-Officer) and
enumerators. At the beginning of every enumeration stage (pick-up, drop-off),
supervisors accompanied enumerators on their route, and no enumerator could begin
work without this escorted activity. In addition, supervisors were instructed to
conduct fieldwork checks, to meet every enumerator in his EA and to verify
coverage of the EA. Since enumeration is conducted along the enumeration route
drawn by the supervisor in the preparation stage, checking that a sample of
dwellings along the route appear in the expected order would give the supervisor
the ability to assess the quality of coverage.

The completeness of response in each questionnaire was checked manually by
supervisors and, where questions designated 'critical' had not been answered, the
questionnaire was returned to the enumerator for further processing.

During the pick-up stage, quality inspectors would appear without warning at randomly
selected sub-regional-offices to verify that the supervisor had processed
questionnaires correctly, that no critical variables were missing, and that there were
no inconsistencies between questionnaire responses and ERB records. These
checks would be conducted on a sample of questionnaires.

In addition, senior staffers from the Central Bureau of Statistics were sent out to
the sub-regional-offices after a short period of training. Their task was to evaluate
the enumeration process with the eye of a layman -- on the assumption that 'a guest
in a hurry sees all the worry' -- to warn of requirements not being met or
deviations from methods and procedure. Members of the planning team and census
Headquarters staffers traveled almost daily to regional and sub-regional-offices, to
see up close that field procedures were being properly implemented as planned.

3. Difficulty with and Objections to
Computerized "Assist and Control" Field
System

At the planning stage, it was not simple to convince people of the value of
computerizing field activity, back-up and oversight. At all census pilot trials, we
introduced the computerized elements gradually. Beginnings were modest (electronic
spreadsheet and a few computerized checks). Seeing the advantage of computerized
checks - helped convince doubters. It was easier to make a decision based on an
existing product, rather than to make a decision based on a planned product.

At the development stage, an expansive array of desires and aspirations was
defined. From a project which began with the idea of computerizing only the
enumeration report book alone, a system was developed that also included
computerization of the areas of manpower, administration and payments. Because of
other tasks imposed on the development process, this process was prolonged until
very close to the census commencement date, and the early days of the census were
marked by difficulties in integrating the various systems. In some cases,
modifications were necessary in areas of manpower administration, such as the
absorption and mobility of enumerators, even during the census itself.

Another problem at the development stage, which stemmed from disagreement between
planners and developers, was regarding features of the human-machine interface.
With hindsight, we could probably have produced a more user-friendly system.

At the training stage, keying operators received training which was too superficial,
but they were quickly able to overcome the initial difficulties, by trial and error.
Enumerators and supervisors were very enthusiastic about the computer system,
were positive and very interested. They saw the computerized system as an
indispensable basic tool. It was the senior ranks that had the most difficulty in
developing correct work habits, since they were generally older, less computer-
literate and less able to exploit the advantages offered by the computerized system.

4. Results: One File Serving Many Different
Activities and Applications

The very day census enumeration ended we had at our disposal a complete ERB
computer file, containing summary data with respect to all categories of dwellings, all
categories of households, and population size, all at maximal geographical level of
detail. Consequently, it was possible to begin publishing provisional data on the
results of the census.

Other Bureau units are using the ERB file for a wide range of post-census
activities. The file enabled a systematic sampling of addresses for the purpose of
surveying enumeration evaluation. During the census, once the drop-off stage was
terminated, we sampled the enumeration routes used in the census itself, to get a
sample for the Post Enumeration Survey (PES). The PES enumerators could be sent
to specific enumeration routes, armed with special questionnaires, for evaluating
coverage. Other units at the Central Bureau of Statistics use the ERB file to draw
samples for other surveys, pending the availability of the full census file.

The computerized ERB file is used to research the nationwide proportion of vacant
dwellings and non-residential buildings in Israel. This information, by the way, had
not been set as an objective in the process; in practice, it is a by-product of the
enumeration process and of the computerization of the ERB.

The computerized file serves as evidence that the field procedures had been
uniformly observed and did not deviate from plan, so that it could be known, to a
high degree of precision, just what errors the file does and does not contain. One
result of this is that the ERB file serves us at every stage of control and cleaning
of the census file (from the questionnaires). All census questionnaire data were
compared against all aggregate ERB data to ascertain that the data-processing
procedures were not flawed.

The addresses in the ERB file are a source of updating the GIS addresses:
Addresses are corrected, synonyms are assigned, new addresses are added, and
the zoning category of all buildings at those addresses are updated.

Computerized Overview of a Complex Field Process

Introduction

1. Target of Computerization: to construct a "parachute" as well as a "harness" on fieldwork activity

2. Control Methods for Verification of Coverage and Quality