Methodological approach to measuring the impact of traffic education and information measures.

Link to the publisher of the dissertation

Abstract

Traffic education and awareness seek to increase traffic safety by influencing human behavior. Traffic education and information encompass a range of different measures. They can be distinguished in terms of target group, mode of traffic participation, mode of implementation, level of delivery, spatial scope, and duration. Many measures are financed by public funds. In general, it is important that these funds in particular are used effectively. This is becoming increasingly important due to financial constraints in the public sector. The effectiveness of traffic education measures is viewed controversially. Even though on the one hand it is assumed that an effect of such measures cannot be proven, on the other hand impact measurements document such an effect. The dissertation presented here shows methodical procedures for effect measurements of traffic education measures. In a first step, general steps in impact measurement were identified. These include the definition of the research hypotheses, the selection and design of the survey instrument, the study design, the sample selection, and the application of statistical tests for ignificance testing. Furthermore, a systematic review of 36 international studies that included an impact measurement of a traffic education intervention was conducted. Studies were used in which the methodological procedures were documented. The systematic review shows that the methodological requirements are often not met. This concerns, for example, the lack of specification of objectives, impact models, and hypotheses. Furthermore, weaker study designs are often used. Although predominantly no random sampling was used, significance tests are carried out. Here, a concentration on particularly well-known methods, such as the t-test or the chi-square test,can be observed.Building on the theoretical requirements for impact measurements and the systematic review, recommendations for action were derived. On the one hand, correlations between the distinguishing characteristics of the measures and the methodological approaches were described. For example, the forms of implementation influence the possibilities for forming a control group. In addition, it was analyzed how the methodological approaches influence each other. For example, the survey instrument influences the outcome indicators, the possibilities for random selection and data collection. Furthermore, the following important standards were defined:

• A clear target definition of the measure (inc!. impact duration and intensity).
• A description of the theoretical impact model
• A description of the goals of the impact measurement, whereby different impact areas or levels are to be specified
• A statement of the hypotheses, including statistical hypotheses if applicableA statement of the survey unit
• A (conscious) consideration of the advantages and disadvantages of different survey instruments, if necessary the use of a mix of instruments
• The use of existing data sets, if applicable
• The use of "stronger" survey designs
• The similarity or equal size of the study group or room and the control group or room
• The measurement of long-term effects (approximately one year after the intervention)
• A reasonable combination of different outcome indicators
• The formation of a random sample
• The collection and use of relative data
• The minimization of confounding factors in data collection
• Attention to the "response problem" (in the case of written surveys)
• The attention of the data shrinkage (with multiple measurements)
• If necessary, the qualitative "upgrading" of the sample
• The use of the appropriate statistical method
• Consideration of potential methodological weaknesses
• The execution of plausibility checks
• Comparison of the results with existing studies, if necessary
• Documentation of the methodological approach

The standards serve as a guide for conducting an impact measurement of a traffic education intervention. It is important that some minimum standards are followed at each methodological step. However, impact measurement is a complex field and individual considerations must continue to be made in each specific case.The applicability of the standards could be demonstrated on the basis of five case studies. Even for smaller measures such as actions, it is possible to achieve a high methodological quality. However, it could also be shown that in individual cases there are good reasons for not following a standard. The effort of impact measurement should be in proportion to the measure. In any case, potential methodological weaknesses should be taken into account when interpreting the results. Limiting factors for impact measurement in practice are likely to be the willingness to measure impact and the amount of funding available. It is recommended that fewer interventions be subjected to high quality impact measurement rather than many of lower quality. The impact measurements should be published and be exemplary for other measures.

This Abstract was translated from German with deepL and could be faulty.