Value Neutral Items Increase the Theoretical Relevance and Practical Usefulness of Personality Inventories
Personality is an important aspect of human functioning. It is often measured by self-rating inventories, which are widely used and have some strong advantages, such as quickly and effortlessly providing an overview of an individual’s personality. But there is also a risk that test scores do not reflect the actual personality, but rather how the respondent wants to portray him/herself (i.e. social desirability). The current project will try to do something about this. Earlier attempts to create self-rating inventories without social desirability have often clashed with other aspects of measurement. Our method avoids this by starting out with pre-existing inventories. The guiding principle is to find a phrasing of each item that reduces its desirability without distorting its content. For if it can be avoided that some response alternatives are perceived as more socially desirable than others, the test scores should reflect the actual personality to a higher degree. Since the new version of the inventory has a reformulated version of each item from the original, the risk of missing a trait that the inventory was designed to capture is reduced. Neutralized inventories should have less social desirability-related ”noise” in the measurement of personality, which would make it easier to relate the test scores to actual behavior. The project will also study whether neutralized tests are more resistant to attempts by job applicants to portray themselves in an overly positive way.
Martin Bäckström Lund Uniersity
2010-2015
The main aim of the project was to investigate the effects of evaluative neutralization on the validity of personality inventories. With the method of evaluative neutralization, the items of a personality inventory are rephrased so that the rater does not feel that there is a normative way to rate them. In other words, the items do not have a clear evaluative component in terms of good/bad. This method had already been shown to reduce correlations between factor scales in personality inventories.
The three most important results were:
1. We could show that the method actually diminished the evaluative component in the items and that it could be used successfully by laypeople.
2. We could show that the method does not impair the reliability or validity of the personality inventory. Some aspects of the construct validity of the inventory were improved thanks to the method, and the predictive validity was maintained.
3. We could show that the variation in participants' responses that emerges when the inventory contains many evaluative items is more or less the same variation that several researches have referred to as the general factor of personality (GFP). We could also show that this factor is relatively independent from other personality factors, that the factor is not observed by others (informants), and that persons scoring high on the factor when making self-ratings also tend to score high on the factor when rating others. In other words, some people are more evaluative in terms of good/bad than others when rating personality, but this trait is not perceived by others.
So far the project has generated five papers, three of which are published in peer-reviewed journals and one in a chapter of an edited volume. The fifth paper is under review. The two most important are:
1. Bäckström, Björklund and Larsson (2014). It soon became clear to us that the research community required that we show how our neutralization method can be used to construct a full-scale personality inventory. We worked on developing such an instrument during several years, improving the items based on responses from new respondents and evaluated the final version in an extensive study. The article describes this work and reports the results mentioned at point 2 above.
2. Bäckström and Björklund (2014). This article is a direct follow-up of our first article on evaluative neutralization, from 2009. It contains three studies, where the first relates the evaluative factor to other personality dimensions. The second study includes both self-ratings and informant ratings as a method to validate the evaluative factor. However, the evaluative factor in the target persons' self-ratings did not correspond to the evaluative factor in the informants' ratings of the target persons. In other words, the degree to which my self-ratings are affected by evaluative items does not correlate with the degree to which others' ratings of me are affected by evaluative items. In the third study we showed that people who evaluated themselves positively received higher job-performance ratings by their superiors, also when controlling for the Big 5. This result suggests that people who score high on the evaluative factor may be perceived to behave differently from others, in some contexts.
In addition to these studies we have also collected a data material regarding whether an evaluatively neutralized inventory is resistant to faking and a context that evokes socially desirable responding. There were no clear results and the results have not been published. We have also collected a data material based on experience sampling. Project funds were used to buy mobile phones and we created an application. Several times a day participants responded to questions regarding what they were doing and how they experienced the situation. The goal was to investigate whether those who are evaluative when making self-ratings also are evaluative when rating their everyday behavior. Unfortunately we did not manage to finish the data analysis before writing this report, but hope to be done later in the spring of 2015.
To facilitate international dissemination we are developing evaluatively neturalized instruments in English, German and Chinese. This will make it easier to collaborate with researchers around the world.
Publication strategy
Approximately half of the results that the project has generated have already been published. The results from the experience-sampling study will hopefully be published in a well-reputed personality journal whereas the test instruments in foreign languages will be made accessible in journals in the area of personality measurement.
Results from the project have been presented at several international conferences, such as the European Conference on Personality (ECP16) 2012, Trieste, Italy, the 1st World Conference on Personality, 2013, Stellenbosch, South Africa, and the European Conference on Personality (ECP17) 2014, Lausanne, Switzerland.
During the project there has been an intense debate in journals and conferences concerning whether there is a general factor in personality or not. The project has contributed to this debate and our studies are frequently cited among those critical to the interpretation of a general factor in terms of personality content alone. On this note, our data suggest that the general factor of personality (GFP, e.g. Musek, 2007, Rushton et al. 2008) which is frequently referred to in personality research is practically the same as the evaluative factor that we measure. Furthermore, we have found that when the neutralization process is reversed, so that items are made more (rather than less) evaluative, the general factor increases in size. Taken together, this would suggest that the results of the project, in addition to showing the psychometric feasibility and merits of evaluative neutralization, also has weakened the support for the idea that personality structure is best described in terms of a model with a superordinate factor integrating the factors at lower levels. Instead it opens up for representing the factor as separate, i.e. as parallel to the other factors.
Publications
Bäckström, M., Björklund, F., & Larsson, M. R. (2014). Criterion validity is maintained when items are evaluatively neutralized: Evidence from a full scale FFM-inventory. European Journal of Personality, 28, 620-633.
Bäckström, M., & Björklund, F. (2014). Social desirability in personality inventories: The nature of the evaluative factor. Journal of Individual Differences, 35, 144-157.
Bäckström, M., & Björklund, F. (2013). Social desirability in personality inventories: Symptoms, diagnosis and prescribed cure. Scandinavian Journal of Psychology, 54, 152-159.
Bäckström, M., Björklund, F., & Larsson, M. R. (2012). Social desirability in personality assessment: Outline of a model to explain individual differences. In M. Ziegler, C. MacCann, & R. D. Roberts (Eds.) New perspectives on faking in personality assessment (pp. 201-213). New York: Oxford University Press.