Validity and Reliability of iWAM

A question asked by individuals and organizations encountering assessments in general, or the iWAM specifically for the first time is: "Has it been validated?"

The Three Crucial Questions

1. What is the iWAM measuring and is it relevant?
2. Does it measure what it says it does (the validity question)?
3. Does it measure it consistently over time (the reliability question)?

Relevance

The iWAM measures Motivational and Attitudinal Patterns in the work context, which influence what we pay attention to, how we think, and how we behave.

These patterns (using an iWAM Model of Excellence) can predict and explain from 40% to 70% of the actual work performance of managers, professionals, and staff. This suggests what the iWAM measures has a significantly higher correlation with performance and relevance in the work-context than factors measured by other assessment tools.

Validity

There are several kinds of validity, each having a different way of assessing a test. "Face validity," for example, asks generally whether the test appears to be a valid way of measuring what it says it measures. The original validation of the motivational and attitudinal patterns (what neurolinguistic psychologists call "metaprograms") was done with the Language and Behavior (LAB) Profile interview technique, which is the other tool for measuring metaprograms. The metaprograms and the iWAM proved to be valid. 

  • Based on the feedback form, on average people agree with 90% of the patterns. Further analysis shows that half of the disagreement comes from people not liking the 'verdict', even though the description is correct.
  • Each of the 200 statements embedded in the 40 questions of the iWAM have been designed to measure a specific metaprogram (motivational or attitudinal pattern). The correlation between the metaprograms in different categories are below 0.25, indicating that they are independent constructs.

Reliability

The reliability of an assessment is most often established in a test-retest study where the instrument is re-administered to individuals who took it within a given period of time.

  • 95% of the results can be confirmed during a retest using a LAB Profile interview, another validated tool for measuring metaprograms.
  • In a test-retest research, where people took the test a second time one month later, the score for most patterns remained within 5% of the original score, giving virtually the same results.

Traditional test-retest techniques designed for traits that remain stable are not as effective when measuring motivational and attitudinal patterns that will often shift with context or over time. Any shift in context may cause a shift in the re-test condition. For example, interpreting the results of the test to someone may well have an effect on how they respond to the test a second time. We also find that explaining some scales changes how the individual thinks of an item. Note, however, that some isolated test-retest cases with individual executives yielded some changes in MAPs that were consistent with changes in the positions they occupied.


Materials on iWAM's Validity and Reliability

Rather than fill the page with text, we provide some reference materials and articles to review under the research section (e.g. controlled reliability studies).

Latest News: