width=116While corporations routinely utilize scorecard measurements to evaluate their strategic performance, the interpretation of this data can sometimes be clouded by the preconceived notions and motivations of those in charge, says William B. Tayler, assistant professor of accounting at Emory University's Goizueta Business School. In a research paper titled The Balanced Scorecard as a Strategy-Evaluation Tool: The Effects of Responsibility and Causal-Chain Focus, Tayler discusses how managers are more likely to favorably review the corporate initiatives they created.

The paper notes that prior research indicates that about half of all corporations use the balanced scorecard for performance measurement. Given this widespread use, Tayler believes it is critical to understand the scorecard's pitfalls. His research paper specifically focuses on ways to prevent the impact of motivated reasoning on interpretation of scorecard data, or what the author refers to as one of many possible psychological forces that are likely to limit the effectiveness of the scorecard. Tayler adds, Motivated reasoning is a powerful bias; just knowing about the problem is not likely to be enough to avoid it in all cases. Often, the solution will be to adjust motivations so that motivated reasoning isn't one-sided.

Specifically, the results of the research indicate that managers involved in designing a strategic initiative tend to perceive those initiatives as having been more successful than managers who are not directly involved in the process. Tayler concludes that motivated reasoning can explain why some firms have found more success than others with scorecards. The paper states that allowing managers to take charge of the selection of scorecard measures can mitigate the effect.

But this mitigating effect happens only when the balanced scorecard is framed as a hypothesized causal chain of performance vs. a classic four separate balanced-scorecard perspective. A causal chain is described as a mapping of cause-and-effect relationships among measures. A traditional four-part balanced scorecard specifically groups measures by category: financial, customer satisfaction, internal processes, and learning and growth. The paper notes, More recently, balanced-scorecard proponents have focused on the need to tie measures together into a causal chain of performance.

Tayler adds that even the best-designed scorecards can be ineffective, unless the performance measurement system is linked explicitly to strategy. These linkages help to ensure that the firm is measuring the performance that matters the most for strategic success, he says. These systems also allow managers to analyze the appropriateness of strategy, and then to adjust it as necessary.

In the research experiment, MBA students with an understanding of balanced-scorecard concepts assumed the manager role, and they were asked to review a given strategic initiative for a firm-wide rollout. Specifically, the students assumed the role of managers over a fictitious pizzeria chain. In preparation for the task, the study participants were given background information on the chain, as well as two different initiatives for consideration.

The first strategy consisted of offering a free side order with every five pizzas sold. The second initiative employed high-quality ingredients (relative to the existing quality of the pizza ingredients). The students were then asked to consider two different performance measurements. The first measurement used a customer survey score, which rated a customer's intentions to return to order pizza. The second measurement was termed the returning customer score, which involved the actual number of return customers (based on credit card data).

Participants were then given a variety of scenarios, with the research testing low to high levels of participant responsibility over the initiative selection of a free side order vs. the high quality ingredient approach. These results were then tested in a balanced scorecard and a causal chain measurement setup. The balanced scorecard used the classic four-part approach with financial results (gross margin), customer satisfaction (customer survey score or returning customer score), internal processes (on-time delivery), and learning and growth (employee retention).

The causal chain approach used these same measurements as the balanced scorecard, but the approach involved linkages between these various measurements. For example, strong learning and growth led to improved internal processes. In turn, improved internal processes resulted in increased customer satisfaction, which then led to improved financial performance.

Not surprisingly, the data confirmed that managers given initiative responsibility are more likely to push the initiative and rate it more favorably. However, the research suggests that drafting the balanced scorecard in a causal chain format is not sufficient enough to deter motivated reasoning. The researchers conclude that using a traditional scorecard measurement set up as a causal chain, as well as allowing managers to select the measurements, can mitigate the impact of motivated reasoning.

Given the ubiquitous nature of scorecard measurements, Tayler notes that overcoming psychological forces limiting the usefulness of the data is an essential part of getting reliable results. He concedes that research in behavioral psychology shows that motivated reasoning is a common problem in a variety of decision-making settings. Professionals ranging from auditors to physicians have been shown to succumb to this bias. Very likely, management at corporations is not immune to it. For example, the major ratings agencies have been criticized for conflicts of interest when it came to the ratings assigned to their banking clients' structured finance securities.

Though there is doubtless a conscious tendency on the part of some professionals to make judgments consistent with their preferences (and incentives), the more pernicious problem may be the subconscious tendency to analyze data and come to preference-consistent conclusions. The new financial reform legislation (known as the Dodd-Frank Wall Street Reform and Consumer Protection Act), signed into law by U.S. President Obama on July 21, purports to make the rating agencies liable for their opinions. How this will be enforced, and whether it will address individuals' hard-wired, psychological biases, remains to be seen.

Still, says Tayler, removing the reason for the motivation is certainly a good way to help remedy the problem. It also helps for corporations to explicitly set up performance measurements, establish how these measurements are expected to influence each other, and then give managers responsibility for measure selection.

The research also reaffirms the importance of causal chain scorecards versus the traditional four-part balanced scorecard measurement system in this process. A performance measurement system, with measures linked explicitly to strategy, helps to ensure that the firm is measuring the performance that matters most for strategic success. This type of system also allows managers to analyze the appropriateness of strategy and adjust it as necessary. Essentially, causal chain scorecards provide a more detailed and nuanced result, allowing corporate leaders to more effectively evaluate company strategy.