Unvalidated: The Truth Behind 99% of Board Surveys

By

2 men discussing tips for a successful board chair

Over 80% of drivers think they’re above average, and boards are no different. Many members view themselves as more effective than their peers.

Some boards conduct formal surveys among directors and sometimes even include senior executives. This seems like a smart move, right? You’re gathering data and involving more voices, and it’s all in the name of self-improvement. But when these surveys are handled internally, you might not get the whole, unvarnished truth. People tend to be less candid when they know their colleagues might scrutinise their responses.

Typically, the company secretary administers the board survey. Occasionally, an external party is brought in to handle this task, which can help get more honest feedback. But the real question we must ask is: Can we trust that these board surveys and the resulting reports reflect the board’s performance and effectiveness?

The self-perception trap

There’s an issue of self-perception. Just like the drivers who believe they are better than average, board members often think they’re doing better than they are. This cognitive bias can skew the results of any survey. If everyone on the board thinks they’re effective, the survey results might reflect this overconfidence rather than reality. This is especially true in internal surveys where the fear of rocking the boat might lead to overly positive responses.

The internal vs. external debate

When a company secretary administers the survey, there’s a risk of bias creeping in. Directors might not want to be too critical if they know their feedback will be seen closely by someone they work with. On the other hand, bringing in an external party to conduct the survey can mitigate this risk. External facilitators can ensure anonymity and impartiality, providing more honest and useful feedback. But even then, the effectiveness of the survey hinges on how it’s designed and conducted.

Candidness and anonymity

For a board survey to be truly effective, candidness is key. Respondents must feel safe to share honest opinions without fear of retribution or awkwardness. Anonymity can help, but it’s not a cure-all. The way questions are phrased, the context in which the survey is conducted, and the follow-up actions all play significant roles in determining whether the survey results are valid.

Understanding the survey validation process

Validation is key to ensuring that your board survey is telling you something useful. Larger professional service groups have validated their employee, customer, and other surveys for years, often partnering with universities or qualified empirical researchers to get the job done right.

So, what does the validation process involve? It’s not just a quick check to see if the questions make sense. Instead, it’s a deep dive involving psychometric and statistical analysis of a large dataset of survey responses. This includes both exploratory and confirmatory factor analysis. Essentially, these analyses check if the survey questions (or factors) reliably measure what they’re supposed to.

Think of it like this: if you’re trying to measure different aspects of board effectiveness, you want to ensure each question accurately reflects the specific factor it’s targeting. The exploratory analysis helps identify whether these factors hold up under scrutiny, while the confirmatory analysis verifies that the identified factors are consistent and reliable measures.

But it doesn’t stop there. The validation process also involves reviewing and combining the existing literature with significant practical experience. This step ensures that the survey aligns with contemporary practices and current research. It’s about making sure the survey isn’t just theoretically sound and practically relevant.

The final step in the validation process is rigorous reporting, often culminating in publication in a peer-reviewed journal. This isn’t just a formality — it’s a way to ensure the process and results are scrutinised and validated by the broader academic and professional community. This level of rigour helps guarantee that the survey is a reliable measure of board effectiveness.

Why go through all this trouble? Well, a validated survey means you can trust the results. It means that when the survey says your board is effective (or not), it’s based on solid, reliable data. Without validation, you’re flying blind, basing decisions on potentially flawed measures.

The validation gap: Why 99% of board surveys haven’t been validated

Every year, hundreds, if not thousands, of different board surveys are conducted worldwide. Yet, despite their widespread use, a staggering 99% of these surveys haven’t been validated. Most in-house surveys don’t even claim to be valid or reliable measures of a board’s effectiveness. They’re often designed for quick feedback rather than rigorous assessment.

Even when consultants step in to help with board surveys, many fall short in the validation department. Often, they haven’t gathered enough data or conducted the necessary analysis to ensure the survey’s reliability. Without sufficient data, these surveys can’t undergo the thorough validation process needed to confirm their accuracy.

Our board survey stands out because it has undergone comprehensive validation. We partnered with Deakin University to ensure that every step of the validation process was meticulously followed. This includes psychometric and statistical analyses, literature reviews, and rigorous testing. The results of this process were so robust that they were reported in a peer-reviewed journal, adding an extra layer of credibility.

So, why does this matter for your board? Simply put, a survey can only be considered a reliable measure of board performance and effectiveness if it has been properly validated. Without this, the results are just educated guesses at best. We are unaware of any other board survey that has undergone such a stringent validation process. This makes our validated survey a tool and a benchmark for truly understanding and improving board effectiveness.

Rigorous validation pinpoints key factors in board performance

When understanding what makes a board truly effective, rigorous validation is vital. It’s about asking the right questions and then analysing the answers to uncover the key factors driving board performance.

Our journey began with a collaboration involving a global governance expert, KPMG, and numerous experienced directors and governance practitioners. With our two decades of survey expertise, we designed a comprehensive board survey that initially included ten factors of a board’s effectiveness. This wasn’t just a shot in the dark; it was a carefully crafted tool aimed at covering all bases.

Through a rigorous validation process, we went beyond surface-level analysis. We systematically removed unsuitable or irrelevant survey items, ensuring only the most pertinent questions remained. This thorough process allowed us to pinpoint the distinct components crucial to a board’s effectiveness.

Over 15 years of validation work, supported by Deakin University, helped us identify what is now widely recognised as the 20 most important factors of a board’s effectiveness. This wasn’t a quick or easy task, but it was essential. We covered every critical element of board performance, leaving no stone unturned.

Why it matters

The result is a survey that doesn’t just skim the surface but dives deep into what makes a board effective. All 20 factors identified are essential, addressing every critical element without leaving gaps. This comprehensive approach means that when you use our validated survey, you get a complete measure of your board’s performance and effectiveness.

In the end, rigorous validation separates a good survey from a great one. It ensures that the insights you gain are accurate, reliable, and actionable. A validated survey becomes an invaluable tool for driving genuine improvement and achieving governance excellence by identifying the main factors contributing to a board’s effectiveness.

Why benchmarking is the most important factor

Having a reliable measure of board effectiveness is crucial, but the real value shines through when you benchmark your board survey responses against those of comparable boards. This comparative approach amplifies the insights you gain, helping you understand where you truly stand.

Benchmarking your board against similar-sized organisations and within the same industry is essential. Imagine comparing the board of a financial services organisation to those of not-for-profits or government entities—it would be completely misleading. Each sector has its unique challenges and standards, so it’s vital to benchmark within the right context.

At Board Surveys, we offer comparisons against over 500 boards. This extensive database ensures that your benchmarking efforts are both relevant and accurate. By comparing your board’s performance with those of similar organisations, you get a clear picture of where you excel and where there’s room for improvement.

A validated board survey is a powerful tool on its own, but it becomes invaluable when paired with comprehensive benchmarking. Our fit-for-purpose, affordable, and world-class survey options are designed to provide you with the most accurate and actionable insights.

Get the true measure of effectiveness for your board today

A board survey is just one tool in the toolbox for assessing effectiveness. It can provide valuable insights but should not be the sole measure. Combining survey results with other methods, such as external evaluations, performance metrics, and stakeholder feedback, can paint a more comprehensive picture.

In the end, the validity of a board survey as a measure of effectiveness depends on several factors: who conducts it, how it’s conducted, and how the results are used. If done correctly, it can be a useful tool. However, boards should be wary of over-relying on self-assessments. They should seek to complement them with other evaluative methods to ensure they get a true sense of their performance. So, while a board survey is a step in the right direction and extremely important, it’s just one piece of the quest for board effectiveness.

Share
Share

Benchmarking Board Performance

Gain actionable insights that are benchmarked with comparable boards within your chosen industry

Want to try a Board Effectiveness Survey?

Try a free 3 minute board effectiveness sample survey and get a benchmarked performance report

Stay up to date with our Insights & Ideas

"*" indicates required fields

A female asking team if their board survey benchmarked
Try our FREE Board Effectiveness survey
and receive a
benchmarked report