How to Use a Competitive Intelligence Resource-Scoring System for Quality Control
As you evaluate competitive intelligence sources, articles, white papers, and so on, assign them a score of A, B, C, D, or F to indicate how reliable and useful you think the source and the particular piece of information is.
Scoring helps you and others develop a feel for the relative value of different sources and quickly determine how useful a specific piece of intel is likely to be. Assign three scores and a composite:
Score the source of the article or other item.
Score the author.
Score the content of the article or other item.
Assign a composite score that accounts for both the source and the content.
General source score
When you obtain information from a third party, score the general source (the type of data and the context in which it was gathered). A general source can be a trade show presentation that's historically been very revealing or a specific publication.
For example, if you receive a CI report from one of your internal marketing people about a presentation that a competitor delivered at a recent event, you may be able to assign a source score based on how valuable similar information collected under similar circumstances has been in the past.
Give a low score if the information comes from journalists or analysts who got it from organizations that may want to influence it. Analysts and journalists sometimes go easy on certain organizations that feed them stories or grant interviews because writing is easier when someone is feeding you information. Unless the journalist or analyst conducts an independent investigation, the integrity of the reporting is questionable.
Information that's freely presented at trade shows and conventions is often grade C or D material. It's highly suspect, if not intentionally designed to mislead, and you really shouldn't use it as a prima facie (accepted as true until proven otherwise) source. The only real value of such information is that you could use it to try to figure out what the organization's motive was in making the information available.
Grade-A experts and analysts are consistently accurate in their predictions. These are your go-to guys and gals. You know about the accuracy of their reporting. Rate them accordingly.
Superior authors are usually experts in the field, and they may be analysts who've already done the heavy lifting to ensure accuracy, depth, and clarity.
Over time, you may also encounter less reliable experts and analysts — people who seem to be out of touch or out of step with what's going on and have a poor track record of predicting where certain markets are headed.
Additionally, some publications and other media outlets have been known to sell out to organizations that advertise with them. Assign low ratings to these folks so the rest of the people in your organization remain skeptical when accessing their information.
The article score represents the actual value of the information you collected — the content of the publication, speech, presentation, or interview you conducted. When assigning an article score, consult your intuition and then give your opinion of just how good the article or other information really is.
Composite (overall) score
To determine a composite score for the source, don't simply add the three scores and divide by three. The composite score is actually just an abbreviation of your three ratings; for example, BAD:
B General source quality, fairly high
A Quality of the author's historic work, very high
D Quality assessment of the article itself, sub-standard
As you look at the above scores, note that the analyst rated the quality of the article as a "D" (substandard). In this case, the publication and the author are trustworthy, but the author's conclusions simply don't triangulate with other sources. Thus, the analyst wants the reader to know to approach the conclusions in the article with skepticism.
Low scores serve as red flags, alerting your organization that the source and the information are questionable and should be approached with skepticism.