When Gartner’s Magic Quadrant Doesn’t Tell the Whole Story

Posted by LPA Software Solutions on Aug 23, 2021 5:05:45 PM

Gartner’s Peer Insights Tell a More Balanced Story about IBM’s Cognos Analytics 

Business intelligence (BI) analysts understand you must consider your sources when analyzing data. Gartner's Magic Quadrant is one source technologists rely on to make purchasing decisions, including business intelligence tools. There are several problems with relying too heavily on this one data point as we have seen with Gartner’s 2021 Magic Quadrant for Analytics and Business Intelligence Platforms, published in February 2021.

Gartner Peer Insights - Cognos

There, Gartner ranks IBM’s Cognos Analytics as Niche, defining the category as “Niche Players focus successfully on a small segment, or are unfocused and do not out-innovate or outperform others.” Our extensive experience with Cognos in real-world daily scenarios doesn’t align with Gartner’s ranking.

Other published Gartner resources agree with our assessment. In fact, Gartner named IBM as Customers’ Choice in the August 2021 Gartner Peer Insights ‘Voice of the Customer’: Analytics and Business Intelligence Platforms.  Read the full report

IBM is an LPA partner and one we stand behind for its many effective AI solutions, including Cognos. We feel comfortable disclosing our extensive experience with IBM and Cognos Analytics at this juncture because it’s the reason we spotted a problem with the Magic Quadrant ranking of Cognos.

Magic Quadrant Methodology vs. Peer Review

Gartner’s own Peer Insight User Reviews do agree with our experience with Cognos Analytics. Unlike the Magic Quadrant, Gartner’s Peer Insights includes reviews by verified users rather than analyst opinion. There, Cognos ranks higher than Tableau Server, Domo, Power BI, Qlik, Oracle, Looker, Amazon, SAP, Yellowfin, Zoho, and Infor. Compared with vendors in the Leaders’ quadrant, Cognos has a higher willingness to recommend score than the majority of the competition and an overall peer rating of 4.4/5.

The disconnect seems to be in Gartner’s Magic Quadrant research methodology. Contrary to popular belief, Gartner doesn’t neutrally identify providers and compare the platforms’ performance, features and usability. Gartner typically conducts surveys and interviews for its rankings, but it relies on the number of inquiries to its own consultancy about tools it ranks. That methodology is more a measurement of brand recognition than product value.

Note IBM’s response to Gartner’s reliance on inquiries:

Since IBM sellers and business partners place significant focus soon our customer base, Cognos generates fewer inquires and product evaluations than our competitors. Contrary to Gartner’s perception, we continue to build our user base, as demonstrated by a 187% increase in community engagement and on-boarding of over 650 new clients in 2020.

startup business, software developer working on desktop  computer at modern officeApplying data science methodology to Gartner’s 2021 Analytics and BI Magic Quadrant leads us to conclude that its rankings are more magic than science when it comes to Cognos.

For more specifics, read IBM’s rebuttal of the 2021 BI Magic Quadrant Ranking.

How to Select a Business Intelligence Platform

  • Use the Magic Quadrant in its entirety, not just the scatterplot rankings
  • Compare the strengths and cautions for each platform and consider which, if any, are most meaningful to you. For instance, Gartner’s concern about IBM’s fading brand value is more a subjective statement. It may not be of as much concern to users as Microsoft’s lack of platform flexibility or Tableau’s integration challenges. Gartner also cautions about Tableau and Qlik’s cost and price complexity. Cognos Analytics Standard Edition includes authoring vs. read-only for $15/mo/user
  • Read the Peer Reviews, and again carefully consider which points bear the most weight for your use case
  • Review the analysis of other consultants, like BARC and Ventana. These analyses both rank IBM Cognos Analytics among the leaders
  • Consult with trusted partners, including LPA, who work with a variety of BI solutions for real-world applications
  • Compare features, usability, and price
  • Then, make your shortlist based on your actual needs, budget, and goals

Our Conclusions on Relying Too Heavily on the Magic Quadrant

Too many enterprises take the Magic Quadrant graphic as gospel and mistakenly shortcut their potential selections based on this one data point. Instead, we recommend a more scientific approach. Yes, include the Magic Quadrant as one valuable data point, but also include other data points—especially peer reviews.

Read the full report rather than simply relying on the scatterplot rankings. A closer look at the Magic Quadrant full report has some telling information:

  • IBM’s primary competitors all lost position while Cognos did not.
  • Out of the box, Cognos supports all 12 use cases Gartner considers differentiators.

Also, follow Gartner’s own advice. Gartner’s published research methodology states:

Use a Gartner Magic Quadrant as a first step to understanding the technology providers you might consider for a specific investment opportunity.

Keep in mind that focusing on the leaders ’quadrant isn’t always the best course of action. There are good reasons to consider market challengers. And a niche player may support your needs better than a market leader. It all depends on how the provider aligns with your business goals.

Selecting the right business intelligence platform for your business requires understanding current and future needs, strategy, research, planning, and adoption. Contact LPA for help in navigating your analytics journey.

New call-to-action

Topics: Cognos Analytics, Gartner