It sounds like a wild claim, considering that billion dollar companies like Splunk, Datadog, New Relic, and Solarwinds are consistently making national headlines, for both good and bad reasons. Observability leaders are anything but invisible, so how can the perfect solution be different? Are they that far off?
To create a vision of the perfect log management solution, we need to understand what makes observability valuable. In some cases it’s employed as a simple compliance requirement. The value added in those circumstances is binary – either a firm meets compliance, or doesn’t. In a recent blog post, New Solutions to New Observability Needs, we talked about compliance as a barrier to entry, and observability solutions that make surmounting that barrier easier and more affordable as empowering resources that create beneficial competition in industries with strict compliance standards. Compliance reports are important, but today compliance is only a portion of the observability market. Most tech companies invest in some form of log management. It can be tedious and expensive, but companies pay millions of dollars and dedicate entire teams to implement and oversee log management solutions. They must be getting something out of it. What’s the value of log management beyond security and compliance needs?
The answer is nuanced for any specific case, but generally it boils down to three high-level sources of value:
Log management is valuable. Often essential. So why isn’t it considered a solved problem? New players emerge in the observability industry every year. Why is there so much competition? The simple answer is that no existing observability solution is perfect. “Perfect” is an unattainable standard, but it is clearly visible in the minds of observability experts, so nothing can stop them from trying. In a nutshell, a perfect observability solution implements itself across entire networks, configures with no human effort, and delivers actionable insights to teams that are catered specifically to the needs of the users while maintaining security and privacy (the “perfect” solution is also free, but that is true of any product). Easy, right?
It’s safe to say that as long as no perfect solution exists, there will always be competition to improve. Safe, but not satisfying. The more complicated answer, which lands closer to truth, is that in the pursuit of perfect observability, different players focus on perfecting different aspects of observability, optimizing for certain value propositions and minimizing related pains. For example, a company like Splunk, the current leader in observability for large businesses, focussed on scale and expansive featuresets, made trade-offs with price and ease of use. They appeal to devops teams maintaining massive ecosystems with deep pockets to pay for it. Other companies target niches that large, expensive solutions cannot service perfectly, or simply undercut them and target large businesses that don’t need all the bells and whistles. What are the major pain points that new observability solutions aim to solve?
Maximize the value, minimize the pain, and a vision emerges of a “perfect” solution. It’s secure, insightful, efficient, and requires no human attention to setup or maintain. It delivers insights and alerts to the right people, while omitting unnecessary clutter. It’s affordable, or free, so teams of any size can comfortably use it. It doesn’t exist. It will probably never exist, but that won’t stop us from trying.
At observIQ, we are pursuing our vision of the perfect observability solution. We maximize our users’ value, while minimizing effort, and keeping costs low. It might never be perfect, but it’s well on its way. It only takes minutes for someone with no technical experience to sign up and ship logs to observIQ. There’s alerts, real time insights, analytics, and unlimited users for collaborative work. Try it out and let us know what you think.