|By Leon Fayer||
|July 27, 2012 12:00 PM EDT||
I've talked at length about the importance of business process monitoring alongside of system monitoring, but in discussions I found that sometimes an overview and simple examples are not enough to convince people about the benefits of this approach. Business owners think they don't need to know anything about the operational performance of their systems as long as they have their numbers, and engineers often don't feel they need to invest time into understanding the business they are supporting in detail, finding examples shown too "common sense."
One question we ask our engineers during an interview at work is to describe the process of how they would go about troubleshooting a hypothetical issue, given only the minimum information. We often hear from our clients things like, "our website is slow" and "something's wrong with registration" with no additional information, and in order to figure out the potential issue we need to review the whole system at a glance. For large applications, with a myriad of moving and interweaving components, this is not an easy task. This is one of the reasons we are looking for best of the best. But if you are monitoring all of those components, in a lot of cases, the task can be simplified.
So let's examine a real problem. A large e-commerce company called and said that they are seeing less money coming in from web transactions. They have a pretty complex system with a lot of different revenue generation points, so this observation shed very little light on the root cause of the problem. Luckily, both systems and business processes were being monitored with Circonus, so the data was available to review.
As any engineer knows - step one of troubleshooting the problem is to confirm the problem, so looking at the revenue trends seemed like a good starting point.
The graph clearly shows that, starting around April 30th, the trend looked abnormal in comparison to the previous few weeks. So it seemed like there was an actual problem, and potentially, the issue could lie in payment processor itself or somewhere in the system, preventing certain users from making a purchase. So let's overlay the traffic trends, collected from Google Analytics, against revenue graph and see if there are any common trends.
Even though the traffic showed a clear drop at the same time as revenue, the ratio remained the same, allowing us to exclude payment processor and other application logic from the equation (for now).
Note: This is the first potential breaking point in the process. It is very tempting to look at the ratios, attribute revenue decrease to traffic decrease, and stop the investigation. 99% of the time, unfortunately, nothing "just" happens, so on we go.
Now for the next step - what would be a logical cause for a drop in overall traffic to the site? Response time is probably the first thought that should come to mind. So let's look at what the HTTP checks collected.
Load times didn't seem to be deviating from the norm, but the HTTP response metric doesn't provide full visibility into the load times for a dynamic application, so let's check the health of the database and CPU usage on the server(s), to validate that the underlying platform is not the bottleneck. There are numerous metrics to monitor database and system health that should be, and in this case, are collected, but when researching the root cause of the elusive problem, diving deep into a specific component can waste time early in the process.
Both of the metrics appear well within norm, so at first glance, it seems like the problem is not a systems issue.
Note: This is the second point of the investigation where the process can break down. A lot of technologists will either report that there is no confirmation to the problem reported; the reported problem is just an anomaly because the system monitors don't exhibit any issues. This is exactly why understanding of the business by the technology team is vital.
With that said, what would be the next logical process to validate? It is not uncommon for an e-commerce site to see a drop in purchases if they either stop promoting or if their marketing campaign is ineffective: traffic to the site slows down, subsequently decreasing the number of transactions. This company, in particular, sends out tens of millions of emails a day which bring in new users, and subsequently, new conversions. So let's take a look at the email deliverability and bounce rates collected from the company's MTAs.
Bingo! The bounce rates sky rocketed at the same time as the drop in traffic and revenue stream occurred. Upon closer investigation, it appeared that one of the major ESPs accidentally blocked the delivery domain, and the emails did not go through to the recipients. The issue was resolved (after some discussions with the ESP) and the trends returned back to the expected level.
Keep in mind, if email deliverability was not the issue - there are multiple other metrics that were on a list to be verified, both system (operational and development alike) and business. The amazing part of all of this is that I was able to view the whole system at a glance in just one graph. Granted, stacking everything on one graph is probably not the most optimal every day approach, but it is very useful in a certain cases when the direct overlay correlation is needed. For everything else - a real-time dashboard that displays all the vital points of the business at any given moment is a must-have for anyone responsible for business and/or system health.
Everyone responsible for the success of a business, regardless of the role, needs an ability to see the status of the whole business at a glance at any given point. System engineers don't need to know all the ins and out of marketing, but they should be aware of the overall organizational goals, and should be able to spot irregularities in the business trends. Similarly, CEOs don't need to know how systems work in the background, but should be able to correlate high email bounce rates (if it's critical to the business) to a decrease in purchases.
The point of all of this is that everything should be monitored, and to suggest some tools and methods that can enable users in all roles--within any organization--to ensure the success of the business. Get ‘em, learn 'em, use 'em! You will thank me later.
"We've discovered that after shows 80% if leads that people get, 80% of the conversations end up on the show floor, meaning people forget about it, people forget who they talk to, people forget that there are actual business opportunities to be had here so we try to help out and keep the conversations going," explained Jeff Mesnik, Founder and President of ContentMX, in this SYS-CON.tv interview at 18th Cloud Expo, held June 7-9, 2016, at the Javits Center in New York City, NY.
Jul. 27, 2016 10:30 PM EDT Reads: 1,403
Internet of @ThingsExpo, taking place November 1-3, 2016, at the Santa Clara Convention Center in Santa Clara, CA, is co-located with the 19th International Cloud Expo and will feature technical sessions from a rock star conference faculty and the leading industry players in the world and ThingsExpo Silicon Valley Call for Papers is now open.
Jul. 27, 2016 10:00 PM EDT Reads: 2,658
The IoT is changing the way enterprises conduct business. In his session at @ThingsExpo, Eric Hoffman, Vice President at EastBanc Technologies, discussed how businesses can gain an edge over competitors by empowering consumers to take control through IoT. He cited examples such as a Washington, D.C.-based sports club that leveraged IoT and the cloud to develop a comprehensive booking system. He also highlighted how IoT can revitalize and restore outdated business models, making them profitable ...
Jul. 27, 2016 09:45 PM EDT Reads: 1,997
"delaPlex is a software development company. We do team-based outsourcing development," explained Mark Rivers, COO and Co-founder of delaPlex Software, in this SYS-CON.tv interview at 18th Cloud Expo, held June 7-9, 2016, at the Javits Center in New York City, NY.
Jul. 27, 2016 08:00 PM EDT Reads: 2,030
We all know the latest numbers: Gartner, Inc. forecasts that 6.4 billion connected things will be in use worldwide in 2016, up 30 percent from last year, and will reach 20.8 billion by 2020. We're rapidly approaching a data production of 40 zettabytes a day – more than we can every physically store, and exabytes and yottabytes are just around the corner. For many that’s a good sign, as data has been proven to equal money – IF it’s ingested, integrated, and analyzed fast enough. Without real-ti...
Jul. 27, 2016 07:30 PM EDT Reads: 1,101
I wanted to gather all of my Internet of Things (IOT) blogs into a single blog (that I could later use with my University of San Francisco (USF) Big Data “MBA” course). However as I started to pull these blogs together, I realized that my IOT discussion lacked a vision; it lacked an end point towards which an organization could drive their IOT envisioning, proof of value, app dev, data engineering and data science efforts. And I think that the IOT end point is really quite simple…
Jul. 27, 2016 06:45 PM EDT Reads: 1,168
"There's a growing demand from users for things to be faster. When you think about all the transactions or interactions users will have with your product and everything that is between those transactions and interactions - what drives us at Catchpoint Systems is the idea to measure that and to analyze it," explained Leo Vasiliou, Director of Web Performance Engineering at Catchpoint Systems, in this SYS-CON.tv interview at 18th Cloud Expo, held June 7-9, 2016, at the Javits Center in New York Ci...
Jul. 27, 2016 06:45 PM EDT Reads: 2,064
A critical component of any IoT project is what to do with all the data being generated. This data needs to be captured, processed, structured, and stored in a way to facilitate different kinds of queries. Traditional data warehouse and analytical systems are mature technologies that can be used to handle certain kinds of queries, but they are not always well suited to many problems, particularly when there is a need for real-time insights.
Jul. 27, 2016 04:30 PM EDT Reads: 1,870
Big Data, cloud, analytics, contextual information, wearable tech, sensors, mobility, and WebRTC: together, these advances have created a perfect storm of technologies that are disrupting and transforming classic communications models and ecosystems. In his session at @ThingsExpo, Erik Perotti, Senior Manager of New Ventures on Plantronics’ Innovation team, provided an overview of this technological shift, including associated business and consumer communications impacts, and opportunities it ...
Jul. 27, 2016 04:30 PM EDT Reads: 205
You think you know what’s in your data. But do you? Most organizations are now aware of the business intelligence represented by their data. Data science stands to take this to a level you never thought of – literally. The techniques of data science, when used with the capabilities of Big Data technologies, can make connections you had not yet imagined, helping you discover new insights and ask new questions of your data. In his session at @ThingsExpo, Sarbjit Sarkaria, data science team lead ...
Jul. 27, 2016 04:15 PM EDT Reads: 1,139
Is your aging software platform suffering from technical debt while the market changes and demands new solutions at a faster clip? It’s a bold move, but you might consider walking away from your core platform and starting fresh. ReadyTalk did exactly that. In his General Session at 19th Cloud Expo, Michael Chambliss, Head of Engineering at ReadyTalk, will discuss why and how ReadyTalk diverted from healthy revenue and over a decade of audio conferencing product development to start an innovati...
Jul. 27, 2016 04:00 PM EDT Reads: 1,063
Extracting business value from Internet of Things (IoT) data doesn’t happen overnight. There are several requirements that must be satisfied, including IoT device enablement, data analysis, real-time detection of complex events and automated orchestration of actions. Unfortunately, too many companies fall short in achieving their business goals by implementing incomplete solutions or not focusing on tangible use cases. In his general session at @ThingsExpo, Dave McCarthy, Director of Products...
Jul. 27, 2016 04:00 PM EDT Reads: 1,741
WebRTC is bringing significant change to the communications landscape that will bridge the worlds of web and telephony, making the Internet the new standard for communications. Cloud9 took the road less traveled and used WebRTC to create a downloadable enterprise-grade communications platform that is changing the communication dynamic in the financial sector. In his session at @ThingsExpo, Leo Papadopoulos, CTO of Cloud9, discussed the importance of WebRTC and how it enables companies to focus...
Jul. 27, 2016 03:30 PM EDT Reads: 977
SYS-CON Events announced today that 910Telecom will exhibit at the 19th International Cloud Expo, which will take place on November 1–3, 2016, at the Santa Clara Convention Center in Santa Clara, CA. Housed in the classic Denver Gas & Electric Building, 910 15th St., 910Telecom is a carrier-neutral telecom hotel located in the heart of Denver. Adjacent to CenturyLink, AT&T, and Denver Main, 910Telecom offers connectivity to all major carriers, Internet service providers, Internet backbones and ...
Jul. 27, 2016 11:00 AM EDT Reads: 734
SYS-CON Events announced today that LeaseWeb USA, a cloud Infrastructure-as-a-Service (IaaS) provider, will exhibit at the 19th International Cloud Expo, which will take place on November 1–3, 2016, at the Santa Clara Convention Center in Santa Clara, CA. LeaseWeb is one of the world's largest hosting brands. The company helps customers define, develop and deploy IT infrastructure tailored to their exact business needs, by combining various kinds cloud solutions.
Jul. 27, 2016 10:30 AM EDT Reads: 1,234
Manufacturers are embracing the Industrial Internet the same way consumers are leveraging Fitbits – to improve overall health and wellness. Both can provide consistent measurement, visibility, and suggest performance improvements customized to help reach goals. Fitbit users can view real-time data and make adjustments to increase their activity. In his session at @ThingsExpo, Mark Bernardo Professional Services Leader, Americas, at GE Digital, discussed how leveraging the Industrial Internet a...
Jul. 27, 2016 09:45 AM EDT Reads: 490
The cloud market growth today is largely in public clouds. While there is a lot of spend in IT departments in virtualization, these aren’t yet translating into a true “cloud” experience within the enterprise. What is stopping the growth of the “private cloud” market? In his general session at 18th Cloud Expo, Nara Rajagopalan, CEO of Accelerite, explored the challenges in deploying, managing, and getting adoption for a private cloud within an enterprise. What are the key differences between wh...
Jul. 27, 2016 09:30 AM EDT Reads: 2,087
SYS-CON Events announced today that Venafi, the Immune System for the Internet™ and the leading provider of Next Generation Trust Protection, will exhibit at @DevOpsSummit at 19th International Cloud Expo, which will take place on November 1–3, 2016, at the Santa Clara Convention Center in Santa Clara, CA. Venafi is the Immune System for the Internet™ that protects the foundation of all cybersecurity – cryptographic keys and digital certificates – so they can’t be misused by bad guys in attacks...
Jul. 27, 2016 09:15 AM EDT Reads: 1,383
The best-practices for building IoT applications with Go Code that attendees can use to build their own IoT applications. In his session at @ThingsExpo, Indraneel Mitra, Senior Solutions Architect & Technology Evangelist at Cognizant, provided valuable information and resources for both novice and experienced developers on how to get started with IoT and Golang in a day. He also provided information on how to use Intel Arduino Kit, Go Robotics API and AWS IoT stack to build an application tha...
Jul. 27, 2016 09:15 AM EDT Reads: 1,185
Amazon has gradually rolled out parts of its IoT offerings in the last year, but these are just the tip of the iceberg. In addition to optimizing their back-end AWS offerings, Amazon is laying the ground work to be a major force in IoT – especially in the connected home and office. Amazon is extending its reach by building on its dominant Cloud IoT platform, its Dash Button strategy, recently announced Replenishment Services, the Echo/Alexa voice recognition control platform, the 6-7 strategic...
Jul. 27, 2016 09:00 AM EDT Reads: 486