Why Performance Monitoring?

In understanding Service Levels for Web Sites, moving quickly to how to measure for SLAs, we quickly turn to Performance Monitoring to see how a web site is doing. In Avinash's book, "Web Analytics An Hour A Day", he describes the Web Analytics trinity, a framework for understanding "Why customers visit", "Who actually visits", and "What Happens when they visit". I propose to expand the trinity to "How is the Web Site Doing?" and that is why we use performance monitoring.

The critical issue not found with Web Analytics is the critical metric of Time. "How are we doing?" easily goes to "When did it Happen?" Most web analysis does not account for time, since the insights are more strategic in nature. Performance Monitoring is more operational in nature to provide valuable insight into how the site is reacting "Right Now".

The Crux is the Business Process

From my background in rock climbing, the hardest part of the climb is called the 'crux'. It's the pivotal point in the whole application performance business. Why you ask? Because business people think this way, and so do the web site customers. People don't think servers and disk drives. When discussing Business Processes, IT can get the business engaged. Now you provide a Business Process, SLM-driven metric dashboard, and you've got everyone up the mountain, past the hard parts and on the path to optimal metrics and business nirvana!

The following graphic shows this exactly:

Note: This Graphic is from the following article: http://www.enterpriseleadership.org/c/journal/view_article_content?groupId=32294&articleId=36254&version=1.0.

Now let's get a bit more concrete in this. Pick your favorite web site, any site. We'll pick a few. Let's take a look at Progressive.com. (Disclosure: they were my customer at Gomez and I am a auto insurance customer). When you load their home page, take a step back from your screen and what do you see.... first I notice the words "Get a Quote". Hmm. What is this? Right. It's a quoting process, enabled to get folks quick auto quotes. Hence it's a new customer acquisition tool. Do you think this is an important business process for Progressive? You bet. Inside the company, they probably know their conversion rate for this process. Looking further, you see 'Login', which is the self-service portal. Wouldn't it make sense to monitor this login process to make sure the customer can always log in? Absolutely.

It becomes much easier to dialog about the 'login' process with customers and employees, especially on the business side. The old school IT view of talking about Weblogic Server 6788989 is slow doesn't really help anyone.

More About Business Processes

Think of a business process as a multiple page work flow or use case that end users navigate on a web site. For example, an end user who logs into their bank to look up their latest account transactions usually navigates three pages to get their answer.

In addition to monitoring breadth, business process work flows provide the richest and most valuable performance data for analytical purposes. These work flows usually require data from most of an organization's technical infrastructure, including global load balancers, web servers, application servers, message queues, and numerous database servers. Monitoring all the key business processes not only provides critical end user performance metrics, but also helps in quickly identifying performance issues in the infrastructure.

Sample Business Processes

The following processes are a sample of what you find out on the web today:

• Account Status. For many organizations, such as those in financial services, retail, or manufacturing, critical customer information such as account, order, or shipping status. This product usually requires three steps: one to login to the site, one to display the summary, and one to logout.

• Search for Product . Most sites work hard on search engines. Improving their search engine are critical to many in the retail and travel industry. Search for Product usually has two or three steps. First the keyword search box, the search results page, and then the item description.

• Account Registration. Whether registering for a site profile, online gaming, an email account, or product sweepstakes, product managers want to get people registered to deliver personalization and cross selling products later. Financial institutions are registering users for online banking and trading stocks. These products have two or three pages, one where the user enters form information and the final confirmation page.

• Service Application. A difficult product is the service application, whether applying for a credit card, filling out a tax return, health insurance claim, or consumer item rebate. These products are front ends to complex business processes. As more organizations move to compress the process time line, managing the end user and the work flow through third parties is critical.

• Shopping Cart. For retail and financial institutions, this is the key to booking revenue. Getting the customer to choose a product they desire and then get the order placed is a very important focus. These products focus on three main steps, Add to Cart, Llogin page, then navigation to final order step.

• Generate Report. Many service providers provide value by producing charts or data reports. These product managers aggressively work to improve the experience of the service. These products have three steps, beginning with a site login, navigating to a reports page, selecting report parameters, and generate the report.

As seen the attached figure, an Account Status business process can be broken down into steps that impact each part of the technical infrastructure. Building such sequence diagrams is very useful to discuss each process across internal and external organizations.

Why SLM Matters

Service Level Management (SLM) the ability to help organizations monitor application performance against a defined set of objectives agreed to by either internal or external service providers. By proactively managing SLM performance, organizations can:

• Provide a collaborative and productive environment to discuss issues
• Leverage independent metrics are easy to distribute via reports
• Focus on end user and business process perspectives rather than internal event systems

There are dozens of organizations, consultants, and web sites which discuss Service Level Management (SLM), and as usual, the terminology and your mileage will certainly vary. In the context of Optimal Metrics, SLM matters because it really helps to focus the organization and it's partners on the customers. In my Internet searching, most of the content on this topic of Service Level Management discusses the ITIL framework. It is a great framework and management tool, but it really is not a methodology. While I do like strategic frameworks, the engineer in me loves to jump into the process, technology, and people to get things done.

The closest reference I have found is 'Implementing Service Level Management' and the article has two wonderful pictures that sum up most of my approach to understanding SLM:


The summary of this great article is that the way to implement SLM is to leverage Performance Management disciplines to manage and monitor business processes.

Another way to look at SLM is from the top down. Many in IT do things the other way, from the bottom up. As the diagram shows, focusing on technology components, such as routers, switches, and servers makes for a complicated approach. How do you measure across all the devices? With a router you can measure throughput in Megabytes per second. A server can toss out SNMP trap alerts when the processor goes more than 80% busy. And what does all this do to manage the application? Event and Alert management is great for the IT Operations team to know if the component is broken, but rarely can it directly tell you that the business process is not working right.

So, in summary, the reason behind SLM is to provide metrics and management insight into the technology. This is key to running IT inside a company, or even more when outsourcing occurs. SLM disciplines are required more and more as IT is disintermediated outside of the IT department. Think about the components in your standard B2C page: analytics, ads, content, CDNs, data centers. A lot of content is outside your control, so how do you make sense of it? How do you insure the provider is doing the right thing?