Chapter 4: Monitoring: Look Outside the Data Center

Creating Unified IT Monitoring and Management in Your Environment Don Jones sponsored by Creating Unified IT Monitoring and Management in Your Environment          Don Jones    Chapter 4: Monitoring: Look Outside the Data Center .......................................................................... 45  Monitoring Technical Counters vs. the End‐User Experience ...................................................... 45  . How the EUE Drives Better SLAs ............................................................................................................... 46  How It’s Done: Synthetic Transactions, Transaction Tracking, and More ............................... 49  Top‐Down Monitoring: From the EUE to the Root Problem ......................................................... 50  . Agent vs. Agentless Monitoring .................................................................................................................. 51  Monitoring What Isn’t Yours ....................................................................................................................... 54  Critical Capability: You Need to Monitor Everything ........................................................................ 57  Conclusion ........................................................................................................................................................... 59  Coming Up Next… ............................................................................................................................................. 59      i Creating Unified IT Monitoring and Management in Your Environment          Don Jones    Copyright Statement © 2011 Realtime Publishers. All rights reserved. This site contains materials that have been created, developed, or commissioned by, and published with the permission of, Realtime Publishers (the “Materials”) and this site and any such Materials are protected by international copyright and trademark laws. THE MATERIALS ARE PROVIDED “AS IS” WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, TITLE AND NON-INFRINGEMENT. The Materials are subject to change without notice and do not represent a commitment on the part of Realtime Publishers its web site sponsors. In no event shall Realtime Publishers or its web site sponsors be held liable for technical or editorial errors or omissions contained in the Materials, including without limitation, for any direct, indirect, incidental, special, exemplary or consequential damages whatsoever resulting from the use of any information contained in the Materials. The Materials (including but not limited to the text, images, audio, and/or video) may not be copied, reproduced, republished, uploaded, posted, transmitted, or distributed in any way, in whole or in part, except that one copy may be downloaded for your personal, non- commercial use on a single computer. In connection with such use, you may not modify or obscure any copyright or other proprietary notice. The Materials may contain trademarks, services marks and logos that are the property of third parties. You are not permitted to use these trademarks, services marks or logos without prior written consent of such third parties. Realtime Publishers and the Realtime Publishers logo are registered in the US Patent & Trademark Office. All other product or service names are the property of their respective owners. If you have any questions about these terms, or if you would like information about licensing materials from Realtime Publishers, please contact us via e-mail at info@realtimepublishers.com.   ii Creating Unified IT Monitoring and Management in Your Environment          Don Jones    Chapter 4: Monitoring: Look Outside the  Data Center  IT has moved beyond our own data centers, and nearly every organization has at least one  or two outsourced IT services. Although we’re probably always going to have on‐premise  assets to manage and monitor, we need to realize that in most cases, monitoring has to  start outside the data center—both in the sense of accommodating off‐premises services as  well as focusing more closely on what end users are actually experiencing.  Monitoring Technical Counters vs. the End‐User Experience  The traditional IT monitoring approach is what I call inside out: It starts within the data  center and moves outward toward the end user. Figure 4.1 provides a visual for this idea,  illustrating how typical monitoring focuses on the backend: database servers, application  servers, Web servers, cloud services, and so forth. The general reason for this approach is  that we have the best control and insight over what’s inside the data center. If everything  inside the data center is running smoothly, it stands to reason that the end users who  consume the data center’s services will be happy.    Figure 4.1: Monitoring from the data center outward.    45 Creating Unified IT Monitoring and Management in Your Environment          Don Jones    Most Service Level Agreements (SLAs) derive from this approach: We promise a certain  amount of uptime, and we set up monitoring thresholds around data center‐centric  measurements like CPU utilization, network utilization, disk utilization, and so forth. We  also tend to look at low‐level response times: query response time, disk response time,  network latency, and so on.  There’s something deeply and inherently inaccurate about the underlying assumption of  this approach: Even if you start with a perfect pile of bricks, there’s no guarantee that  you’re going to end up with a stable building in the end. In other words, what end users  experience isn’t merely the sum of the data center’s various metrics. A smoothly‐running  data center usually leads to satisfied users, but that isn’t always the case.  It’s obviously important for us to continue monitoring these data center‐centric  measurements, but those can’t be the only thing we monitor and measure. Current thinking  in the industry is that we need to more directly measure what the end user experiences. In  fact, “end user experience,” or EUE, has become a common term in more forward‐thinking  management circles.  Here’s another way to think of it: Suppose you go to a restaurant to eat. Your steak comes  out cooked wrong, they brought the wrong side items, and the waitress is rude. The  manager, standing back in the kitchen, thinks everything is fine: the steaks are hot, the  veggies are hot, and the waitress smiles at him every time she goes back there. He’s focused  on the backend, with no knowledge of your expectations. Restaurants address this by  having the manager periodically roam around and ask, “Is everything okay?” That’s  monitoring the EUE: Rather than looking at his back‐of‐house metrics, he’s going out into  the cube farm—er, restaurant floor—and testing the waters.  How the EUE Drives Better SLAs  You establish metrics for what the EUE should be for various operations: so many number  of seconds to complete such‐and‐such a transaction, and so forth. When that metric isn’t  met, you start drilling down into the infrastructure to find out why. That’s where more‐ traditional data center monitoring re‐enters the picture. Rather than using query response  time or whatever to derive the end user’s experience, we’re using it to troubleshoot things  when the end user’s experience is clearly not where we need it to be. Figure 4.2 shows how  EUE monitoring sort of reverses the model.    46 Creating Unified IT Monitoring and Management in Your Environment          Don Jones      Figure 4.2: Monitoring the EUE.  You’ll still have thresholds and other considerations, but they’re set at levels that have  historically been able to deliver an acceptable EUE. As Figure 4.3 shows, a failed EUE is  your cue to start looking at deeper, more technical‐level measurements so that you can see  what’s contributing to the end user problem.