What IBM DS8000 Should Be Reporting in RMF/SMF – But Aren’t

This is the second in a series of four blogs by Dr. Houtekamer on the status of RMF as a storage performance monitoring tool. This installment is specifically based on experience using available instrumentation for IBM DS8000. What RMF Should Be Telling You About Your Storage – But Isn’t is the first in the series of blogs.

While more advanced capabilities are added with every new generation of the DS8000, these also introduce extra levels of ‘virtualization’ between the host and disk drives. One good example is FlashCopy, which delivers an instantly usable second copy, but also causes hard to predict backend copy workload. Others are Global Mirror or EasyTier, which both lag behind when it comes to the measurement instrumentation.

Although users value added functionality, it is wise to be mindful of the inevitable performance impact caused by increased complexity. Ironically, despite the rush to add more functionality, we have not seen a major update to RMF since ESS was first introduced and the 74.8 records were added to provide back-end RAID groups and ports statistics.

Today, if you want to know replication status, you need to issue a manual console command. Likewise to see how volumes are provisioned by automated tiering or to get FlashCopy counters.  If you want to track the progress of a FlashCopy over time, you need to issue the lsflash -dev IBM.2107-<SERIAL> –l CLI command every few minutes to see how many out-of-sync tracks remain.

Wouldn’t it be better if these capabilities and more were automatically included in RMF?

Critical information currently missing in the RMF records for DS8000 include:

• Configuration data about replication. You’ve invested substantially in replication yet it is hard to see whether replication runs properly. Replication status should be recorded at every RMF interval.
• FlashCopy status and activity information.  FlashCopy provides immediate logical copies, but this can involve significant back-end activity that is not recorded as such in RMF.  FlashCopy is a frequent player in hard-to-identify performance issues.
• EasyTier. While it will supposedly always give you excellent performance, EasyTier cannot provide performance capacity that is not there. In other words, the back-end needs to be capable of handling your workload. Furthermore, as Lee LaFrese said in his recent blog post, “What Enterprise Storage system vendors won’t tell you about SSDs,” EasyTier placement will only work well if history repeats itself.  You may want to know how much overhead it adds and the amount of migration activity that takes place. RMF provides no visibility into migration activity, nor does it show for each volume how many extents are on each tier.
• Replication performance data. There is some data in RMF, but it is very limited and does not even keep separate counters for synchronous and asynchronous copies.  It is very hard to determine the activity for each replication method, especially for sites that use both replication methods.
• Port statistics. With current port counters, determining which ports handle the traffic for which LCU is not possible.

For all of these cases the microcode does maintain this information, or something close to it, but it is cumbersome to get to and work with.

Wouldn’t it be great if the RMF and DS8000 teams got together to work on defining the interface? That way, all relevant information could be part of RMF and we could all use it in our daily proactive performance management.

Alternatively, what RMF doesn’t do can sometimes be done by other tools. Recently the GDPS product started to cut SMF records with Global Mirror session statistics, which is fine (although purchasing GDPS only to get performance reporting on your Global Mirror environment might be a bit expensive).

IBM does not always seem to realize the power and importance of RMF and SMF recording. Indeed, increasing customer pressure already has resulted in some changes. A recent example of this is the inclusion of zBX information in SMF records, which was introduced through customer demand.

When you consider your next DS8000 purchase, also consider discussing the ability to manage it with the mainframe tools that IBM created for this purpose: RMF and SMF.  If your company’s order is big enough, IBM will surely listen