10.9 DESIGN STORAGE, ANALYSIS AND FEEDBACK MECHANISMS

< Day Day Up >

Having determined where and how you are going to get your raw data it does help if you have somewhere to put it.

Try not to fall into the trap of waiting until the data starts streaming in the hope that you will sort it out then. You have your high-level program requirements, you have your composite metrics and you have your base metrics definitions. These all act as inputs to the process of designing and implementing your metrics database.

Now the database may be something as simple as a spreadsheet or a PC database or it can be as complex as a corporate-wide Management Information System. It can be constrained by existing systems that are used by management or it may be an area that has never been addressed by your organization. There are strong arguments for, if circumstances allow, keeping the first implementation of your metrics database simple. For one thing, you will be feeling your way and, with the best will in world, you are unlikely to get it perfectly correct first time around.

My advice is to treat the first implementation of this database as a prototype and to use a simple spreadsheet or database application as the initial platform.

In one organization, a senior management decision has been taken to bring a large number of disparate internal MIS systems together under a single system. Tempting as it could be to consider this approach, it is educational to learn that the initiative is planned over twenty years and has an engineering head count that peaks in hundreds!

The system that you end up with will depend heavily on the scope of your own metrics initiative. My own preferred approach is to use a form of prototype. If you adopt a phased approach to the implementation of your metrics program you may well be able to build a metrics database within a PC quickly and easily using resources from within the metrics team. Operating that system gives you the opportunity to learn and then to modify the system with relatively little cost or public loss of face. As the metrics program expands you can migrate the database to a larger platform within the overall MIS strategy of your organization using the operational prototype as a major input to the development of the eventual system.

As far as analysis and feedback is concerned you may have a problem. The specifics of your analysis may depend upon the data that you get in. For example, whether you use means or medians as the "average" will depend upon the distribution of your observed values. While this kind of detailed question cannot be answered at this time, although you can make some educated guesses as data from software development of products is seldom "normally" distributed, you should consider the type of analysis you will be performing. Incidentally, if the data is normally distributed then the mean equals the median equals the mode anyway.

Within your program design you should consider the frequency of reports, the target audience and the style of the reports. You should also aim to store and analyze information that can be used to show the effectiveness of the program itself. For example, if one of the high-level program requirements is to improve the effectiveness of software estimation then it makes sense to record and store estimates within projects and the actuals against those projects. This enables you to demonstrate the effectiveness of the program over time.

This book is not about database design and a great deal has been written elsewhere about this subject. I suggest that information on that subject be obtained from a better source than myself and you should use expertise internal to your organization as a first port of call. Another subject that has had a great deal of attention over the years is the representation of information. Interestingly, most organizations totally ignore this advice.

Organizations seem to have a preferred style of graph with which they are comfortable. In some, you see histograms or bar charts everywhere; in others you hardly ever see a bar chart but there are pie charts in abundance. This is a great shame because the different styles of pictorial or graphical representation have different purposes and are suited to different requirements.

For example, if you want to show information changing over time you should not use a series of pie charts. It is much more effective to use a simple XY graph perhaps with an associated rolling average.

Another major mistake is to send a senior manager a matrix of raw data. I have seen so-called reports that consist of nothing more than page after page of figures. Managers do not have time to wade through all this information and if you want to get a point across then get the point across, not the data.

Of course, some people go to the other extreme. No raw data is presented but either lots of graphs suddenly land on the managers desk, or even worse, a couple of graphs covered in bars, lines and with small tables stuck in corners. The poor manager does not know what is being said and, if the choice has been to go for lots of graphs, cannot remember what was shown on the first page when he gets to page ten, or twenty or thirty. You may feel I am exaggerating but these are real examples of reports I have seen presented to managers.

Of course, at this stage of your project you cannot expect to design your feedback mechanisms in fine detail but you should determine the style of your reports and, based on the metrics you will be using, you will be able to plan the type of graphical representation best suited to your requirements for information. If in doubt ask an expert, a statistician.

My own choice for style of presentation is to mix words with simple graphs. This recognizes the fact that some people prefer to see pictures while others prefer text. It also means that you can forge links between different sections of a report perhaps linking productivity to your quality indicators.

You should also remember your engineers. They have supplied you with information. Can you give them anything back that they can make use of? If not then can you give them information so that they at least realize that the effort they put into giving you data is being utilized by the organization?

One way to do this is through a quarterly news sheet to all the teams contributing to the metrics program. It is amazing how well disposed towards a metrics initiative this can make people feel.

< Day Day Up >