Did you forget your: Username / Password ?

Microsoft

TechWell

Home

CM Journal Articles

The Importance of a Directory Structure for the Development Process

Written by Charles Edwards

Sunday, 31 March 2002 16:00

Software Process is becoming more and more important in SCM. Gone are the days of simple configuration managing the source code and software release builds. Now we need to manage versions of the UML Models, versions of the requirements, versions of the tests, versions of the iteration plans and be able to create integration streams for all the different disciplines too. Requirement Analysts should be able to work on a branch of a Use Case model, Testers should be able to work on a branch of their test model, etc. At the end of an iteration, all of this should be brought together and released as an iteration release, for the process to be properly controlled.
I see software configuration and change management coming to the rescue for process control, but before they do, they need to take a broader look around and be aware of more than their traditional domain of source code, because otherwise they could end up being part of the process problem, rather than part of the process solution.

Assumption

This article assumes the use of some of the Rational Tools for software development process. This does not imply it cannot be done with other tools, provided they use a file saving paradigm and not a database container for all data elements, so that the CM tools can manage each configuration item independently.

Introduction

Having worked on a couple large financial software projects over the last few years (one of 120+ people and another of 150+ people over multiple sites) and been part of trying to implement the Rational Unified Process while the team were learning an array of new technologies and concepts such as requirements gathering, use case modeling, testing, iterative building, change and configuration management, software development tools, etc., I can tell you that

one of the biggest lessons I learned was how important it is to organise and align what I've termed container structures.

As in the UML Class Diagram to the left, for the term container structures, think "directory structures", directories and files within directories and files recursively.

However what I am trying to describe here are concepts not only relating to a file management system, but rather all software development tools, such as a UML modeling tools container structures, Test Tools, Requirements gathering tools, Content management tools, etc. such that they all align and co-operate along a similar philosophy.

Any software which has an explorer type interface, which allows storage in a tree structure. Think of the browser window in Rational Rose, or the Folders Window in Microsoft Outlook for example. If we organise a Rose UML Model, why not align the package structure of this UML Model to the Directory structure below it?

Problem

The problems I have seen on projects where directory structures philosophies were not thought out carefully enough; or kept being changed through the project life cycle were huge causes of unnecessary problems. Problems that inadvertently generate huge volumes of work in converting a directory structure from one philosophy to another, moving current contents of the old structure to the new structure to keep the artifacts in a place where they could easily be located for builds, audits, etc. Of course in the rush of the project and under deadlines pressure, this task becomes such a low priority that it never gets done.

On one project I saw the fundamental directory structure change three times, on the grounds that it was either too complex and no-one knew where to put of find things, or it was too simple and couldn't cope with the varying demands! This just served to confuse everyone even more and eventually some people were using one structure while others were using another!

Without getting the directory structure of the software system under development correct and in synchronisation with the Configuration Management policy and the process spells disaster before you've even started a project.

Background

Lets take a look at how the software development process has evolved and still continues to evolve. While this progress has in no way been linear and sequential, it has from my experience tended to start from the writing of code and worked its way outwards (see diagram with red horizontal arrows below), with other practises and disciplines coming towards it from the outside, such as project management and configuration management (see diagram with vertical arrows up and down).

First off we had the raw production of Code by thinking it through, and then coding and testing. Around the 60's and early 70's.
This evolved into Code Unit Testing and Project Management for software development. Around the late 70's early 80's.
Later the importance of Architecture, Product Design and at the same time Configuration Management was recognised. Around the mid to late 80's.
Design gave way to Analysis while Unit testing gave way to User Acceptance Testing and this was added into the picture. Also mid to late 80's.
Then people realised Requirements should be gathered & Deployment should be better managed. Late 80's into early 90's.
As changes kept undermining projects Change Management was brought into the picture. Early 90's.
As more and more tools were being required for the software development process over multiple projects Environment & Programme Management were brought in to the software development process. Mid to late 90's.
More recently system Requirements has started tying into Business Modeling & Knowledge Architecture leaving us with a group if disciplines as shown in this diagram. Been around since the early 90's, but more recently being implemented to a greater extent.

The point of the diagram above is really to illustrate how more and more configuration items have started turning up in our typical software development projects. What started out as a simply necessity to branch and merge the source code, has evolved into complex branching and merging of Designs, Requirements, Models, Tests and other general items. The red horizontal arrows show how the coding process expanded outwards, while the Red vertical arrows show how other areas of business specialization have been brought in to manage the complex area of software development.

Areas of Specialization

Different areas of business specialisation have a different emphases in terms of what they try to achieve and what they consider significant for success. However the boundaries do cross and blur.

Architects and Designers tend to be Product and Code centric.

Architects organise their lives around the product they design and build. In an attempt to isolate systems into loosely coupled but highly cohesive components, the container organisation tends to express itself in the system components with their constituent sub-systems. The teams also usually align themselves around the components so that certain people are responsible for certain components. Where and What does the delivery comprise of?

Project Management Tend to be Features and Time Centric

The project managers on the other hand tend to try and organise themselves around all the teams disciplines and within that focus on a features delivery based set of containers over time. Who is going to deliver what when?

Business, Requirements and Testing tend to be Feature Centric

The Business, Requirements and Testing people, like to organise their lives around the business and system features that are to be delivered. What and Why does the business and the system do for this iteration delivery?

Configuration Management & Legal requirements tend to be Standards & Audit Centric

Pressure from the Business in terms of adhering to audit, legal and quality fall on the shoulders of the configuration managers or testers (depending upon the organisation), which tends to be standards centric and forces implementation of some CMM level or ISO 9001, SPICE, etc. CM have to try and bridge all these different perspectives, and its easy to fall into the trap of just satisfying the Architects and Designers in being product and code centric.

We need to make sure we cater for all these different forces and requirements on the structure we implement so that the software development processes all work harmoniously together.

Container Organisation Structures

File directory organisation

There is a very fine line in balancing the understanding of project members about the structure of the file system with the complexity of the file system you deploy. Too complex and they won't understand it or use it properly, too simple and each person will start inventing their own more complex one.

Good guidelines and training early with fairly disciplined rules on directory creation at certain levels should help with this problem.

This UML Class diagram attempts to explain the container structure for file systems (I'm sure it could be more elegantly modeled.)

This suggested directory structure would consist of conceptual levels of decomposition starting at the Enterprise level. Each Enterprise may have one or many Programs or Systems under development. Each System would be made up of some sub-systems. Each sub-system would be made up of components, which may be shared across sub-systems and so on. At each level which is just a containing directory, one would have either a set of lower level directories and a set of discipline containers. Within the Discipline containers, would be Artifacts such as Models, Project Plans, Word Documents, Spreadsheets, XML files, Test scripts, Deployment information, Environment Guidelines, etc.

Requirements Analysts could use the System level Requirements Discipline Container for all their Use Case Models, for example. Source Code would be stored in either the Component level or the Sub-system level Implementation Discipline. On one of our projects we chose to call Implementation "Src" instead.

While this might not make immediate sense to someone on the team who is not too knowledgeable on UML modeling or the Rational Unified Process, if it was explained using example directory structures and a set of guidelines and rules, this concept is really quite simple, as it embodies recursion and fits in with the general understanding of system decomposition.

UML Model Directory Organisation

The Rational Unified Process suggests that there are various different views one can take of the same system. These are the Use Case View from a Business and Requirements perspective, the Logical, the Implementation, the Process and the Deployment View from the Architecture's perspective. These are called the 4+1 views. Four Architectural and One Requirements views.

Within this framework of Views, the next level down is organised by Model type, or Model more specifically Model Purpose.

So Under the Use Case View, we could typically have for example Business Use Case Model and System Use Case Model.
Under the Logical view we could typically have an Analysis Model, a Design Model an Implementation of the design model and a Process Model.
We could have the Component Model under the Component View.
We could have the deployment Model under the Deployment View.

Rational Rose allows the use of one Model file with many .cat files, similar to the concept of a subdirectories (.cat) within a directory (.mdl). Each specific model would then live in one or many .cat files which could then be saved in the directory structure as outlined above. The single overall Model file could then be stored according to the owner of the overall model, either the Architect, Configuration manager or the Process Engineer, whoever ultimately takes or is given ownership of it. This is where the alignment between container structures starts coming in.

Other Configuration Items Organisation

Without going into detail on the items below, they could be considered for configuration management in the same way:

Project / Programme overall and iteration plans
Project Web layout
Business Use Case Models
Non-Functional requirements
Use Case documentation and trace-ability to models
Software Architecture Document
Change Management structures and processes
Template documents
Vision documents
Risk lists
Security Models
Candidate Architectures and Prototypes
Project Metrics
Infrastructure Models
Configuration Management Plans
Deployment Plans
many more...

Conclusion

With the experience I have had to date of large software engineering projects, the single biggest stumbling block to speedy progress was not having the basics in place early enough. Trying to change the basics in full gallop is like shooting yourself in the foot at full gallop.

Make sure the container structures align across all disciplines. Make sure all tools structures have a place in the directory structure. Make sure the that the configuration management process and change management process work within these structures.

Ideally if you can have your CM structure and process in place ready, working and tested from the last project, you are off to a fantastic start on your new project. If not, the next best is to have it ready in the inception phase of the new project, so that when the work begins in earnest, everything will have a place to live.

Appeal

We are trying to build up a set of ready made documents such as Configuration Management Plans, Directory Structure guidelines, Software Development Plans, etc. Anything which you have found useful on your Unified Process project that could help others get a kick start on their Unified Process project.

We envisage an internet site which Process Engineers can visit and download existing documents that can save them time and energy. Either you or we can convert all specific company information into general or anonymous terms to preserve and protect company specifics and integrity. Please submit them to us at Submit a Document to Publish

References

Rational Corporation. Rational Unified Process. 2002.

Senge, Peter. The Fifth Discipline. The Art & Practise of the Learning Organisation. 1990.

Edwards, Charles. http://www.processwave.com/ April 2002

Charles Edwards is a contributing editor for CM Crossroads and an independent consultant who performs RUP implementation for organizations that recognize the need to improve their software development process. He works with and contributes to the www.processwave.com web site for process engineers.

You can reach Charles by email at charles.edwards@processwave.co.uk

Set as favorite

Bookmark

Email this