Project Management in Practice

Beyond the alphabet soup of PRINCE2, MSP, MoP, PMBOK, ITIL, Agile

Tag Archives: Programming

Proof of Concept, Prototype, Pilot, Agile … confused?


Proof of Concept, Prototype, Pilot, Agile … confusedEvery industry sector has its own jargon that users or system developers use. IT is no different and plenty of words are used interchangeably that different stakeholders take to mean different things. This often makes for difficult conversations down the track. My colleague Nathan Heazlewood recently put in some effort in defining some of these. Thought he did an excellent job, well worth sharing. Hopefully it comes in handy in avoiding mismatches of expectation.

Proof of Concept (PoC): a small exercise to test a discrete design idea or assumption. An example of a POC is testing whether one technology talks to another.

Prototype: a system that tries to simulate the full system or at least a material part of it. May be completely discarded should a production version follow.

Pilot (or trial): A Pilot uses the full production system and tests it against a subset of the general intended audience. The reason for doing a pilot is to get a better understanding of how the product will be used in the field and to refine the product.

Agile development: An ‘agile’ application is one that is built to production quality in small incremental steps, some of which may be put into ‘production’ before the complete system is complete.

Production: A production system is the full system suitable for use by the complete target user group with the level of robustness and functional and non-functional agreed within the associated contractual documents.

Proof of Concept

A Proof of Concept (POC) is a small exercise to test a discrete design idea or assumption. An example of a POC is testing whether one technology talks to another. To be deliberate, a POC should clearly state what it is to be proven and to what degree. For example, it may not be enough that a message can be transferred from system A to system B if the complete solution calls a specific level of performance and reliability. You may need to test under certain conditions. POC’s often take the outward form of a simple “Hello World” but that may not be enough if the system needs to support an extended character set.

Characteristics In Scope Out of Scope
  • In the interests of minimising time and cost a Proof of Concept will relate to only small parts of a complete system.
  • A POC will utilise a development or test environment where possible to minimise change authorisation overheads of using a production environment.
  • Non-functional considerations will generally be ignored unless specifically requested.
  • Assessment of one or more important aspects of what might be involved with a production system. Only aspects specifically stated as being ‘in scope’ will be regarded as part of the ‘Proof of Concept’.
  • Functionality
  • Performance
  • User data for retention
  • Re-usable code
  • Scalability
  • Concurrency/capacity
  • Usability
  • Backup of data or code
  • Security
  • Backwards or forwards compatibility
  • Multi-browser or multi-OS support
  • Reliability
  • Availability
  • Documentation
  • Monitoring

Prototype

A prototype is a system that tries to simulate the full system or at least a material part of it. Prototypes are used to test the viability or usefulness of a system or subsystem. The only objective of a Prototype is to get a defined group of intended users of the projected system in front of the prototype and he/she/it (the user might be another system) should be able to visualize the experience of using the complete system. A prototype may provide some re-usable components that can be re-used in a pilot or production version, however it is also possible that it will be more efficient to re-do most or all of the system.

Characteristics In Scope Out of Scope
  • In the interests of minimising time and cost a Prototype will be assembled quickly with a level of quality intended for demonstration purposes only.
  • Functional aspects may be mimicked utilising ‘short-cuts’ or ‘work-arounds’ to minimise time spent on the prototype. These should be replaced with robust measures for a production system (e.g. ‘local copies’ of data may used rather than connections to data sources, components will not be fully documented or commented, testing may be minimised etc).
  • Non-functional considerations will generally be ignored unless specifically requested.
  • A prototype should only be tested for a defined period after which it will be ‘switched off’ while a production version is established.
  • Data used in a prototype should be ‘test’ data and will be disposed of at the end of the prototype demonstration period.
  • Components of a prototype may be completely discarded prior to building a pilot or production version.
  • Functionality
  • Usability
  • Backup of code
  • User testing
  • Limited user familiarisation
  • Performance
  • User data for retention
  • Re-usable code
  • Scalability
  • Concurrency/capacity
  • Backup of data
  • Security
  • Backwards or forwards compatibility
  • Multi-browser or multi-OS support
  • Reliability
  • Availability
  • Documentation
  • Monitoring
  • Full ‘factory’ or ‘unit’ testing
  • Training

Pilot (or Trial)

A Pilot uses the full production system and tests it against a subset of the general intended audience. The reason for doing a pilot is to get a better understanding of how the product will be used in the field and to refine the product. Maybe there are some open ended questions about scalability that only a live audience using the product can answer. You may consider prepping your pilot group to evaluate the full breadth of the application and give them channels to provide feedback.

Characteristics In Scope Out of Scope
  • A system as robust as stated within contractual documentation.

 

  • Functionality
  • User data for retention
  • Re-usable code
  • Usability
  • Backup of data or code
  • Security
  • Limited documentation
  • User testing
  • Training for pilot personnel
  • Other considerations As stated within contractual documentation (e.g. Statement of Work)

 

  • As stated within contractual documentation (e.g. Statement of Work)

 

Agile

An ‘agile’ application development is one that is built to production quality in small incremental steps.

Characteristics In Scope Out of Scope
  • Incremental releases of components of a system to a level of robustness as defined within the Agile documentation
  • Quality/robustness is intended from the outset, however only for a limited part of the overall system to begin with, with additional functions added over time.
  • As stated within contractual documentation (e.g. Statement of Work)
  • As stated within contractual documentation (e.g. Statement of Work)

Production

A production system is the full system suitable for use by the complete target user group with the level of robustness and functional and non-functional agreed within the associated contractual documents.

Characteristics In Scope Out of Scope
  • A system as robust as stated within contractual documentation.
  • As stated within contractual documentation (e.g. Statement of Work)
  • As stated within contractual documentation (e.g. Statement of Work)

Credits: Nathan Heazlewood, who adapted it from http://www.contenthere.net/2007/03/poc-prototype-or-pilot-when-and-why.html

Image Credit: austinstar.hubpages.com

Is Agile for me?


If you talk to any project team, you will find varying range of opinions regarding the best methodology for development – all very passionately putting their points of view across. Barring the occasional accident, most of these teams have given considerable thought to what is most applicable to them and have followed that course. When each of them argue in favor of a different methodology, it is well founded in facts. However, the one thing that is missed in all these discussions is that the challenges facing all the different teams are not the same. Let us consider various scenarios.

Follow this scenario … an organization realizes the need for a new system. They go through an internal requirements gathering process and invite tenders for the system, to be implemented by a certain date. You are a professional services organization. You are required to estimate the effort required to design and implement the system. In your response, you have to provide a price, as that would undoubtably be one of the selection criteria. You happen to win the tender. What is your status? You are now locked into a fixed deliverable to the client, to be completed by a date within a cost you have specified. Is there any room for agility? When you think about the triangle of constraints, effectively you can move none.

You can legitimately ask why not estimate to do the work following Agile principles, that would solve this dilemma. Let us then review how business is done in the real world. For all of us IT professionals that like to think we’re God’s gift to mankind :o), the reality is that we’re a support cast in most organizations, that only utilize IT as a means to achieving their business goals. In business, everything is driven by supply and demand. Think of it this way … if you bought something from a store, you may be able to take it back and get something else that is in a slightly different color, or size. However, if you ordered something custom to be made, and then changed your mind on the size, would you expect that to be accommodated? This is the difference between a product sale and custom work. Software development is no different.

When you are developing a product, or are in a development or support team within the organization, you have a steady commitment to resources. You have the ability to select your sprint schedule. Your costs are also therefore fixed. You have some leeway on scope. In such scenarios the organization has already accepted the cost of this continued effort and therefore scrutiny on the resources (and by extension cost) is not the same. The scrutiny is likely to be more along the lines of value for money … is the overhead we have agreed to getting us the improvements (or new functionalities if it is a product development) we’re after.

Custom development is an entirely different beast. Here uncertainty reigns. Most likely reason the organization is getting you to do the project is they do not have the resources or capability to do it themselves. At the same time many of the users will also not have the understanding of what is possible to achieve. In such scenarios you will end up spending significant effort in communicating possibilities and success criteria for your work to be accepted. Encouraging change of requirements, even at late stages like Agile methodologies will ensure you never come in on budget or are forever locked into a change control battle.

The second key when considering using Agile methodologies is the likely size of the project team. In essence the Agile teams are self organizing teams that work based on a given set of requirements and set their own schedule. The emphasis is on responding to change over following a plan. This works really well and reduces overheads. However, this is only true for a small team. When the team size goes to beyond six or seven, this becomes more and more difficult. Adding each new team member adds many new lines of communication necessary to keep this self-organization working efficiently. Bigger the team lesser the chance of success using Agile methodologies.

If you have been landed with an Agile project because no-one knows the requirements, think again. That is a guarantee for chaos. Agile is not a substitute for requirements gathering. In fact, Agile requires a continued emphasis on requirements through product backlogs and user stories. There is a constant requirement to refine user stories, even for items that do not make it to the current sprint. Many a time customers will not object to time spent writing code and testing it. They will object to this continuous refinement and management efforts. Before you jump in, consider if your client is inclined this way.

Does this mean that you cannot use Agile principles at all in some of these scenarios? No, that is not necessarily the case. You can use Agile principles in your teams, but will most likely struggle to run the entire project using Agile practices. You will need something overarching to ensure adequate control and communication. Agile is best suited for product development and in-house support teams.

Handling emotions in projects


Most people get emotionally attached to any creative work. While there is considerable element of science in breaking problems down, in software development, there is an equally significant element of creativity in resolving complex requirements. This is further heightened if you get to design the solution from ground up. I was part of such a solution – my role included business analysis and project management, but no development. However, I still felt significant ownership of the application, as I had contributed to ultimately what was delivered to the client and it had won awards in geospatial and port domains.

We had built the application with a view to making it modular enough to take individual functionality off and build another solution without having to start from first principles. Our lead developer had put it this way – a box of Legos – build what you will with it. We had tried to build this level of re-usability for some time and had finally looked like we had made a breakthrough. We were really excited with the prospect and the awards seemed a perfect vindication of our approach.

In a similar timeline our company became re-seller for a similar product. That product had a significantly larger development team behind it. As we provide implementation and customization on that product, it is becoming increasingly obvious that our original development would eventually be swallowed up by this product. One of the weaknesses of our development was a lack of configuration utility – which the new product addresses quite elegantly.

We are currently undertaking an exercise to upgrade the existing application to the latest Silverlight design patterns. We have had to think long and hard about whether that is indeed something that makes sense. What we have now decided is to produce a migration plan that would eventually merge the two together – hopefully leveraging best of both. This relies on design patterns of both being compatible. We’re in the process of undertaking a study to establish whether that is a realistic option.

From a business and technical point of view this makes good sense. We are simply not large enough sustain both and do justice to either. However, when you have your blood, sweat and tears invested in something, it is hard to let go. Mine was less than the developers … and it was no easier.

%d bloggers like this: