The 4 basic phases of a software evaluation project

Posted August 28, 2009 by Gerry Tolentino
Categories: Project Methodology

Tags: , ,

Selecting the proper software solution that will satisfy both business and technical requirements can be a daunting task for anyone who has never gone through this process.  Vendors with expertise in a certain domain (ie: CRM, ERP, LMS, etc.) will typically boast a “proprietary” methodology that can be used to help in identifying the best software solution for a business problem.  However, these “propriety” methodologies tend to utilize criteria that skew the advantage towards the vendor’s product.  I have been engaged by clients in various software evaluation projects to deliver a non-biased recommendation.  I have found that there are 4 basic phases that are common no matter the business solution is being evaluated.   

   

The 4 basic phases

The 4 basic phases

The following example below is a 4 phase approach utilized by PTCI for selecting specifically a LMS (Learning Management System) solution.  One can substitute the “type” of solution being evaluated depending on the particular initiative.   In this example, “learning management”  can be replaced with another solution such as “customer relationship management” and “LMS” can be replaced with “CRM (Customer Relationship Management).

 Phase 1: The Discover Phase

This phase will evaluate the responses from the LMS vendors. Utilizing a scoring system developed by PTCI and with input from the client, the responses to the RFI is evaluated and a score is associated with each response from the vendors.  Based on the score, the vendor list will be trimmed to the top 3 or 4 vendors.  The top 3 or 4 vendors will be invited “on-site” for a 1 to 2 hour demonstration of their product. This will provide a chance for all interested users to get a first hand look at each LMS application.  Based on feedback from users, the list may be trimmed down to the top 2 vendors.

 As part of this phase, test case scenarios will be generated by PTCI that will be used during the on-site demonstration.  The test case scenarios will be given to vendors in advance of the on-site demonstration.  Vendors will be asked to demonstrate product capability based on the custom test case scenarios.  The test cases will demonstrate how closely the “out-of-the-box” LMS application can meet the specific user requirements. 

 Deliverables for this phase are:

  1.  Test case scenarios – based on “use cases” and functionality matrix documented in the Discover Phase.
  2.  Updated Functionality Requirements Matrix – matrix will be updated using a scoring system developed by PTCI and client.
  3.  Identification of top 2 LMS vendors
  4. Gap analysis and documentation for the top 2 LMS vendors.  This will include changes to workflow or user interface
  5. Project Plan – the plan will updated if necessary

Phase 2: The Evaluate Phase

This phase will evaluate the responses from the LMS vendors. Utilizing a scoring system developed by PTCI and with input from the client, the responses to the RFI is evaluated and a score is associated with each response from the vendors.  Based on the score, the vendor list will be trimmed to the top 3 or 4 vendors.  The top 3 or 4 vendors will be invited “on-site” for a 1 to 2 hour demonstration of their product. This will provide a chance for all interested users to get a first hand look at each LMS application.  Based on feedback from users, the list may be trimmed down to the top 2 vendors.

 As part of this phase, test case scenarios will be generated by PTCI that will be used during the on-site demonstration.  The test case scenarios will be given to vendors in advance of the on-site demonstration.  Vendors will be asked to demonstrate product capability based on the custom test case scenarios.  The test cases will demonstrate how closely the “out-of-the-box” LMS application can meet the specific user requirements. 

Deliverables for this phase are:

  1.  Test case scenarios – based on “use cases” and functionality matrix documented in the Discover Phase.
  2.  Updated Functionality Requirements Matrix – matrix will be updated using a scoring system developed by PTCI and client.
  3. Identification of top 2 LMS vendors
  4. Gap analysis and documentation for the top 2 LMS vendors.  This will include changes to workflow or user interface
  5. Project Plan – the plan will updated if necessary

 Phase 3: The Select Phase

In the event that the LMS is to be installed and supported internally (vs. ASP), it is  recommend that vendors agree to install their product on-site for a 1-week trial period.  This provides the opportunity to:

  1. Determine product installation issues
  2. Compatibility with organization’s technical architecture
  3. Additional opportunity for interested users to “demo” the product

Documents that were generated from the prior phases will be utilized to provide the client with the recommendation for the appropriate LMS product that meets the highest percentage of the requirements.  Based on our findings, PTCI will make a recommendation for the LMS suited for the client’s specific requirements.   If required,

PTCI will also develop a “Build vs. Buy” Comparison document in the event the client is interested in a custom solution. 

 Deliverables for this phase are:

  1. Recommendation of the Learning Management System
  2. LMS Vendor Evaluation and Selection Executive Summary
  3. Build vs. Buy Comparison (if required)
  4. Implementation Cost Estimate (if required)
  5. Project Plan –  the plan will updated if necessary

 Phase 4: The Implement Phase

 Once the client has selected the vendor, PTCI will develop implementation detail design necessary to insure that the base employee data is converted and ongoing data interfaces to/from the LMS system is properly documented.  PTCI will work closely with the selected vendor to ensure that conversion and interfaces work seamlessly with the LMS package.  An implementation project plan will be developed to ensure that the appropriate tasks and milestones are identified and can be easily tracked.  PTCI can also provide the technical resources to work on the conversion and system interfaces to/from the LMS application.

Deliverables for this phase are:

  1. Detailed conversion design and plan
  2. Detailed system interface design
  3. Updated Cost Estimate (if required)
  4. Additional technical resources (if required)
  5. Project Plan –  Detailed Implementation work plan with identified tasks and milestones.

 This basic 4 phase methodology can be used with flexibility in mind.  With a little thought, the format of the deliverables for each phase can easily be created as either an Excel or a Word document.  Hopefully, this information can provide a basis of understanding the basic phases of a system selection process.  At the very least the 4 basic phases discussed here can be used to compare with other “proprietary” vendor evaluation methodology.

Advertisements

Seasoned IT Professional’s first blog

Posted August 6, 2009 by Gerry Tolentino
Categories: Reflections of the President

Welcome to Gerry Tolentino’s blog page.  This is my first post after finally making the decision to expand my internet presence.  As the title of this blog states, I am the President of a consulting firm called Praemium Technology Consulting Inc (PTCI) based out of Schaumburg, IL.  It has taken me a while to make the step to create a blog due to a number of factors.  One being the time commitment required to post and maintain a blog.   PTCI is considered a small consulting firm that provides IT technology services to companies within the Chicagoland area.  Being the founder and president, I wear multiple hats that includes onsite client project/program management consulting enagagements to managing the daily operations of PTCI. 

I consider myself a well seasoned IT professional with roots with a variety of projects under my belt starting with developing BAL (Basic Assembly Language) systems as new graduate out of  college working for a consulting company called Software Architech in Chicago.  Yes – I came up as a mainframe programmer and at this time “internet” was only a pipe dream.  At that time, technology advancement did not come as quickly as it does now.  I’ve always considered myself as open minded and progressive with regards to new technology.  I embraced the new technology paradigm shifts from mainframe to client server all through the adoption of internet technologies.  My initial introduction to social networking was through LinkedIn. I’ve heard colleagues discuss this site so I decided to sign up.  I was not convinced that social networking platforms would be accepted by the business world.  However, I since have realized the power and potential of these new technologies and platform to extend your personal and business presence.  I guess better late than never – hence my first blog. In the future I wanted to share the challenges small technology companies such as PTCI have faced in the past few years and how I hope to utilize the power of social networking tools and platform to compete in this tough business environment – stay tuned and come with me on this journey.