Evaluating the CIS Procurement Process to Avoid Getting Burned: 10 Palatable Ways to Set Realistic Expectations

Greg Galluzzi, President, TMG Consulting, Inc. speaks with Tamara Halbritter of Municipal Technology Journal, March 2003

Just before dinner is served at the home of an excellent cook, your mouth starts watering in anticipation of the entrée du jour. As you bat lively dinner conversation back and forth to four other guests, you eye the textures of several colorful side dishes. The rich aromas fill the room while the host brings in the main course. Once you receive your plate and dish up a generous helping, you enjoy every morsel.

After dinner, with a cup of steaming coffee at your side and a forkful of a luscious dessert in hand, you compliment the chef on the meal. Why? Because the chef met your expectations.

Expectations can affect much more than whether or not a meal is enjoyed; they can also make the difference between the success and failure of a local government’s CIS implementation.

Greg Galluzzi, President of TMG Consulting says, “Local government CIS implementations are a powerful tool in managing costs and improving customer service. However, a recent review of 50 utility CIS projects from planning through installation indicates there’s only a 27 percent chance the CIS installation (projects with signed contracts and full funding) will be completed on time, within budget, and exceed delivery expectations.”

“I don’t know what your definition of success is, but 27 percent isn't a very encouraging number,” says Galluzzi. "One way to prevent this huge room for error is to define and oversee a structured and proven evaluation process as part of your CIS procurement, to create realistic expectations, before the implementation occurs."

The following guidelines are based on Galluzzi's 23 years of IT, CIS, and utility industry experience. These tips will help you define your evaluation process to mitigate risk and give your CIS project a better chance of success.

Defining your evaluation process
Galluzzi advises, "Every single CIS project is critical and complicated." He quotes from the same review of 50 projects, "There is a 26 percent chance the CIS initiative will never make it out of the planning and selection effort; because this failure rate is very high for such projects, local governments need to think of these projects as a strategic effort, with a complete focus, and have complete buy-in from everyone involved."

From Galluzzi’s perspective, lack of support can bring the involved utilities and in some circumstances an entire city to their knees. Prevent this type of disaster by giving your highest priority to the utilities involved in the project, and assign your best people to it.

In addition, Galluzzi says, "You'd be surprised how many cities rely on a request for proposal (RFP) as their first step in the process, before they understand fully what they want to do with the CIS. In fact, preparation before RFP development is usually ignored."

Though the RFP is a useful tool, asking questions and establishing the purpose of the CIS is recommended before searching for and selecting a vendor(s). Develop an evaluation outlining the procurement process and defining what your municipality wants out of the implementation will help you get the best bang for your buck. Galluzzi suggests using the following tips as guidelines to better organize the procurement process and set up realistic expectations for all parties involved.

10 tips for designing an effective evaluation
Galluzzi provides the following 10 tips to ensure the best candidate is chosen from the proposals submitted to insure a successful implementation effort.

1. Develop a blueprint
The scope of every project should be agreed upon in advance, so that everyone has an understanding of what you are trying to do. Include strategies, objectives, and requirements in this blueprint. Focus on who will be affected by the new system: How may users will be affected and to what extent? How much will it cost? Who will pay for it? Also consider how it will affect the utility: How many people will be affected? Where will it be housed? This blueprint should provide a broad overview and begin to define expectations

2. Bring your utilities together
Usually the biggest issues in CIS implementations occur when multiple utilities aren't communicating effectively. Most CIS projects coordinate changes to three to seven utilities such as electricity, natural gas, water, wastewater, solid waste, drainage, and transportation. These utilities may not deal with one another on a regular basis. One of the first challenges of a CIS implementation is bringing these utilities together in the planning stages.

Galluzzi says, "An effective way of opening the lines of communication is to ask each utility what they hope to accomplish with the new system, and what is driving their replacement effort." Often, with multiple utilities, not all of them are affected by the same replacement drivers. In fact, one or more may not want the system due to the difference between the benefits they'll receive from the new system and the ultimate replacement cost to them.

For example, deregulation of markets is one driver for electric utilities to implement a new system. The electric utility wants to enter the market to purchase a new system that is deregulation capable. Because a new CIS may cost millions of dollars, the solid waste and water utility may say, "We are not impacted by deregulation, therefore we don't want to spend the money on a new system."

If you understand each utility’s drivers, and make sure all utilities know what the others' drivers are, everyone will know where they stand in relation to what you are trying to accomplish with the CIS, and you can begin to sort out the pros and cons for all.

3. Decide who’s picking up the tab
A lot of times, the electric company may pay for 70 percent of the project and the rest of the utilities combined will split the other 30 percent. Galluzzi cautions, “Money can become political and provoke sensitivities around the purchase of a system.” If you can clearly state how each utility will benefit, and figure out their sensitivities up front, the ride will be smoother.

4. Decide who is invited to submit an RFP
Most cities are used to "tire kicking," looking at systems that have been on the market for years and seeing how they'll hold up to your requirements. When initiating procurement, you may find out these old standbys aren't doing the complete mix of things you need. At the same time, you might discover only a certain amount of vendors do provide what you want. Who do you issue the RFP out to, then?

"Usually, there are about a hundred billing and customer information system solutions in the market place at any point in time," Galluzzi states. "Of this 100, there might be 30 which may meet your needs. Of this 30, 20 of the vendors may submit proposals. Of those 20, by examining their proposal and comparing it against your stated minimum requirements (see tip #6), you can cut the number to no more than seven vendors that, before a demonstration, you really need to look at in detail and analyze their proposal. From this seven, narrow it down to one or two finalists."

Unfortunately, in Galluzzi’s experience, as many as 350 RFPs have been requested during a single procurement effort. With only 100 possible CIS solutions, that’s a huge number. Municipalities typically don't restrict how many RFPs they receive, but maybe that's something to consider.

5. Keep the RFP flexible
Structure your RFP to allow flexible responses. For example, the application software and related database software are submitted by the CIS vendors; services components may be submitted by the CIS vendor or a solution integrator; and the third component, regarding hardware and system software, is bid by hardware manufacturers, value added resellers, or purchased through state contracts. Sometimes everything comes from one vendor; although the majority of the time, solutions represent offerings from several vendors.

The key to making your RFP flexible is by not restricting it to the bundled solution proposed. Galluzzi suggests, "Request a proposal consisting of a bundled solution with the ability toun-bundle and award a contract for individual components, and allow vendors to submit an alternate proposal for one or more individual solution components, i.e. server hardware, client desktops, conversion services, etc. Without this flexibility, the evaluation team may find they like the software solution proposed. However, if they don’t believe the proposed solution integrator is the most qualified to implement the solution, and they would prefer the software implementation services of another company, this may cost more time. When the structure of the RFP and associated vendor proposal is restricted, the evaluation team must accept the proposed bundled solution or re-issue the RFP to solicit solution components."

6. Focus on minimum requirements up front
When you receive proposals for review, focus first on your minimum requirements to weed out the RFPs that don't meet them. There's no need to further consider a solution if it doesn't meet stated minimum requirements; this is a time waster if proposals, that can’t possibly be placed in front of a council or board for approval, continue to be reviewed and evaluated.

Galluzzi gives an example, "An evaluation team decided to select an IBM AS400 based solution. Unfortunately, it was not part of the city's technical plan. When we rolled it up to the executive team for approval, the chief technology officer said, ‘Not an AS400, we’re not allowing that platform in our shop.’ If the evaluation team had identified these types of minimum requirements up front, it would have saved the team valuable time."

Another frequent minimum requirement, which usually affects municipalities, is: the solution must be implemented at another utility similar in size and nature to the city. Making a list of these types of required minimums and initially applying these to the proposals can help you quickly identify a short list of qualified proposals.

7. Structure the one-day product demonstration
For each qualified proposal, schedule a one-day product demonstration. Structure these one-day product demos to glean a set amount of information and score the vendor on the specific points of review. Assign questions or specific demonstrations desired from the vendor, such as, "Please demonstrate how to access a customer account in the new system." Make sure everyone in attendance knows your grading scale. In response to the example given, have them grade how well the vendor's system lets you access customer accounts.

If the product demonstration is not structured with a clear scoring system, people leave the room unsure. Frequently, this leads to evaluation nightmares where people review vendors repeatedly and never move out of the evaluation stage to name a finalist for the confirmation stage.

8. Only take one solution through the confirmation process
One mistake cities make is trying to take more than one vendor through a due diligence process. When evaluation teams insist on taking more than one vendor through the confirmation process, it doubles or triples the time required for the five-day product demonstration, client site and corporate office visits, scope development workshops, contract discussions, and installation plan development. It causes an extreme amount of work if the team can’t agree on one vendor finalist.

It's strongly recommended to narrow these proposals down to one choice, with a second selected as a backup should the first one not work out. You can keep this vendor on its toes by making it clear to the first choice that their company can easily be eliminated and replaced by the second choice.

9. Allow sufficient time for confirmation and due diligence activities
Many cities make the mistake of moving from evaluation activities directly into contract negotiation and award of contract. A major deficiency in most procurement efforts is the lack of time spent in performing necessary due diligence and confirmation activities. At a minimum these activities should include the following:

Conduct a detailed vendor product review
Conduct a five-day detailed product review, using a combination of business process and detailed functions and features to achieve a detailed understanding of integration points and interfaces, modifications, configuration, reports, forms, and anything else concerning product implementation. Drill down to the detail of what the utilities want to customize and configure to make the implementation work for everyone. Determine the impact to your existing system and what it takes to perform activities such as convert your data, configure the system, and modify base reports. Those things are addressed in the RFP, but the answers may not be clear until the two parties, city (with utility representatives) and vendors are face to face. A detailed product review is the time to talk interactively, a way to drive the formal solution toward what you need.

Fully develop and confirm solution cost
Based on the result of the detailed product review, vendors should provide detailed pricing for identified modifications. They will come back with: “Here are the 10 modifications you are asking for and the cost.” Then you can decide, is it worth $100K to make it work this way? Often, the dollars to customize are not worth it; but this way, you can see if you want to Move forward with customization or not.

Identify organizational impact
Be clear on how the system will impact the existing environments and the business itself. Try not to focus on only the vendor side of things with hardware, software, and services. Determine if you have training facilities, if you are going to pay your workers overtime for this, how many desktop versions of required software you'll need, how the cost will impact other areas of your municipality and the utilities.

Discuss and negotiate the contract package
Because of the scope of these kinds of projects, you usually end up with many contracts within the contract package. These contracts require significant time to develop? It can take up to four months to craft hosting or outsourced contracts, and to negotiate reasonable terms and conditions. Make sure you allow for time for creating contracts and for obtaining agreement.