Project Rescue

Problem Solving

Able to think creatively, understand the technical complexities and de-construct multi-layered problems. One troubled project suffered from a database that required workload separation, removal of Wide Area Network consequences and stream-lined code in specific areas. By making these changes, the company was able to adopt the upgraded product and avoid ongoing over-time for their HR workers.

Technical Complexity

Resolution of the SAP technical problem occurred just prior to go-live of a mult-million dollar SAP project. Because the problem with the SAN sub-system was identified and resolved prior to go-live, the company was able to deploy on time and avoid significant delay and cost. Problem resolution include a very large multi-vendor multi-discipline team where potential causes were identified, possible tests identified and then causes ruled out.

Troubled Project Rescue Experience

Project #1 - SAP Performance Problem

Background: A large Oil and Gas company had done considerable work to create a new SAP environment which brought together functionality from previous separate SAP systems. Prior to go-live of the new SAP environment, concerns with performance of the new environment were observed.

Resolution: Following investigation with vendors and multiple technical teams the issue was determined to exist in the IO subsystem. Additional paths to the SAN were added and other IO workload removed to ensure adequate performance was obtained.

Technique: brainstorming sessions followed by a methodical removal of possible causes. The team established a mechanism to prove/dis-prove the issue existed following each change and tracked progress carefully.

Results: SAP went live on schedule.

Project #2 - HR Application Upgrade Problem

Background: A large multi-national company had recently upgraded the Human Resource application to the latest version. After go-live, it became apparent that the system was too slow to meet minimum levels of required performance.

Resolution: The database was split into a primary DB available for online activity and a copy was made for reporting purposes to isolate OLTP from DSS activity. Key portions of heavily used code were re-written to remove all non-essential code, thereby streamlining the code used for data entry. Additional hardware was deployed to enable geographically dispersed users to have acceptable performance. The vendor was unable to resolve a memory leak problem entirely but did build in a "catch and release" mechanism to minimize the impact of the leak. Numerous SQL queries were re-written and indexes added to the database to provide faster response.

Technique: Twice a week meetings with daily follow-up. Engaged database experts to focus on the database issues and found a way to validate performance outside of the application. Focused the vendor and outsource providers on the path forwards versus how the problem had occurred. Did site visits to see the end users and the impact of the over-burdened data entry code.

Results: The HR system was deemed to have acceptable performance and the problem resolution team dissolved.

Project #3 - Web Site Performance Problem

Background: A large multi-national company was about to deploy a new internal web site on Sharepoint. Prior to go-live, automated testing found that page loads of the home page were taking 72 seconds, on average, under heavy load.

Resolution: Restructured the architecture of the Sharepoint solution. Re-installed where necessary or brought up to consistent patch levels. Assessed the web pages with the vendor with numerous recommedations. The majority of the 29 memory leaks were resolved by go-live. Removed the live stock price from the page until the vendor could separate the activity of pulling the stock price from an external location from the home page.

Technique: Engaged infrastructure experts. Focused the vendor, the outsource partners, the internal technical teams and management on the goal. Multiple areas were scrutinized simultaneously with significant measurable improvements in all areas.

Results: Web page loads provided acceptable, measurable performance. The stock price was added back into the home page within weeks of go-live.