box10.gif (1299 bytes)

 

 

 

 

 

 

Government & policy

SSHA riddled with problems, operational review finds

TORONTO – An independent review of Ontario’s Smart Systems for Health Agency (SSHA), commissioned by the provincial government and SSHA, has found the organization operates with inadequate strategic planning and project management, poor levels of service delivery, governance and accountability problems, little measurement of return-on-investment, and low levels of customer satisfaction.

According to Deloitte Consulting, which conducted the review, the value of SSHA’s accomplishments against a $458 million investment over the last four years (including the current year plan) is difficult to assess.

Nevertheless, the 100-page operational review asserts that SSHA could and should play an important role in Ontario’s eHealth strategy. It also emphasizes, however, that major improvements will be necessary before Ontarians can be assured that investments in the agency are resulting in value-for-money.

The following text consists of extracts from the report, which can be found at www.ssha.on.ca/main.asp.


Key Findings

SSHA has been operating in a fluid and complex business environment without formal strategic direction from the Ministry. As a consequence, it has been unable to develop a concrete and detailed strategic plan. This situation has made it difficult for SSHA to establish its business priorities and to meet client and stakeholder expectations. Despite these issues there is overriding support within the broader healthcare community for a provincial agency such as SSHA, dedicated to providing eHealth solutions. In fact, the need for an effective eHealth Agency is expected to increase in the emerging eHealth strategy.

Consultations with clients and stakeholders determined that the SSHA brand is not well regarded. Much of the client dissatisfaction with SSHA’s service is warranted, although the Agency has also faced criticism about issues that fall outside of its mandate. This is a reflection of the nebulous understanding about SSHA’s role and core business within the broader healthcare community. Clients and stakeholders do not distinguish the Agency from the Ministry especially since the Ministry acts as funding provider, oversight manager and client. The broader healthcare community has a range of views on the role that an improved SSHA should play. The spectrum of roles spans from strictly a secure infrastructure provider to a full-service Electronic Health Record (‘EHR’) developer, integrator and operator. The brand and reputation of SSHA can be improved over time, but only through a more disciplined adherence to fewer approved priorities. SSHA can no longer attempt to be ‘all things to all people’ in the eHealth sector, and at the same time deliver on key commitments.

SSHA has not demonstrated its value proposition to clients. The initial infrastructure deployment by SSHA has been complete since 2003. The Agency has been slow to shift its focus from deployment to operations and service excellence. It has not articulated a standardized product and service catalogue with associated pricing that can be compared to the market. The scope expansion inFY04/5 from seven priority areas to the inclusion of client business solutions has tested the capacity of the organization. It has eroded focus on achieving operational excellence within its core products and services. The result has been an unfocused array of clients, products and services that reflect a legitimate desire to utilize deployed assets, but lack a clear commitment to client service and outcome measurement. In a client focused organization, stakeholder committees and user groups would be formed. Client relationships would be supported by service-level agreements (SLAs) and performance reporting.

The advantages afforded by an agency construct have been diminished by SSHA operating more as a Ministry division than an independent operating entity. The current legislation (Regulation 54/05) under the Development Corporation Act restricts the Agency to Ministry-based annual funding as part of the Results Based Planning (‘RBP’) cycle used by the Province. This leaves the Agency susceptible to operational and tactical direction from the Ministry that blurs overall accountability for results. Revenue collection from other sources is prohibited under the Act, yet opportunities exist for joint funding, or cost recovery of services that are being left unrealized. Enabling legislation specific to SSHA is required to support a multiyear funding model with an annual accountability agreement, allowing SSHA to operate more like a transfer payment agency than a division of the Ministry.

The Agency has made significant progress in addressing its founding mandate and business purpose, namely the introduction of secure networks and data centres. However, there is a considerable way to go before value will be realized from the enabling infrastructure. SSHA has installed over 1,700 circuits to connect public hospitals, CCACs, public health units, physicians, and other provincial healthcare providers. The Agency has provisioned two data centre facilities. The result is an ability to host its current 19 applications in a highly available environment. However, significant questions exist relating to the effectiveness of certain decisions and the value obtained from significant investments.

The Agency has faced substantial challenges with respect to the rate of user adoption as well as security architecture methods and expectations. Adoption of secure email as a means of transmitting electronic health information has not met Ministry and SSHA expectations. Of the 60,000 secure email boxes that have been installed since 2003, only one-third mailboxes are in active use. The Agency does not report on whether the sites in use are for the purposes intended, and has not established a position regarding the future of this program.

Similarly, there are issues to overcome with respect to access to a secure eHealth environment. The initial vision of PKI certification of end users has proven to be problematic (e.g., only a fraction of the initial investment has been deployed). In order for the province to leverage value from its investment in a secure eHealth environment, issues regarding user adoption and security will need to be addressed. There is a need to incorporate the lessons learned from previous investment decisions into future plans.

Since cost information by product, service or client has not been maintained historically (nor has performance measures) it is not possible to assess the cost/benefit of past investments. It may also be too early to assess the value of these investments as the healthcare sector has been slower to adopt electronic information management than other sectors. Management recently identified several areas where performance must be improved to ensure control over operational, financial and client service practices. The Agency has also identified the need for substantial performance improvement in its core business of network and data centre operations. However, progress in these areas has been too slow.

The current financial management regime is inadequate to ensure the cost-effective execution of a $145 million expenditure plan for FY06/7. The primary concern is the lack of an effective, centralized, controllership function required to instill financial discipline, control and education across the Agency.

Financial management practices have been inadequate to manage the increasing size and complexity of the organization. We did not find evidence of inappropriate use of funds, although it was not part of our mandate to conduct a detailed review of financial controls. While recent improvements have been made, strengthening SSHA’s management and accountability will require the elevation of financial management on the corporate agenda. In response to these challenges, the Agency will be adding the position of Chief Financial Officer (CFO) to the executive team.

The Agency faces management, performance and organizational capacity issues which must be addressed in the short term. SSHA lacks the rigorous operational and financial planning required to function effectively. In October 2005, the Agency began to implement a new planning process and to develop its first integrated project plan. A resource planning and forecasting process was introduced in October 2005, but has not yet evolved to a consistent, repeatable, Agency-wide process.

Project management capability is insufficient given the complexity of the eHealth field. There is still no single, integrated project management methodology across the organization. The organization structure also needs to be strengthened. The current structure is unnecessarily complex and does not reflect leading practices for Information Services providers.

At the request of the Ministry, SSHA has also been dealing with a transition from a consultant-based to an employee-based organization. Because this transition took place in parallel with capacity building activities, SSHA had difficulty managing the significant degree of change and acquiring appropriately skilled staff. This has led to a skills gap in some areas such as business and financial management. Fundamentally, the culture of the organization needs to change to one that is customer focused, deliverable-oriented, and able to partner effectively with the Ministry and other health sector stakeholders.

SSHA does not have a clearly defined strategy. SSHA does not have an overarching documented strategy that defines the organization’s development over the next 3-5 years, including its strategic goals and objectives. The business plan, with divisional goals and output-based accomplishments, is a mechanism for obtaining annual funding, and cannot be used as a substitute for a comprehensive strategy.

Recently the SSHA Strategic Planning and Communications Committee, led by the Board Chair, submitted a short-term proposed strategy and plan to the Ministry. While this initiative is a good first step, the current status of this proposed strategy remains unclear as it is not formally approved by the Ministry. Therefore, the overall strategic direction for SSHA remains unclear. As such, maintaining focus will continue to be a challenge for the organization. Until a stable, clearly articulated eHealth strategy is defined and approved by the Ministry, SSHA will continue to face changes in its strategic direction and mandate which will impact operational and service performance. The relationships between strategy and governance documents, including gaps in documentation

Performance measurement is not linked to strategic outcomes. A performance agreement has not been documented between the Ministry and the Agency. SSHA’s current corporate balanced scorecard is not tied to its business plans (including financial and operational plans). To improve alignment, a new performance management initiative to develop a corporate balanced scorecard and associated divisional scorecards was launched in early 2006.

At the time of this report, this initiative is still in its early stages and its full impact will be difficult to assess until FY06/7 fiscal year end. In general, the Ministry, SSHA clients and other stakeholders are seeking more timely and consistent reporting, including qualitative and quantitative information on performance and the Agency’s contribution to overall health system performance.

Management decision-making processes have been inadequate to effectively steer the Agency. Strategic decisions regarding fundamental services, major scope changes or the addition of new projects are not well-documented. In the past, in-year changes in direction at short notice have created additional resource and funding pressure on SSHA.

In 2005, a new ‘funneling process’ was put in place to formally evaluate change requests or new project requests made by the Ministry or by SSHA. Each project is now required to go through feasibility and business case analysis before a go no-go decision can be made. All scope-change decisions must now be approved by the SSHA Board and the Ministry in writing. This is a very important, recent improvement in internal decision making, and reflects recognition on the part of the Agency that decision-making at the Ministry and the Agency has been ad hoc and uncoordinated, leaving stakeholders unclear as to the priority and rate of progress of various initiatives.

The recently implemented business and operational planning process (Q3 FY05/6), as well as the funnelling process are allowing SSHA executives and the Board to begin to make more informed decisions. These decisions must be ratified by the Board and submitted to the Ministry via the business plan document for approval. Variances from the business plan are reviewed on a monthly basis. Decisions regarding variances must also be approved by the Board and documented in the Board meeting minutes. Tactical decisions are made during regular weekly status meetings between project leads and the Executive Committee.

The Executive Change Management committee, supported by a director-level group facilitates the resolution of resource and scheduling conflicts between projects and divisions. Additional forums for decision making include a monthly, full-day SSHA Executive meeting for a comprehensive review of project and budget status; a quarterly, full-day SSHA Executive meeting to review the annual budget; and a monthly SSHA Board Meeting where key decisions are ratified.

With these mechanisms in place, the Agency will be better equipped to assess progress against approved plans and to stay focused on the achievement of formal commitments. In late 2005, formal meetings between SSHA management and the eHealth Office were instituted, which has resulted in more structured communications between SSHA and the Ministry.

The governance model has not encouraged the degree of partnering and collaboration required to deliver complex eHealth solutions. The principal governance relationship has been between the Ministry and SSHA management. The predominant focus is on budgeting, monitoring and control from a central planning perspective (e.g., eHealth Office).

At the same time, the Agency serves a number of clients within the Ministry (e.g., program owners) for whom solutions are being delivered by SSHA. These program owners often set the terms and conditions under which SSHA interfaces with the end-users of its services (e.g. health system providers). The result is that SSHA management is often faced with competing or contradictory demands from policy setters, program owners, funding managers, and users of its services. This reinforces a relationship between the Ministry and SSHA that is based on operational negotiation and control rather than collaboration.

It also leaves end-user customers unclear as to their role in priority setting. The Ministry, not having received the level of reporting it asked for from SSHA, has placed increasingly tight controls on the Agency, thereby inhibiting its operational independence. Administratively, the MOU between MOHLTC and SSHA is outdated and does not reflect the broader scope and accountabilities in the amended regulation. This leads to multiple interpretations of the mandate adding to confusion about SSHA’s role.

In 2005, the lack of a stable SSHA Board led to a governance gap that impacted decision-making over several months. As such, the FY06/7 multi-year business plan is not formally approved in writing by the newly instituted Board and Ministry. The introduction of a new Board for SSHA offers the opportunity to establish a more collaborative environment between the Agency and the Ministry. There is also a need to establish increased autonomy for the Agency to effectively build meaningful service relationships with the end-users of its eHealth solutions.

A clear accountability agreement with measurable targets and results does not exist between SSHA and the Ministry. Well-defined expectations, deliverables and measurable performance targets have not been established and documented by the Ministry for SSHA. For example, an annual accountability agreement with specific performance expectations has not been part of the governance model. This makes it difficult to assess the true performance of SSHA relative to expectations. As a result, the actual or perceived value of SSHA’s accomplishments against a $458 million investment over the last four years (including current year plan) is difficult to assess. Outcomes have also not been defined by the Ministry for the Agency, making it difficult for SSHA to understand the end-state of a project or initiative and work towards that state. The result has been a task-based approach to project delivery.

The Agency and the Ministry have different indicators and tools for measuring SSHA’s results. The new SSHA balanced scorecard has been strengthened over previous versions to include four categories – Client Perspective, Internal Perspective, Learning and Growth Perspective, and Stakeholder Perspective. Some basic metrics are in place. SSHA is in the process of researching other technical and financial metrics. Divisional scorecards are at the early stages of development in the eHealth Solutions Division and the CIO’s office. Many traditional metrics used for network and data centre management are not in place. The Ministry has also asked SSHA to report on four areas – Quality, Client Satisfaction, Pace of Delivery and Cost. The SSHA and Ministry performance measures and tools need to be aligned to a common set of agreed upon targets. A formal request for SSHA to present a performance scorecard to the Ontario e-Health Council in early 2005 has not been done to date.

The current legislation restricts SSHA from operating effectively as an arms length agency. SSHA operates like a division of government with its involvement in the RBP process, reliance on funding envelopes and working with holdbacks for in-year funds. Consequently, the Ministry (through the eHealth office) and central agencies are overseeing, reviewing and commenting on the financials and operations of SSHA at a detailed level.

By comparison, a transfer payment agency (such as Cancer Care Ontario or a major hospital) would operate with greater Board and management accountability for performance against approved strategies and plans. Such a structure could provide a number of benefits to SSHA including: the ability to carry-forward approved project funds into next fiscal year where legitimate issues had resulted in project delays; the potential to obtain revenue from clients for certain services where it has been proven economical to do so and performance targets are not compromised; and, the ability to enter into long-term, performance-based partnering relationships with industry or other stakeholders to leverage expertise and resources beyond those existing within SSHA.

Products and services lack clear, detailed definition. The lack of a detailed, standardized product and service catalogue leads to confusion about the scope and scale of SSHA’s offerings. Clients are unclear about the products and services provided by SSHA. In response, the Agency has started to develop detailed descriptions of the products and services it provides. Product descriptions for ONE Network and ONE Hosting are more clear and thorough than for ONE Mail, ONE ID, ONE Support and ONE Pages. The Agency has not yet progressed to a product and service catalogue with clearly stated service standards.

Client dissatisfaction with SSHA service levels is high. SLAs are not consistently defined or implemented with all clients. As a result, customized agreements are in place for most clients. Standardized client reporting against SLAs is minimal to non-existent. Management dashboard tracking of service level targets for incident resolution has only been in place for a few months (e.g. IS Enterprise Services Reports).

Service level performance is currently only managed for high-priority incidents. While some clients were appreciative of the efforts and service quality provided by individual SSHA employees, overall feedback from clients on service levels has been highly critical. Clients must deal with multiple SSHA or vendor contacts for service or technical issues. Clients find this confusing and are unsure of the escalation path for their complaints. SSHA has recently established a client service manager role to mitigate such issues.

Network and data centre costs can be more tightly managed. Network costs account for 22%of the FY06/7 operating budget. The budget for network operations has been increasing by 10% year over-year. The Agency is establishing a new network contract (currently in negotiations) that will become operational in 2007, with the objective of containing the growth of network costs. The two data centres constitute 20% of the FY06/7 budget, with facility rental expenses being the most significant contributor to data centre costs. The current proliferation of racks in the data centre may result in escalating facilities costs if the Agency does not take measures (e.g. server consolidation) to reduce the existing rate of floor space consumption.

SSHA created the eHealth Solutions Division (eHSD) to assist the broader healthcare community in developing and deploying applications. This division recently has taken on the responsibility of client service management. The division is comprised of dedicated project resources and resources that spend a portion of their time on operational and maintenance activities. SSHA is in the early stages of developing client reports but has not developed a definitive approach to such reporting. User groups or client councils have not been part of SSHA’s approach to managing client and stakeholder relationships.

Users’ expectations of SSHA often do not align with its actual mandate and capabilities. With one or two exceptions, external stakeholders were unable to provide a reasonably accurate list of SSHA’s products and services. Interviewees consistently stated their belief that the products and services they expect from SSHA are vital to delivering on their government-mandated responsibilities and objectives. These objectives include such things as: improved access to healthcare, progress against the provincial wait time strategy, improved health outcomes, and cost containment.

With the constant push by government to create greater integration across the Ontario healthcare system, stakeholders view SSHA as a key enabler for their specific technology needs. There is broad support for a provincial eHealth agency to promote integration and economies of scale. At the same time, stakeholders are frustrated by what they view to be slow progress with eHealth in Ontario. The lack of alignment between external stakeholders, the Ministry and SSHA has been exacerbated by performance issues and by SSHA’s limited engagement of stakeholders in processes that define Ministry and SSHA priorities and solutions.

As a result SSHA’s products and services are perceived to be ineffective. This has resulted in some client organizations moving ahead with their own technology solutions, security protocols and eMail systems. In time these standalone client projects may have serious implications for the overall eHealth environment and SSHA’s future role.

There is a perception that quality of products and services delivered by SSHA is poor. Stakeholders generally expressed dissatisfaction with the quality of SSHA’s products and services. They find varying degrees of quality and consistency in SSHA’s provision of basic infrastructure needs, network and connectivity services, as well as secure eMail. Numerous examples were provided relating to the lack of system availability, long turn-around times on requests for service, lack of bandwidth and lack of warnings or communications during system down-time. Other frustrations include:

• While SSHA employees are perceived to be technically proficient, they demonstrate little business acumen. This translates into an inability to understand client issues, manage client expectations and describe SSHA products and services in a meaningful way.

• SSHA is perceived as rigid and inflexible by many clients.

• The inability of SSHA to describe how SSHA measures and reports on its performance.

• Most users do not have Service Level Agreements, mainly because the process to agree on SLAs is too complex and long and where SLAs are in place, no feedback/reporting on it is done.

• SSHA’s communication and general engagement with stakeholders is insufficient for them to feel satisfied that the Agency values client relationships.

Stakeholders recognize that some of SSHA’s operational problems can be explained by the lack of maturity within the organization. They also realize that the lack of a clear mandate contributes to SSHA’s difficulties with operational planning and that SSHA’s execution of activities is shaped by the Government’s policies and priorities. However, on balance, our consultations with external stakeholders highlighted the widely held perceptions that SSHA:

• Offers products and services that are relatively ineffective;

• Delivers products and services inefficiently; and

• Does not measurably contribute to the efficiency of the healthcare system.

This has led to an erosion of confidence. Restoring credibility requires a concerted effort by the Agency.


Recommendations

To address the increased importance of the Agency in the future Ontario eHealth agenda, a number of recommendations for improvement and an associated implementation plan have been developed. Given the increased urgency for substantive progress with eHealth in Ontario, an aggressive, Agency-turnaround plan is required. As such, the high-level implementation plan included in this Report calls for the fundamentals to be implemented by March 31, 2007. Fiscal year 07/8 will also be required to fully implement the required improvements. The Ministry needs to own the eHealth program and be engaged at the level of making strategy, policy, and program decisions, and work with the Agency to implement appropriate solutions.

 

HOME - CURRENT ISSUE - ABOUT US - SUBSCRIBE - ADVERTISE - ARCHIVES - CONTACT US - EVENTS - LINKS