In the case of a business or of a non-profit organization, governance relates to consistent management, cohesive policies, processes and decision-rights for a given area of responsibility. Governance includes roles, rights and responsibilities of those providing governance and those being governed, as well as the explicit policies, procedures, metrics, rewards, and sanctions required to ensure people and systems perform within defined parameters to meet business expectations. Determining the least amount of governance an organization needs to invest in and still succeed depends on what they wish to accomplish. The goal of this article is to examine the kind of governance efforts needed to ensure success, and to help determine how decide how much governance is enough. Current state of SOA governance According to Farquharson, "The first cause of failure is typically exposing everything as a service. Companies make a large initial investment, end up with hundreds of services, and cannot measure any business value at all." The second wave of failure "comes after organizations try to solve the issue with infrastructure alone, such as an ESB." Farquharson claims that the term "Enterprise Service Bus is a misnomer, and most are deployed at the departmental level than enterprise level solutions." Developers from one platform cannot leverage another platform technology as their ESB due to the specialized skill set required. When organizations take a technology approach to governance, they may well end up with what Tony Baer has called "islands of governance."

There are different tools for directory/repositories and run time monitoring and management. There are separate repositories for asset management and security permissions. It is pretty much a foregone conclusion that just about all large organizations have multiple platforms, tools, and languages. Clearly, while technology can help automate policies, the real work of governance is defining the policies, processes, roles and responsibilities. The real key to governance is "getting the organizational and people aspects right, obtaining stakeholder buy-in, and defining decision rights, then identifying the technology necessary to support service and policy development, deployment and enforcement."  The SOA adoption curve spread out over eight years. Establishing governance within the organization should start with education and collaboration. That is the key to changing the cultural aspects of siloed command and control to agile collaboration and the ability to consume and reuse services. This is a totally different mindset. The second step in governance adoption is policy definition and policy enforcement. The third stage is formalization of the consumer-provider relationship. The real key to success is knowing what it is you are trying to do, and who is going to be responsible for enforcing it. The biggest mistake an organization can make is trying to "boil the ocean, plan everything up front, move it all into one repository or registry. This tends to bog down the organization, and not deliver an ROI."  While too much governance can stifle innovation, too little governance leaves an organization with no way to monitor or measure success. If you can't monitor or measure success, increased risk of failure is an inevitable result. So how much governance is just enough? Well, that all depends on what you are trying to do.

Rightsizing governance

There are a number of ways to categorize SOA implementations and levels of governance. One categorization that seemed to come up a lot was level of maturity in terms of SOA adoption. Many organizations are still in the early stages and just want a way to catalog their services. Some are in the wild-west stage. They've implemented a number of tactical services and are now looking for a way to rein it in by discovering what is out there and starting to implement some controls. Some organizations have mature implementations and are seeking ways to manage federated environments. While this may be a useful way of looking at customers and what they may be ready to buy, from an organizational point of view, in order to ensure success which is, after all, the purpose of governance, it is preferable to first define explicitly what you are trying to achieve, and then determine the appropriate level of governance required to ensure success. At a minimum, a governance framework should include policies, processes, roles, responsibilities, metrics and implementation plan for each type of service, at each stage of the life cycle. The number and scope of policies, processes, stakeholders and implementation infrastructure may vary by maturity level. Governance is the process of measuring, monitoring, and managing for success. In the long run, we only hit what we aim at.

The bottom line is that SOA without proper governance is a crap shoot. When determining how much to invest in governance efforts organizations should assess the cost of failure. Life cycle stages The vendors report that their customers most often start their governance efforts either with a registry/repository or run-time monitoring and management system. They reported that organizations with more mature SOA implementations or those who already have IT governance efforts in place may practice governance throughout the full life cycle. To ensure success it is critical to ensure the original business objectives are met. This requires full life cycle governance, including inception, design, provisioning, maintenance phases. While it is possible to scale back governance activities in each stage, skipping an entire phase means you cannot ensure the success of that life cycle phase. Project/service initiation governance Organizations that are implementing SOA in order to achieve better IT and business alignment, a much touted benefit, need to define governance processes, procedures, roles and responsibilities at the very beginning of each project life cycle. Before an organization can get value out of a service or a program, they need to be sure they are investing in the right initiative. Most organizations have processes in place to determine which IT projects to invest in. A number of organizations are adopting portfolio management techniques, tools and processes, such as COBIT or ITIL and CMDBs to manage IT assets. These organizations are focusing on gaining value out of IT investments. Such organizations are also starting to expand these efforts to SOA initiatives, and manage the services as part of the overall IT portfolio.

SOA governance is a subset of IT governance. Even if different tools, processes, and stakeholders are identified for the SOA effort, at the very least at the inception stage you should be able to capture the business metrics that will be used to measure the overall success of the service.  Success factor is picking the right scope. The more focused project you have to begin with, the more likely it will succeed. It's best if the project is directly tied to some critical or new business initiative. Policies must be traceable back to a business objective and if it isn't, you shouldn't implement it. If other IT governance activities define the sponsor, key business goals/metrics, then the SOA governance process should capture/integrate/inherit that information from other processes or systems. At a minimum, the SOA solution/service initiation governance framework should include key stakeholders, including executive sponsorship, key business goals/metrics, and any relevant policies and procedures from the IT governance framework. Down the road, organizations may implement registry integration/federation so information is easily shared across different governance activities, as well as automated dashboards for key stakeholders to monitor key business metrics and enable business improvement. Design governance Many organizations start their SOA governance program at the design stage by implementing a registry/repository to catalog the services. Organizations seeking to implement minimal governance may focus on naming and interoperability standards such as WSI. While this is certainly where many SOA governance efforts begin, it may not be sufficient to enable reuse and cost savings in the future. Reuse requires trust. This has a number of implications. Services listed in the repository need to be tested and certified, and the contract between providers and consumers need to be specified, and managed as part of the implementation. This may require integration between testing suites and run-time management tools, as well as security registries and management tools. SOA design enables agility because it provides loose coupling, thereby minimizing the impact of change. However, doing so requires understanding how services are used by different business solutions. This information can be captured as metadata in the repository. There are also runtime management tools that capture this information and put it in the repository.

Organizations in the wild-west stage of SOA implementation, which have deployed many services with little to no governance, are starting by monitoring the run-time environment, and feeding the information into a repository to determine how services are being used. Over time, change is inevitable. The ability to perform impact analysis should be implemented before any major changes are made to shared services. At a minimum design phase policies should include interface design, security, performance and availability. Different stakeholders may be responsible for different types of policies. For example, a Chief Security Officer may be responsible for security policies. Policies may also be defined at a different level of the organization. For example, in a federated architecture there may be enterprise level policies, departmental level policies, and even external compliance policies. It is important to define who is responsible for which policies, and the processes for ensuring compliance. Provisioning governance Provisioning a service can include custom building it, outsourcing development, purchasing it, reusing it, or provisioning it as a cloud service. The policies regarding provisioning may include Enterprise Architecture standards, or it may include policies for outsourcing development. Metrics and policies for testing and certification are included. Some of the processes and metrics for project management will incorporate service development as well. Most organizations are still in the very early stages of establishing SOA development methodologies. The ability to easily instrument services for governance is a big plus. According to Dhruv Gupta developers do not want to have to change how the work and resent overhead processes that slow them down. You have a much better chance of success if you make it part of the development environment, for example as an Eclipse plug-in. Success depends on implementing enough governance to ensure success, but not too much it interferes with getting projects done. Runtime governance A number of vendors offer the ability to monitor services in run time to see how they are actually used, ensure SLAs are met, and provide alerts when needed. Typically, when organizations have more than 50 services, or even a few business critical services, they find a need to automate the monitoring and management of policies and processes. Doing so manually in a distributed heterogeneous environment is difficult to impossible. Automated monitoring and management facilities can also provide a feedback loop to improving services and processes, as well as autonomic governance, where the environment self monitors and self corrects by initiating automated processes.

Organizations that are providing services for external entities, or which face severe regulatory or SLA compliance penalties, may find it cost effective to invest in automated runtime management tools. These tools generally integrate will the SOA repository tools, and provide run time information which can be used to determine how services are performing and how they are used. This feedback loop can help organizations optimize services, and determine the ROI of service investments. Many organizations are realizing that automating runtime governance is key to a successful SOA implementation. But it is essential to monitor and manage the right metrics. While typical IT performance metrics remain relevant, business metrics, including the ones defined during the project initiation, are key. Automated runtime governance tools can report on how a service is delivering a business ROI as long as the right metrics are defined and the policies to monitor them are implemented. IT organizations should and must define their success in terms of meeting business objectives. The tools can help do that, but more important are the roles, responsibilities, and policies defined to make that happen. Rightsizing scope The most important thing to first understand in order to rightsize your governance effort is scope. SOA initiatives may be ad hoc, departmental or they may be truly enterprise scale. One size does not fit all. Furthermore, a service can be composed of one or more services. Even an entire business process can be wrapped as a service. There are different levels of services, and the different levels of services have different owners, policies, and governance requirements.

Most large organizations will end up with federated SOA environments. When determining the right level of governance or any given initiative or service, it is essential to define scope and levels of governance. The key to any successful governance effort is to pick your battles. Enterprise level policies and processes need to be monitored, managed and measured at the enterprise level. These are the policies and services that will require the highest level of governance, because they have the most wide-ranging impact. At the enterprise level, at a minimum, you need to monitor and measure the business metrics identified for determining the success of the initiative in order to determine the benefit this investment has provided to the enterprise.

Departmental solutions must align their services with enterprise policies and procedures. Even IT level services, designed to make IT more cost effective and agile, can and should be linked back to business initiatives. Identifying governors One of the most important parts of defining a right sized governance effort is to define the roles and responsibilities of all the stakeholders. In fact, one customer modeled it after the government by defining executive, judicial, and legislative roles, policies and procedures. The legislative branch makes the policies, the executive branch implements it and the judicial branch enforces it. In a distributed, federated environment, where services may be composed of other services, it is important to realize there may be different people responsible for different types of policies. For example, a Chief Security Officer may be responsible for defining security policies. A business executive may be responsible for defining the business policies and SLAs governing the service. An enterprise architect may be responsible for defining interface and platform standards. All of these different policies may apply to a single service, so there may be multiple governors involved.

Success requires making roles, responsibilities, policies, and procedures explicit. Organizations that start with small, bottom-up SOA efforts, typically also start with a minimum amount of governance. However, once the services begin to be reused across the organization issues start to emerge. Who can access the service? Who can change it? Who mediates disputes? In the absence of a clearly defined roles and responsibilities vendors offer the advice to ensure executive sponsorship. Make sure you have someone on board with enough authority to mediate disputes. However, over time, organizations will need to explicitly define the roles and responsibilities vis-a-vis service and SOA governance. Best practices While governance is the key to SOA success, when beginning an SOA initiative, it is important to realize that too much governance too soon can stifle innovation and hurt the initiative. Communicate early and often, and collaborate with other teams. The vendors reported that customers who try to legislate with a stick were less successful than those who took the time to communicate the benefits of the new approach. Success requires a change in culture and the way business solutions are developed and implemented. Lack of knowledge and experience is cited as contributing to SOA failures. Everyone recommends starting small and growing as you gain knowledge and experience. Therefore, it is beneficial if your governance tools, including registries/repositories and runtime monitoring and enforcement of policies, can grow with you. Some vendors take the enterprise level approach to their tools. Some give you the option to start small and grow the infrastructure over time with your efforts. However, even if you are starting small, be sure to include all phases of the life cycle. As most organizations have multiple registries and repositories -- for security, for testing, for asset management, for different areas of the organization.

The ability to integrate and federate repositories is also important to avoid islands of governance. Most of the vendors I spoke support registry federation. While many SOA implementations begin from the bottom up, and start with minimal governance, ultimately business success requires the formalization of roles, responsibilities, and expectations of providers and consumers. It is difficult to claim success if the business and IT organizations do not agree what success is. It is also difficult to achieve success unless everyone plays their role. Who can define policies? Who can change them? Who ensures they are enforced? Organizations are implementing SOA solutions because business processes and practices move from command and control structures to more distributed, cooperative federated processes. While the systems can be designed to automate the processes, the organizational structures must also be defined to govern them appropriately. Once again, governance should not be a burden. It should be implemented to ensure success. How much governance is enough depends on what you are trying to achieve. However, achieving success at all is unlikely without appropriate governance.