Overview
Oracle Clusterware 11g release 2 laid the foundation of policy-based cluster management by introducing server pools, which are logical groups of servers hosting the resources managed by Oracle Clusterware. Examples of resources that Oracle Clusterware manages are database instances, database services, application VIPs, and application components. Originally, member servers of server pools were restricted by a set of basic attributes characterizing servers. There was no way to distinguish between types of servers; all servers were considered to be equal in relation to their processors, physical memory, and other characteristics. This could lead to sub-optimal performance of the applications assigned to the server pools whose member servers did not meet the applications' requirements.
Oracle Grid Infrastructure 12c enhances the use of server pools by introducing server attributes e.g. MEMORY, CPU_COUNT etc. which are associated with each server. Server pools can be configured to have members belonging to a category of servers which share a particular set of attributes. Moreover, to reallocate servers to various server pools based on workload, the administrators can maintain a library of policies and switch between them as required rather than doing it manually.
This article discusses in detail the new features of policy based cluster management in 12c.
Introduction
There are two styles of configuring resource management in a cluster:
- Administrator-managed configuration, the only management strategy available in previous releases in which the administrators explicitly assign various Clusterware resource(s) to specific servers in the cluster. The hard coupling between the resources and the servers eliminates resource sharing. There are no policy decisions; all servers are considered equal and the workload-to-server assignment is static and manually defined. This style is best suited for smaller clusters or rather static systems due to the manual placement requirements and cannot scale very well in large clusters having more than 8 nodes.
- Policy-managed configuration, introduced in Oracle Clusterware 11g release 2, allows the cluster to be logically partitioned into groups of servers called server pools. The administrators assign various Clusterware resource(s) to server pools rather than the servers in the cluster. Oracle Clusterware is responsible for placing the resource(s) on the server(s) belonging to the specified server pool(s). It eliminates the need for static definitions physically assigning resources to particular nodes in the cluster. The scalability of resources is controlled by the MIN_SIZE and MAX_SIZE properties of the server pool. Moreover, different IMPORTANCE can be assigned to server pools, thereby allowing mapping of critical workloads to server pools of higher IMPORTANCE. Based on available compute resources in the cluster, Oracle Clusterware will allocate servers to server pools in order of their IMPORTANCE so that business critical workloads assigned to the server pool of greater IMPORTANCE will get the compute resources necessary to satisfy performance or availability service level agreements. Since this style of cluster management is policy based, it is best suited for bigger clusters and scales very well in large clusters having more than 8 nodes.
Considerations:
– A particular server is always part of one server pool at any point in time
– A particular resource can run on more than one server (pool)
– More than one resource can run on one particular server (pool)
– A cluster can host more than one databases
– There can be only one instance of a particular RAC database on a specific server at any point in time
– Database services come as “singletons” or “uniform”. They run either on one server or on all servers in a specific server pool
– A particular service is bound to the database it runs on
– One Service can be assigned to only one server pool.
Thus, the server pool definitions effectively define a policy which dictates placement and prioritization of servers. Since the control is based on policies, this style is workload-management compliant and best for a large number of nodes. Policy-managed cluster management enables:
- Fast resource failover – when the number of nodes in the cluster changes, servers are reallocated online based on defined policy to satisfy workload capacity requirements.
- Dynamic capacity assignment – Server pools configuration can be changed in accordance with business needs or application demand, so that pools provide the required capacity at the right time. Policy management allows clusters and databases to expand or shrink as requirements change.
- Guaranteedallocation of required resourcesfor critical work as defined by the policy.
- Isolation of resources – Since server pools do not share resources, they isolate resources where necessary. Applications running in one pool cannot access resources of another pool, so that dedicated servers can be provided in a cluster for applications and databases.
- Useful for future planning – Now, database administrators can define resource requirements for expected workload. As a result, additional capacity, whenever available, will be used instantaneously in accordance with the policies defined. This is useful for future projects and when planning ahead.
Limitations of policy-based management in Oracle Clusterware11g R2:
In versions earlier than Oracle Clusterware 12c, the assignment of servers to server pools was based on the relative IMPORTANCE and the minimum and maximum number of servers associated with the server pool. Because there was no way to distinguish between types of servers, all servers were assumed to be homogeneous with regard to their CPU count, physical memory, and other characteristics. Hence, placement of servers in server pools could not be governed by server attributes. To prevent sub-optimal performance of applications resulting from their execution on servers with inadequate attributes, administrator needed to manually map workload to servers of differing compute power or other specific attributes accordingly. Besides being a burden to the administrator, such manual resource management is error prone when considering capacity requirements in the event of outages.
Moreover, although the server pools can be configured such that each of the applications is assigned to run in its own server pool, this configuration does not consider the fact that server requirements of applications may be different at different times of the day, week, or month. Reporting applications, for example, typically use more resources on weekends and month ends. To meet such requirements is cumbersome because changes must be made to the server pool configuration manually.
Policy-Based Cluster Management In 12c
With Oracle Clusterware 12c, policy based cluster management is enhanced in three important ways:
- Server Categorization – Allows assignment of servers to different server pools based on their physical attributes.
- Extended Policy framework – Administrators can maintain a library of policy definitions (configurations) for server pools and dynamically switch between them as required.
- Unification of policy based cluster management with QoS Management – Policies for Oracle Clusterware and “Quality of Service Management” have been unified so as to eliminate potential for overlap, confusion, and inconsistency
This article discusses the first two new features.
Server Categorization
With Oracle Clusterware 12c, the notion of server categorization is introduced to support clusters with heterogeneous servers. Categorization allows servers to be differentiated based on their attributes which can further govern placement of servers in the server pools.
Server categorization works as follows:
- Each server is associated with a set of server configuration attributes which are automatically discovered internally on startup of the Oracle Clusterware stack on a particular server and stored persistently until the stack on the server is restarted.
These attributes are:
Using these server attributes, Oracle Grid Infrastructure 12c understands heterogeneous servers in a cluster by storing server attributes on a per server basis.
Example: Consider a cluster having 3 hub nodes and 2 leaf nodes.
– View extended attributes of servers in the cluster. Notice that:
- Servers host01, host02 and host03 are Hub Nodes in the cluster and physical memory associated with them is greater than 2000 MB
- Servers host04 and host05 are Leaf Nodes in the cluster and physical memory associated with them is less than 2000 MB
- A new Clusterware object defines server categories, which enables you to organize servers into particular categories. Two internal categories are created by default – ora.hub.category and ora.leaf.category, used to categorize Hub and Leaf Nodes respectively.
Example:
– List categories created by default
– List servers in the two default categories
New user-defined categories can be created. Various server category attributes are:
Note that a server can belong to multiple categories at the same time. If you change the value of any of the server attributes listed in the EXPRESSION attribute of a server category, then you must restart the Oracle Clusterware technology stack on the affected servers before the new values take effect.
Example:
– Create new categories
Small: Leaf Node and Memory <2000
Big: Hub Node and Memory >= 2000
– View new categories
– List servers in new categories
- Server categories are applied to server pools by adding the new SERVER_CATEGORY attribute to each server pool definition. This attribute allows a server category to be associated with each server pool, so that server pools are defined based on attributes rather than the sole number or name of a server that will be part of the pool. Server attributes such as number of CPUs, CPU speed, Memory, etc. are evaluated at the time a server joins the cluster. This allows for the definition of entry-level requirements for a server to join a server pool and thereby can be used provide an automated and efficient way to manage environments with varying workload requirements and with servers of varying capacities. Applications can be assigned to the server pools so that they will run on all the servers assigned to the respective pool at any time.
Server categories ensure that server assignments to server pools will meet the required minimums. If a server outage arises, criticality of the workload will be determined using IMPORTANCE of the associated server pool and predictable policies will be applied to re-allocate the resources in the cluster accordingly. If the remaining cluster size is insufficient to host the required workload, the same policies will be used to potentially move compute resources of a server pool of lesser IMPORTANCE to the server pool of higher IMPORTANCE in order to satisfy the compute resource requirements of business-critical workloads.
Example:
– Create server pools and associate them with big / small categories as follows:
|
Server pool |
Category |
|
Bigpool |
Big |
|
Smallpool |
Small |
|
Testpool |
Small |
|
Backuppool |
Big |
– View server pools and associated categories:
Extended Policy Framework
Policy-managed environment in Oracle Clusterware 12c provides the infrastructure to automatically handle fluctuating server time demands of various applications in the cluster. It offers the capability to create various policies (server pool configurations) in accordance with business needs or application demand, so that pools provide the right service at the right time.
Typically, administrators create multiple policies to reflect different server time requirements based on business needs or demand, or based on calendar dates or times of the day. A collection of zero or more such user-defined policies is called a policy set. Each policy contains exactly one definition for each server pool controlled by the policy set. There is only one policy set defined for the cluster. These policies can be activated at will such that at any point in time only one policy is in effect for the cluster. Oracle Clusterware manages the server pools according to the active policy in the policy set. With a cluster configuration policy set, for example, more servers can be allocated to an OLTP workload during weekly business, whereas on the weekends and evenings, batch workloads can be assigned more servers, while performing transitions of server pool configuration automatically. The ability to change server pool properties as a group permits Clusterware to optimize the transition to be minimally disruptive, which would be very hard to do by issuing one command at a time.
To understand how to create and use a policy set and its included policies, consider the following case where we have four server pools: bigpool, smallpool, testpool and backuppool. Appropriate services can be registered to the server pools at any time. As soon as a server is assigned to a pool, the corresponding services will automatically start. Our requirement is as follows:
During daytime:
– Small application(s) will run on 2 small servers (smallpool)
– Big application(s) will run on 3 big servers (bigpool)
During nighttime:
– Small application(s) will run on 1 small server (smallpool)
– Big application(s) will run on 2 big servers (bigpool)
– Test applications will run on 1 small server (testpool)
– Backup service will run on 1 big server (backuppool)
Example:
– Add Day and Night Policies
– Set the SERVER_POOL_NAMES policy set attribute to define the scope of the server pools that are controlled by the policy set.
– Set the attributes for the server pools in Day policy.
The Day policy
- Enables small applications assigned to run in smallpool to run on 2 small servers
- Enables big applications assigned to run in bigpool to run on 3 big servers
- Causes test applications and backup service not to run at all
– Set the attributes for the server pools in Night policy.
The night policy enables
- Small applications assigned to smallpool to run on 1 small server
- Big applications assigned to bigpool to run on 2 big servers
- Test applications assigned to testpool to run on 1 small server
- Backup service assigned to backuppool to run on 1 big server
– View Day and Night Policies
– Policy Activation: The policy set is now configured and controlling the four server pools with two different policies. You can activate policies when necessary, prompting Oracle Clusterware to reconfigure a server pool according to each policy’s configuration. Let us activate the Day policy
– Verify that as per the Day policy,
- Smallpool has been assigned both the small nodes (host04,host05)
- Bigpool has been assigned three big nodes (host01,host02,host03)
- Testpool and Backuppool have not been assigned any servers
– Activate Night policy.
– Verify that as per the Night policy
- Smallpool has been assigned one small node (host05)
- Bigpool has been assigned two big nodes (host02,host03)
- Testpool has been assigned one small node (host04)
- Backuppool has been assigned one big node (host01)
Note that, because the server pools have different sizes, as servers move between server pools, some applications will be stopped and others will be started
– Modify the configuration of night policy so that backup runs on 2 big servers and big applications run on 1 big server. Reactivate the night policy
– Verify that as per the modified night policy, one big server (host02) has been taken away from bigpool and has been assigned to backuppool so that bigpool is left with only one server and backuppool has 2 servers as desired.
Thus, using the crsctl modify policyset command, Oracle Clusterware changed server pool configuration, moved servers according to the requirements of the policy, and stopped and started the applications.
Summary
- Oracle Clusterware 11g release 2 (11.2) laid the foundation of policy-based cluster management by introducing server pools, which are logical groups of servers hosting the resources managed by Oracle Clusterware.
- Policy-based management in 11g release 2 had following limitations:
– The assignment of servers to server pools was based on the relative IMPORTANCE and the minimum and maximum number of servers associated with the server pool. As there was no way to distinguish between types of servers, placement of servers in server pools could not be governed by server attributes.
– Although the server pools could be configured such that each of the applications was assigned to run in its own server pool, this configuration, however, did not consider the fact that server requirements of applications may be different at different times of the day, week, or month. Though server pool properties could be changed to meet such requirements, it was very cumbersome to do by issuing one command at a time.
- With Oracle Clusterware 12c, policy based cluster management is enhanced by introduction of:
– Server Categorization – Servers can be differentiated based on their attributes, which can further govern placement of servers in the server pools.
– Extended Policy framework – Policies can be configured to change server pool attributes in accordance with business needs or application demand, so that pools provide the right service at the right time. The administrators can maintain a library of policies and switch between them as required.
References:
Start the discussion at forums.toadworld.com