Cluster Services

From CoolSolutionsWiki

Contents

Summary

Novell Cluster Services is a server clustering system that ensures high availability and manageability of critical network resources including data (file systems), applications, and services. It is a multinode clustering product for Linux that is LDAP enabled via Novell eDirectory and supports failover, failback, and migration (load balancing) of individually managed cluster resources.

External Links

Support Forum HTTP News

Connection Magazine: Tech Talk #3 - Over and Over and Over and Over This is an excellent article about NCS. It covers some of the issues with running mixed NetWare/Linux NCS cluster.

Connection Magazine: Tech Talk #5 - Backup Backup A good overview of NCS. Has a section on clustering GroupWise.

Linux OES Product Documentation: OES Cluster Services 1.8 Administration Guide for Linux

NetWare OES Product Documentation: OES Cluster Services 1.8 Administration Guide for NetWare

NetWare OES Product Documentation: Novell Cluster Services 1.8 Resource Configuration Guide

NetWare 6.5 Product Documentation: Novell Cluster Services

NetWare 6 Product Documentation: Novell Cluster Services

NetWare 5.1 Product Documentation: Novell Cluster Services

AppNote: An Introduction to Novell Cluster Services

AppNote: How to Implement a 2 Node Cluster without External Shared Storage

Resources

Virtually any network service can be a cluster resource. There are a few requirements that a service must meet before being able to be clustered.

1) Relocatable data. The data files that the service uses must be able to be moved to shared storage. On Linux it is usually possible to work around services that don't allow this by using soft links.

2) Power off test. A service must be able to pass the "power off test." A service must be able to start back up successfully after a power outage. If in the case of a power outage the service has corrupt files that prevent it from running, then that service isn't a good candidate for a cluster.

NCS cluster consist primarily of a load and an unload script file. Almost all load script files do the same three things. 1) bind a secondary IP address 2) mount a file system on the shared storage 3) start the service. The unload script of course undoes each of these steps in the reverse order. The unload scripts usually 1) stop the service 2) unmount the file system 3) unbind the IP address.

Below are are few of the most common resource type being created on Novell Cluster Services.

GroupWise

Novell Documentation: GroupWise 8 and Novell Cluster Services

Novell Documentation: GroupWise 7 and Novell Cluster Services

Novell Documentation: GroupWise 6.5 and Novell Cluster Services

Novell Documentation: GroupWise 6 Introduction to GroupWise 6 and Novell Cluster Services

Connection Magazine: Tech Talk #6 - Nuts About Clusters Article about GroupWise High Availability Service.

Cool Solution: Novell Cluster Services Primer for GroupWise Administrators

ZENworks

Novell Documentation: Installing and configuring Novell® ZENworks® 7 Desktop Management components in a Novell Cluster Services

Novell Documentation: ZENworks 6.5 Introduction to Novell Cluster Services and ZENworks Server Management

Novell Documentation: ZENworks for Servers 3.0.2 Introduction to Novell Cluster Services and ZENworks for Servers

Novell Documentation: ZENworks for Servers 3 Introduction to Novell Cluster Services and ZENworks for Servers

AppNote: Cluster-Enabling a ZENworks for Servers 2 Deployment

IPrint

Novell Documentation: How to configure iPrint in a Cluster

AppNote: Installing NDPS on Novell Cluster Services

DNS/DHCP/SLP

AppNote: Cluster-Enabling DHCP, DNS, and SLP Services in NetWare 6

ISCSI

AppNote: Installation and Configuration of 2 Node iSCSI Based Cluster Using NetWare 6.5 SP1

AppNote: Installation and Configuration of 2 Node iSCSI Based Cluster Using NetWare 6.5 SP1

AppNote: iFolder on Open Enterprise Server Linux Cluster using iSCSI

AppNote: How to Implement a 2 Node Cluster without External Shared Storage

TID: 10096996 - What To Know About Tuning Clusters and iSCSI

WIKI Slow iSCSI check list

iSCSI and SLP

Use of iSCSI implies TCP/IP and dual-homed servers. If you properly architect your iSCSI network, you'll have it completely segregated and inaccessible by your client workstations. However, blocking port 524 traffic via ACL, or hiding the network via routing rules is not enough.

NetWare will still present all of its addresses via SLP by default, even if some of them are unreachable. This can cause lengthy delays in service location, drive mappings, etc. To avoid problems with name resolution and login performance, be sure to use the NCP EXCLUDE IP ADDRESS and SLP EXCLUDE IP ADDRESS parameters. Populate them with the IP addresses of the iSCSI network interfaces, to keep clients and other servers on your LAN/WAN from trying to communicate with them on the network you've designated for iSCSI traffic only.

iSCSI and Jumbo Frames

Using at least NetWare 6.5 SP5, performance can be slightly improved in some cases by enabling jumbo ethernet frames. This setting allows a device to increase the payload of each packet from roughly 1,500 bytes to roughly 9,000 bytes, making communications on your iSCSI network more efficient.

Most implementations involving separate "NAS" networks benefit from use of jumbo frames, especially those using NetApp Filers. Linux/Unix hosts, etc. notice a marked performance improvement when using these larger ethernet frames - configuring NetWare to use Jumbo frames in such an environment ensures that it's following best practices.

To do so, you cannot use INETCFG to manage the configuration of your NetWare server's ethernet cards. Manually configuring the cards is beneficial, as it allows you to specify which is loaded first with better predictability. In addition, it allows you to use parameters that are not possible through the INETCFG utility.

Keep in mind that you must have switch hardware capable of supporting jumbo frames, and those switches must be configured properly to do so. For example, the Cisco 3750 stackable switches, and the IBM/Cisco BladeCenter Integrated Gigabit Switch Module (CIGESM), are known to work well in production environments if running the latest versions of IOS for their respective platforms. Also, depending on your server hardware, use of multiple-processors (SMP) may cause stability issues with iSCSI and jumbo frames - be sure that you're always using the latest non-beta support pack, WINSOCK, iSCSI, and network driver files.

Here's the minimum changes necessary to support jumbo ethernet frames on NetWare, as used in a production environment:

  • STARTUP.NCF Settings
    • SET MAXIMUM PHYSICAL RECEIVE PACKET SIZE=16400
  • AUTOEXEC.NCF Settings
    • SET TCP PATH MTU BLACK HOLE DETECTION AND RECOVERY = ON
    • SET TCP MINSHALL ALGORITHM = OFF
    • SET USE SPECIFIED MTU = OFF
  • Loading the driver for your NAS interface - Using Broadcom b57.lan driver as an example
    • LOAD B57.LAN SLOT=100nn FRAME=ETHERNET_II POLL=1 NAME=NAS JUMBO=9000

Updated iSCSI and Winsock Drivers

As of this time, WSOCK6i.EXE is an unreleased yet eagerly awaited component that should improve stability and performance in these configurations


iSCSI and Multiple Bridges for Xen Virtualization

Creating multiple bridges to allow two virtual network cards to communicate with two separate networks

1. Install SLES w/ Xen Virtual Machine Host Server

2. Go to /etc/xen/xend-config.sxp and comment out the line “(network-script network-bridge)”. Add the line (network-script two-bridges)

                #(network-script network-bridge)
                (network-script two-bridges)

3. In /etc/xen/scripts create file two-bridges with following code:

                #!/bin/bash
                /etc/xen/scripts/network-bridge $1 netdev=eth0 vifnum=0 bridge=xenbr0
                /etc/xen/scripts/network-bridge $1 netdev=eth1 vifnum=1 bridge=xenbr1

4. Make two-bridges executable

                chmod +x two-bridges


5. Reboot into Xen or run “network-bridge stop” and “two-bridges start”

6. ifconfig should now display peth0/1, vif0.0/0.1, and xenbr0/1 along with the usual eth0/1

7. In VM creation, for each network card, specify which xen bridge to use by choosing Source->xenbr0/1

Inetcfg and Jumbo Frames

You can use Inetcfg to manage jumbo frames. Cool Solutions: Using INETCFG To Manage Your NIC Cards

iFolder

Documentation: Configuring an iFolder Server Cluster on NetWare 6.5 and Later

AppNote: iFolder on Open Enterprise Server Linux Cluster using iSCSI

AppNote: Multiple instances of iFolder on NetWare Configuration Report

External Link: Clustering iFolder

DFS

AppNote: Using DFS in a Clustered Environment


NLDAP

It is pretty straight forward to cluster NLDAP. Here is a quick summary of how it is done.

1) Setup cluster

2) Put an eDir replica on each node that will host the LDAP resource.

3) Create a LDAP resource which adds/dels the secondary IP address the clients will use to connect to.

4) Make sure NLDAP is loaded. You can either do this in the resource script or at startup. It is common to start NLDAP at server boot then also check that it is running in the resource script and starts it if it is not.

5) Optional - load/unload a health monitor in the resource scripts. The NCS sdk has a sample health monitor for LDAP. It can be found here. You can compile and use this directly or use this as a template and create one that uses SSL.

6) Create the certificates. (If you are using SSL/TLS)

6a) If you are generating your own certificates. Just create one for each server which has the same IP/DNS name as the cluster resource you created. eDir certificates are Key Material Objects (KMO). The default certificate that NLDAP uses a certificat that is for the DNS name of the physical server. Since the client doesn't know the physical server's name it won't accept a certificate from it as valid. So create a new KMO for each server which will host the LDAP resource. When you create it use the custom creation method. Then change the CN in the subject name to match what the client thinks it will be connecting to. Since certificates are tied to a server you will have to do this for each one. Then go to the LDAP Server object for each server and switch its Server Certificate to be the new one that you created. Then when you client request a cert it will get one with name that it is expecting in it. If your clients will be using an IP address to connect set the CN to the IP address. If it will be using a DNS name then put the DNS name as the CN. If you are using ConsoleOne you will have to escape the . (periods) but you don't have to in iManager.

6b) If you bought a certificate from a CA. Then to save some money you can just reuse it. To do this you will have to back it up then restore it on the other nodes.


eDirectory

Since the eDir DIP files are in a fixed location on NetWare it isn't possible to cluster eDirectory. However this was changed for Linux. eDirectory can use DIP files from anywhere. Therefore it should now be possible to cluster eDirectory on Linux. It should be like any other resource. Simply put the DIP files on the shared storage and create a cluster resource that 1) binds an IP 2) mounts the shared volume 3) starts eDir using the DIPs off of the shared volume.

Fibre Channel

Virtualized Cluster on a Single Box using Fibre Channel

A. Install First Node(First VM):

Create a Virtual Machine Summary:

1) Select Disk

2) Select Hard Disk

3) Select Open to the right of the Source

4) Select File System

5) Select dev

6) Select disk

7) Select by-id

8) Select the shared disk

9) Apply the selected Hard Disks

10) Finish the normal installation process with OES


After completing the installation:

11) In DomO, navigate in the console to /etc/xen/vm

12) Enter ls to determine the name of the config file

13) Open the config file in vi

14) Add a bang(!) after the w but within the single quote for the physical disk(SAN)

 Before: 'phy:/dev/disk/by-id/scsi-360060160b656160060080e6a13b6db11,xvdc,w'
 After:  'phy:/dev/disk/by-id/scsi-360060160b656160060080e6a13b6db11,xvdc,w!'

15) Write the change to the file

16) Shut down the virtual machine, xm shutdown (VM name)

17) Start the virtual machine using the updated config file, xm new (VM name)


B. Install Second Node(Second VM):

1) Install the virtual machine without adding the SAN as a physical disk

DomO console

2) Copy the first node's physical device code from the /etc/xen/vm/(config file name)

  'phy:/dev/disk/by-id/scsi-360060160b656160060080e6a13b6db11,xvdc,w!'

3) Open the second node's config file in vi and paste the physical device code into it.

4) Write the change to the file.

5) Shut down the second virtual machine, xm shutdown (VM name)

6) Start the virtual machine using the updated config file, xm new (VM name)


C. Create Cluster


SAN Tips (Netware)

Rules For Operating a NetWare Servers on a SAN with or without Clustering Services

TID 3750194

Avoiding problems with Cluster Services

TID 3084136

Avoid putting the SBD on a GPT initialized drive, stick with MBR until you see that GPT support has been added for the SBD