Novell Cluster Services is a server clustering system that ensures high availability and manageability of critical network resources including data (file systems), applications, and services. It is a multinode clustering product for Linux that is LDAP enabled via Novell eDirectory and supports failover, failback, and migration (load balancing) of individually managed cluster resources.
Connection Magazine: Tech Talk #3 - Over and Over and Over and Over This is an excellent article about NCS. It covers some of the issues with running mixed NetWare/Linux NCS cluster.
Connection Magazine: Tech Talk #5 - Backup Backup A good overview of NCS. Has a section on clustering GroupWise.
Linux OES Product Documentation: OES Cluster Services 1.8 Administration Guide for Linux
NetWare OES Product Documentation: OES Cluster Services 1.8 Administration Guide for NetWare
NetWare OES Product Documentation: Novell Cluster Services 1.8 Resource Configuration Guide
NetWare 6.5 Product Documentation: Novell Cluster Services
NetWare 6 Product Documentation: Novell Cluster Services
NetWare 5.1 Product Documentation: Novell Cluster Services
Virtually any network service can be a cluster resource. There are a few requirements that a service must meet before being able to be clustered.
1) Relocatable data. The data files that the service uses must be able to be moved to shared storage. On Linux it is usually possible to work around services that don't allow this by using soft links.
2) Power off test. A service must be able to pass the "power off test." A service must be able to start back up successfully after a power outage. If in the case of a power outage the service has corrupt files that prevent it from running, then that service isn't a good candidate for a cluster.
NCS cluster consist primarily of a load and an unload script file. Almost all load script files do the same three things. 1) bind a secondary IP address 2) mount a file system on the shared storage 3) start the service. The unload script of course undoes each of these steps in the reverse order. The unload scripts usually 1) stop the service 2) unmount the file system 3) unbind the IP address.
Below are are few of the most common resource type being created on Novell Cluster Services.
Novell Documentation: GroupWise 8 and Novell Cluster Services
Novell Documentation: GroupWise 7 and Novell Cluster Services
Novell Documentation: GroupWise 6.5 and Novell Cluster Services
Novell Documentation: GroupWise 6 Introduction to GroupWise 6 and Novell Cluster Services
Connection Magazine: Tech Talk #6 - Nuts About Clusters Article about GroupWise High Availability Service.
Cool Solution: Novell Cluster Services Primer for GroupWise Administrators
Novell Documentation: ZENworks 6.5 Introduction to Novell Cluster Services and ZENworks Server Management
Novell Documentation: ZENworks for Servers 3 Introduction to Novell Cluster Services and ZENworks for Servers
Novell Documentation: How to configure iPrint in a Cluster
iSCSI and SLP
Use of iSCSI implies TCP/IP and dual-homed servers. If you properly architect your iSCSI network, you'll have it completely segregated and inaccessible by your client workstations. However, blocking port 524 traffic via ACL, or hiding the network via routing rules is not enough.
NetWare will still present all of its addresses via SLP by default, even if some of them are unreachable. This can cause lengthy delays in service location, drive mappings, etc. To avoid problems with name resolution and login performance, be sure to use the NCP EXCLUDE IP ADDRESS and SLP EXCLUDE IP ADDRESS parameters. Populate them with the IP addresses of the iSCSI network interfaces, to keep clients and other servers on your LAN/WAN from trying to communicate with them on the network you've designated for iSCSI traffic only.
iSCSI and Jumbo Frames
Using at least NetWare 6.5 SP5, performance can be slightly improved in some cases by enabling jumbo ethernet frames. This setting allows a device to increase the payload of each packet from roughly 1,500 bytes to roughly 9,000 bytes, making communications on your iSCSI network more efficient.
Most implementations involving separate "NAS" networks benefit from use of jumbo frames, especially those using NetApp Filers. Linux/Unix hosts, etc. notice a marked performance improvement when using these larger ethernet frames - configuring NetWare to use Jumbo frames in such an environment ensures that it's following best practices.
To do so, you cannot use INETCFG to manage the configuration of your NetWare server's ethernet cards. Manually configuring the cards is beneficial, as it allows you to specify which is loaded first with better predictability. In addition, it allows you to use parameters that are not possible through the INETCFG utility.
Keep in mind that you must have switch hardware capable of supporting jumbo frames, and those switches must be configured properly to do so. For example, the Cisco 3750 stackable switches, and the IBM/Cisco BladeCenter Integrated Gigabit Switch Module (CIGESM), are known to work well in production environments if running the latest versions of IOS for their respective platforms. Also, depending on your server hardware, use of multiple-processors (SMP) may cause stability issues with iSCSI and jumbo frames - be sure that you're always using the latest non-beta support pack, WINSOCK, iSCSI, and network driver files.
Here's the minimum changes necessary to support jumbo ethernet frames on NetWare, as used in a production environment:
- STARTUP.NCF Settings
- SET MAXIMUM PHYSICAL RECEIVE PACKET SIZE=16400
- AUTOEXEC.NCF Settings
- SET TCP PATH MTU BLACK HOLE DETECTION AND RECOVERY = ON
- SET TCP MINSHALL ALGORITHM = OFF
- SET USE SPECIFIED MTU = OFF
- Loading the driver for your NAS interface - Using Broadcom b57.lan driver as an example
- LOAD B57.LAN SLOT=100nn FRAME=ETHERNET_II POLL=1 NAME=NAS JUMBO=9000
Updated iSCSI and Winsock Drivers
- As of this time, WSOCK6i.EXE is an unreleased yet eagerly awaited component that should improve stability and performance in these configurations
iSCSI and Multiple Bridges for Xen Virtualization
Creating multiple bridges to allow two virtual network cards to communicate with two separate networks
1. Install SLES w/ Xen Virtual Machine Host Server
2. Go to /etc/xen/xend-config.sxp and comment out the line Ã¢â‚¬Å“(network-script network-bridge)Ã¢â‚¬Â. Add the line (network-script two-bridges)
#(network-script network-bridge) (network-script two-bridges)
3. In /etc/xen/scripts create file two-bridges with following code:
#!/bin/bash /etc/xen/scripts/network-bridge $1 netdev=eth0 vifnum=0 bridge=xenbr0 /etc/xen/scripts/network-bridge $1 netdev=eth1 vifnum=1 bridge=xenbr1
4. Make two-bridges executable
chmod +x two-bridges
5. Reboot into Xen or run Ã¢â‚¬Å“network-bridge stopÃ¢â‚¬Â and Ã¢â‚¬Å“two-bridges startÃ¢â‚¬Â
6. ifconfig should now display peth0/1, vif0.0/0.1, and xenbr0/1 along with the usual eth0/1
7. In VM creation, for each network card, specify which xen bridge to use by choosing Source->xenbr0/1
Inetcfg and Jumbo Frames
You can use Inetcfg to manage jumbo frames. Cool Solutions: Using INETCFG To Manage Your NIC Cards
External Link: Clustering iFolder
AppNote: Using DFS in a Clustered Environment
It is pretty straight forward to cluster NLDAP. Here is a quick summary of how it is done.
1) Setup cluster
2) Put an eDir replica on each node that will host the LDAP resource.
3) Create a LDAP resource which adds/dels the secondary IP address the clients will use to connect to.
4) Make sure NLDAP is loaded. You can either do this in the resource script or at startup. It is common to start NLDAP at server boot then also check that it is running in the resource script and starts it if it is not.
5) Optional - load/unload a health monitor in the resource scripts. The NCS sdk has a sample health monitor for LDAP. It can be found here. You can compile and use this directly or use this as a template and create one that uses SSL.
6) Create the certificates. (If you are using SSL/TLS)
6a) If you are generating your own certificates. Just create one for each server which has the same IP/DNS name as the cluster resource you created. eDir certificates are Key Material Objects (KMO). The default certificate that NLDAP uses a certificat that is for the DNS name of the physical server. Since the client doesn't know the physical server's name it won't accept a certificate from it as valid. So create a new KMO for each server which will host the LDAP resource. When you create it use the custom creation method. Then change the CN in the subject name to match what the client thinks it will be connecting to. Since certificates are tied to a server you will have to do this for each one. Then go to the LDAP Server object for each server and switch its Server Certificate to be the new one that you created. Then when you client request a cert it will get one with name that it is expecting in it. If your clients will be using an IP address to connect set the CN to the IP address. If it will be using a DNS name then put the DNS name as the CN. If you are using ConsoleOne you will have to escape the . (periods) but you don't have to in iManager.
6b) If you bought a certificate from a CA. Then to save some money you can just reuse it. To do this you will have to back it up then restore it on the other nodes.
Since the eDir DIP files are in a fixed location on NetWare it isn't possible to cluster eDirectory. However this was changed for Linux. eDirectory can use DIP files from anywhere. Therefore it should now be possible to cluster eDirectory on Linux. It should be like any other resource. Simply put the DIP files on the shared storage and create a cluster resource that 1) binds an IP 2) mounts the shared volume 3) starts eDir using the DIPs off of the shared volume.
Virtualized Cluster on a Single Box using Fibre Channel
A. Install First Node(First VM):
Create a Virtual Machine Summary:
1) Select Disk
2) Select Hard Disk
3) Select Open to the right of the Source
4) Select File System
5) Select dev
6) Select disk
7) Select by-id
8) Select the shared disk
9) Apply the selected Hard Disks
10) Finish the normal installation process with OES
After completing the installation:
11) In DomO, navigate in the console to /etc/xen/vm
12) Enter ls to determine the name of the config file
13) Open the config file in vi
14) Add a bang(!) after the w but within the single quote for the physical disk(SAN)
Before: 'phy:/dev/disk/by-id/scsi-360060160b656160060080e6a13b6db11,xvdc,w' After: 'phy:/dev/disk/by-id/scsi-360060160b656160060080e6a13b6db11,xvdc,w!'
15) Write the change to the file
16) Shut down the virtual machine, xm shutdown (VM name)
17) Start the virtual machine using the updated config file, xm new (VM name)
B. Install Second Node(Second VM):
1) Install the virtual machine without adding the SAN as a physical disk
2) Copy the first node's physical device code from the /etc/xen/vm/(config file name)
3) Open the second node's config file in vi and paste the physical device code into it.
4) Write the change to the file.
5) Shut down the second virtual machine, xm shutdown (VM name)
6) Start the virtual machine using the updated config file, xm new (VM name)
C. Create Cluster
SAN Tips (Netware)
Rules For Operating a NetWare Servers on a SAN with or without Clustering Services
Avoiding problems with Cluster Services
Avoid putting the SBD on a GPT initialized drive, stick with MBR until you see that GPT support has been added for the SBD