| Previous | \Contents | ^Index |
H When there are multiple virtual circuits between two OpenVMS systems it G is possible for the VMS$VAXCLUSTER to VMS$VAXCLUSTER connection to use D any one of these circuits. All lock traffic between the two systems 1 will then travel on the selected virtual circuit.
E Each port has a "LOAD CLASS" associated with it. This load F class helps to determine which virtual circuit a connection will use. H If one port has a higher load class than all others then this port will F be used. If two or more ports have equally high load classes then the ? connection will use the first of these that it finds. Prior to E enhancements found in V7.3-1 and later, the load class is static and G normally all CI and DSSI ports have a load class of 14(hex), while the F Ethernet and FDDI ports will have a load class of A(hex). With V7.3-1 - and later, the load class values are dynamic.
@ For instance, if you have multiple DSSI busses and an FDDI, the G VMS$VAXCLUSTER connection will chose the DSSI bus as this path has the H system disk, and thus will always be the first DSSI bus discovered when the OpenVMS system boots.
? To force all lock traffic off the DSSI and on to the FDDI, for D instance, an adjustment to the load class value is required, or the DSSI SCS port must be disabled.
? In addition to the load class mechanisms, you can also use the G "preferred path" mechanisms of MSCP and TMSCP services. This G allows you to control the SCS connections used for serving remote disk H and tape storage. The preferred path mechanism is most commonly used to A explicitly spread cluster I/O activity over hosts and/or storage B controllers serving disk or tape storage in parallel. This can be F particularly useful if your hosts or storage controllers individually D lack the necessary I/O bandwidth for the current I/O load, and must 7 thus aggregate bandwidth to serve the cluster I/O load.
E For related tools, see various utilities including LAVC$STOP_BUS and B LAVC$START_BUS, and see DCL commands including SET PREFERRED_PATH. c 15.6.1.2.3 Cluster Communications Control Tools and Utilities?
0 In most OpenVMS versions, you can use the tools:
F These tools permit you to disable or enable all SCS traffic on the on the specified paths.
F You can also use a preferred path mechanism that tells the local MSCP B disk class driver (DUDRIVER) which path to a disk should be used. D Generally, this is used with dual-pathed disks, forcing I/O traffic F through one of the controllers instead of the other. This can be used F to implement a crude form of I/O load balancing at the disk I/O level.
8 Prior to V7.2, the preferred path feature uses the tool:
A In OpenVMS V7.2 and later, you can use the following DCL command:
|
" $ SET PREFERRED_PATH |
H The preferred path mechanism does not disable nor affect SCS operations on the non-preferred path.
F With OpenVMS V7.3 and later, please see the SCACP utility for control ? over cluster communications, SCS virtual circuit control, port selection, and related. N 15.6.2 Cluster System Parameter Settings?
F The following sections contain details of configuring cluster-related system parameters. d 15.6.2.1 What is the correct value for EXPECTED_VOTES in a VMScluster?
G The VMScluster connection manager uses the concept of votes and quorum H to prevent disk and memory data corruptions---when sufficient votes are @ present for quorum, then access to resources is permitted. When E sufficient votes are not present, user activity will be blocked. The G act of blocking user activity is called a "quorum hang", and is better C thought of as a "user data integrity interlock". This mechanism is H designed to prevent a partitioned VMScluster, and the resultant massive F disk data corruptions. The quorum mechanism is expressly intended ; to prevent your data from becoming severely corrupted.
D On each OpenVMS node in a VMScluster, one sets two values in SYSGEN: A VOTES, and EXPECTED_VOTES. The former is how many votes the node G contributes to the VMScluster. The latter is the total number of votes 2 expected when the full VMScluster is bootstrapped.
H Some sites erroneously attempt to set EXPECTED_VOTES too low, believing G that this will allow when only a subset of voting nodes are present in < a VMScluster. It does not. Further, an erroneous setting in F EXPECTED_VOTES is automatically corrected once VMScluster connections ? to other nodes are established; user data is at risk of severe C corruptions during the earliest and most vulnerable portion of the ? system bootstrap, before the connections have been established.
G One can operate a VMScluster with one, two, or many voting nodes. With B any but the two-node configuration, keeping a subset of the nodes H active when some nodes fail can be easily configured. With the two-node E configuration, one must use a primary-secondary configuration (where H the primary has all the votes), a peer configuration (where when either E node is down, the other hangs), or (preferable) a shared quorum disk.
G Use of a quorum disk does slow down VMScluster transitions somewhat -- F the addition of a third voting node that contributes the vote(s) that H would be assigned to the quorum disk makes for faster transitions---but B the use of a quorum disk does mean that either node in a two-node A VMScluster configuration can operate when the other node is down.
= If you choose to use a quoum disk, a QUORUM.DAT file will be automatically H created when OpenVMS first boots and when a quorum disk is specified -- A well, the QUORUM.DAT file will be created when OpenVMS is booted 4 without also needing the votes from the quorum disk.
G In a two-node VMScluster with a shared storage interconnect, typically ? each node has one vote, and the quorum disk also has one vote. EXPECTED_VOTES is set to three.
F Using a quorum disk on a non-shared interconnect is unnecessary---the H use of a quorum disk does not provide any value, and the votes assigned B to the quorum disk should be assigned to the OpenVMS host serving access to the disk.
D For information on quorum hangs, see the OpenVMS documentation. For F information on changing the EXPECTED_VOTES value on a running system, F see the SET CLUSTER/EXPECTED_VOTES command, and see the documentation H for the AMDS and Availability Manager tools. Also of potential interest G is the OpenVMS system console documentation for the processor-specific D console commands used to trigger the IPC (Interrrupt Priority Level F %x0C; IPL C) handler. AMDS, Availability Manager, and the IPC handler F can each be used to clear a quorum hang. Use of AMDS and Availability H Manager is generally recommended over IPC, particularly because IPC can F cause CLUEXIT bugchecks if the system should remain halted beyond the cluster sanity timer limits.
D The quorum scheme is a set of "blade guards" deliberately F implemented by OpenVMS Engineering to provide data integrity---remove G these blade guards at your peril. OpenVMS Engineering did not D implement the quorum mechanism to make a system manager's life more F difficult--- the quorum mechanism was specifically implemented to + keep your data from getting scrambled. X 15.6.2.2 Explain disk (or tape) allocation class settings?
F The allocation class mechanism provides the system manager with a way D to configure and resolve served and direct paths to storage devices H within a cluster. Any served device that provides multiple paths should H be configured using a non-zero allocation class, either at the MSCP (or G TMSCP) storage controllers, at the port (for port allocation classes), E or at the OpenVMS MSCP (or TMSCP) server. All controllers or servers D providing a path to the same device should have the same allocation 1 class (at the port, controller, or server level).
F Each disk (or tape) unit number used within a non-zero disk (or tape) E allocation class must be unique, regardless of the particular device F prefix. For the purposes of multi-path device path determination, any F disk (or tape) device with the same unit number and the same disk (or F tape) allocation class configuration is assumed to be the same device.
G If you are reconfiguring disk device allocation classes, you will want E to avoid the use of allocation class one ($1$) until/unless you have F Fibre Channel storage configured. (Fibre Channel storage specifically 8 requires the use of allocation class $1$. eg: $1$DGA0:.) a 15.6.2.2.1 How to configure allocation classes and Multi-Path SCSI?
F The HSZ allocation class is applied to devices, starting with OpenVMS E V7.2. It is considered a port allocation class (PAC), and all device H names with a PAC have their controller letter forced to "A". (You might ? infer from the the text in the "Guidelines for OpenVMS Cluster F Configurations" that this is something you have to do, though OpenVMS 0 will thoughtfully handle this renaming for you.)
> You can force the device names back to DKB by setting the HSZ G allocation class to zero, and setting the PKB PAC to -1. This will use F the host allocation class, and will leave the controller letter alone E (that is, the DK controller letter will be the same as the SCSI port H (PK) controller). Note that this won't work if the HSZ is configured in G multibus failover mode. In this case, OpenVMS requires that you use an allocation class for the HSZ.
C When your configuration gets even moderately complex, you must pay B careful attention to how you assign the three kinds of allocation C class: node, port and HSZ/HSJ, as otherwise you could wind up with 7 device naming conflicts that can be painful to resolve.
F The display-able path information is for SCSI multi-path, and permits F the multi-path software to distinguish between different paths to the G same device. If you have two paths to $1$DKA100, for example by having H two KZPBA controllers and two SCSI buses to the HSZ, you would have two > UCBs in a multi-path set. The path information is used by the : multi-path software to distinguish between these two UCBs.
G The displayable path information describes the path; in this case, the D SCSI port. If port is PKB, that's the path name you get. The device H name is no longer completely tied to the port name; the device name now D depends on the various allocation class settings of the controller, SCSI port or node.
E The reason the device name's controller letter is forced to "A" when @ you use PACs is because a shared SCSI bus may be configured via H different ports on the various nodes connected to the bus. The port may E be PKB on one node, and PKC on the other. Rather obviously, you will H want to have the shared devices use the same device names on all nodes. B To establish this, you will assign the same PAC on each node, and F OpenVMS will force the controller letter to be the same on each node. G Simply choosing "A" was easier and more deterministic than negotiating @ the controller letter between the nodes, and also parallels the G solution used for this situation when DSSI or SDI/STI storage was used.
E To enable port allocation classes, see the SYSBOOT command SET/BOOT, + and see the DEVICE_NAMING system parameter.
> This information is also described in the Cluster Systems and 6 Guidelines for OpenVMS Cluster Configurations manuals. P 15.6.3 Tell me about SET HOST/DUP and SET HOST/HSC
C The OpenVMS DCL commands SET HOST/DUP and SET HOST/HSC are used to H connect to storage controllers via the Diagnostics and Utility Protocol A (DUP). These commands require that the FYDRIVER device driver be C connected. This device driver connection is typically performed by @ adding the following command(s) into the system startup command procedure:
On OpenVMS Alpha:
|
" $ RUN SYS$SYSTEM:SYSMAN 9 SYSMAN> IO CONNECT FYA0/NOADAPTER/DRIVER=SYS$FYDRIVER |
On OpenVMS VAX:
|
" $ RUN SYS$SYSTEM:SYSGEN " SYSGEN> CONNECT FYA0/NOADAPTER |
E Alternatives to the DCL SET HOST/DUP command include the console SET F HOST command available on various mid- to recent-vintage VAX consoles:
4 Access to Parameters on an Embedded DSSI controller:
|
" 6 SET HOST/DUP/DSSI[/BUS:{0:1}] dssi_node_number PARAMS |
< Access to Directory of tools on an Embedded DSSI controller:
|
" 6 SET HOST/DUP/DSSI[/BUS:{0:1}] dssi_node_number DIRECT |
0 Access to Parameters on a KFQSA DSSI controller:
|
" 2 SHOW UQSSP ! to get port_controller_number PARAMS 1 SET HOST/DUP/UQSSP port_controller_number PARAMS |
E These console commands are available on most MicroVAX and VAXstation B 3xxx series systems, and most (all?) VAX 4xxx series systems. For D further information, see the system documentation and---on most VAX $ systems---see the console HELP text.
F EK-410AB-MG, _DSSI VAXcluster Installation and Troubleshooting_, is a E good resource for setting up a DSSI VMScluster on OpenVMS VAX nodes. H (This manual predates coverage of OpenVMS Alpha systems, but gives good > coverage to all hardware and software aspects of setting up a F DSSI-based VMScluster---and most of the concepts covered are directly E applicable to OpenVMS Alpha systems. This manual specifically covers E the hardware, which is something not covered by the standard OpenVMS VMScluster documentation.)
k Also see Section 15.3.3, and for the SCS name of the OpenVMS host see 0 Section 5.7. K 15.6.4 How do I rename a DSSI disk (or tape?)
A If you want to renumber or rename DSSI disks or DSSI tapes, it's , easy---if you know the secret incantation...
From OpenVMS:
|
" $ RUN SYS$SYSTEM:SYSGEN " SYSGEN> CONNECT FYA0/NOADAPTER SYSGEN> ^Z @ $ SET HOST/DUP/SERV=MSCP$DUP/TASK=PARAMS <DSSI-NODE-NAME> ... PARAMS> STAT CONF F <The software version is normally near the top of the display.> PARAMS> EXIT ... |
E From the console on most 3000- and 4000-class VAX system consoles... < (Obviously, the system must be halted for these commands...)
Integrated DSSI:
|
" 6 SET HOST/DUP/DSSI[/BUS:[0:1]] dssi_node_number PARAMS |
KFQSA:
|
" 1 SET HOST/DUP/UQSSP port_controller_number PARAMS |
F For information on how to get out into the PARAMS subsystem, also see H the HELP at the console prompt for the SET HOST syntax, or see the HELP @ on SET HOST /DUP (once you've connected FYDRIVER under OpenVMS).
E Once you are out into the PARAMS subsystem, you can use the FORCEUNI C option to force the use of the UNITNUM value and then set a unique D UNITNUM inside each DSSI ISE---this causes each DSSI ISE to use the C specfied unit number and not use the DSSI node as the unit number. F Other parameters of interest are NODENAME and ALLCLASS, the node name 0 and the (disk or tape) cluster allocation class.
F Ensure that all disk unit numbers used within an OpenVMS Cluster disk F allocation class are unique, and all tape unit numbers used within an F OpenVMS Cluster tape allocation class are also unique. For details on j the SCS name of the OpenVMS host, see Section 5.7. For details of SET B HOST/DUP, see Section 15.6.3. ] 15.6.5 Where can I get Fibre Channel Storage (SAN) information?
? Review the VMScluster documentation, and the System Management H documentation. The following are the key points, but are likely not the $ only things you will need to change.
B OpenVMS Cluster support is directly integrated into the operating C system, and there is no way to remove it. You can, however, remote @ site-specific tailoring that was added for a particular cluster configuration.
E First: Create restorable image BACKUPs of each of the current system E disks. If something gets messed up, you want a way to recover, right?
F Create standalone BACKUP kits for the OpenVMS VAX systems, and create > or acquire bootable BACKUP kits for the OpenVMS Alpha systems.
F Use CLUSTER_CONFIG or CLUSTER_CONFIG_LAN to remove the various system < roots and to shut off boot services and VMScluster settings.
C Create as many architecture-specific copies of the system disks as B required. Realize that the new systems will all likely be booting G through root SYS0---if you have any system-specific files in any other roots, save them.
H Relocate the copies of the VMScluster common files onto each of the new system disks.
H Reset the console parameters and boot flags on each system for use on a standalone node.
G Reset the VAXCLUSTER and NISCS_LOAD_PEA0 parameters to 0 in SYSGEN and in MODPARAMS.DAT.
: Clobber the VMScluster group ID and password using SYSMAN.
7 Reboot the systems seperately, and run AUTOGEN on each.
@ Shut off MOP services via NCP or LANCP on the boot server nodes.
H Permanent seperation also requires the duplication of shared files. For A a list of the files commonly shared, please see the most current G version of SYS$STARTUP:SYLOGICALS.TEMPLATE, and specifically a version from OpenVMS V7.2 or later. < The following files are typically shared within a cluster:
|
" K Filename: default directory (in common root) and file type: 0 SYSUAF SYS$SYSTEM:.DAT 0 SYSUAFALT SYS$SYSTEM:.DAT 0 SYSALF SYS$SYSTEM:.DAT 0 RIGHTSLIST SYS$SYSTEM:.DAT 0 NETPROXY SYS$SYSTEM:.DAT 0 NET$PROXY SYS$SYSTEM:.DAT 0 NETOBJECT SYS$SYSTEM:.DAT 0 NETNODE_REMOTE SYS$SYSTEM:.DAT M QMAN$MASTER SYS$SYSTEM: (this is a set of related files) 0 LMF$LICENSE SYS$SYSTEM:.LDB 1 VMSMAIL_PROFILE SYS$SYSTEM:.DATA 0 VMS$OBJECTS SYS$SYSTEM:.DAT 1 VMS$AUDIT_SERVER SYS$MANAGER:.DAT 1 VMS$PASSWORD_HISTORY SYS$SYSTEM:.DATA 1 NETNODE_UPDATE SYS$MANAGER:.COM 1 VMS$PASSWORD_POLICY SYS$LIBRARY:.EXE A LAN$NODE_DATABASE SYS$SYSTEM:LAN$NODE_DATABASE.DAT |
B Also see the topics on "cluster divorce" in the Ask The Wizard area.
H For additional information on the OpenVMS Ask The Wizard (ATW) area and B for a pointer to the available ATW Wizard.zip archive, please see 0 Section 3.9.
b Information on changing node names is included in Section 5.7. B 15.6.7 Details on Volume Shadowing?
E This section contains information on host-based volume shadowing; on 9 the disk mirroring capabilities available within OpenVMS. c 15.6.7.1 Does volume shadowing require a non-zero allocation classes?
B Yes, use of host-based Volume Shadowing requires that the disk(s) 6 involved be configured in a non-zero allocation class.
< Edit SYS$SYSTEM:MODPARAMS.DAT to include a declaration of an H non-zero allocation class, such as setting the host allocation class to the value 7:
|
" ALLOCLASS = 7 |
$ Then AUTOGEN the system, and reboot.
H You should now be able to form the shadow set via a command such as the following:
|
" = $ MOUNT dsa1007: /SHADOW=($7$dkb300:,$7$dkb500:) volumelabel |
C When operating in an OpenVMS Cluster, this sequence will typically @ change the disk names from the SCSNODE prefix (scsnode$dkann) to I the allocation-class prefix ($7$dkannn). This may provide you with the H opportunity to move to a device-independent scheme using logical name H constructs such as the DISK$volumelabel logical names in your startup D and application environments; an opportunity to weed out physical device references.
F Allocation class one is used by Fibre Channel devices; it can be best F to use another non-zero allocation class even if Fibre Channel is not / currently configured and not currently planned.
| Index | \Contents |