Table of Contents
Cloning HPUX 11.11 PA-RISC
This procedure & document was created in October of 2020 by Swift Griggs. Here is a procedure for cloning HPUX 11.11 systems which use LVM root volumes, mirrored with MirrorDisk/UX or standalone. HPUX 11.11 was so stingy that they withheld the mirroring feature in LVM to extort more money from their customers. It’s one of many things that drove folks to freeware operating systems, for example NetBSD or Linux. These methods will work for HPUX on 11.11 PA-RISC systems and are also known to work with software emulated (Qemu HPPA / qemu-system-hppa) HPUX setups. It is mostly the same on the 11iv1 Itanium/Integrity system, with the exception of some steps needed for EFI booting (Itanium) versus ISL booting (PA-RISC) which aren’t covered in this particular document (Itanium / Integrity systems).
Why Do It?
You might want to perform this procedure to create a clone of a boot disk. Notice I say “clone” not “mirror” these are different things. Some people want to make a backup of their root disk in case of some issue like a drive failure. Others might want to clone a working system onto new hardware as a “gold image” which has a lot of the custom stuff they use in their environment already pre-installed and perhaps their security pre-hardened to the security policy requirements for their environment. Another great reason to use this procedure is because one can convert to and from new storage back-ends like local disks, fiber channel SANs, internal RAID, flash storage, or an external RAID array. This procedure works with LVM root disks and mirrored LVM root disks. It does not work with Veritas VxVM root disk encapsulation disks. There is a way to do that, too. In Veritas Volume Manager one can create a root disk mirror and break it for an easy clone set. However, this document covers LVM only as it’s the most commonly encountered scenario. With a little cleverness, one can also easily use this method to migrate a physical HPUX system to a virtual Qemu HPPA based system. Pull a 'dd' image and put it on the same bus ID in Qemu. You can also use the manual procedure in stages using a boot DVD and NFS. Get creative. It’s HPUX 11.11 and you’ll need it.
Why Not Use Ignite?
I know many HPUX admins love and rely on Ignite, but there is a huge problem with that. Ignite is often not installed and some older environments are version locked with no original OS media to go grab a PA-RISC version of Ignite from.
It’s not part of the default install unless you mark it. So, one needs a procedure that doesn’t rely on Ignite for cloning. That said, under normal conditions Ignite is easy to get and is pretty worthwhile. It can be used to keep bare-metal backups of your system made over the network. It can also create bootable tapes or clone disks.
Sure, Use DRD (if you have it)
There is, of course the nice DRD feature on Itanium versions of HPUX after 11iv1. That does little good for PA-RISC 11.11 users who have no such tool (bummer!). However, if you do have an Itanium, I’d highly recommend checking it out. It’s included on the 11iv2 and 11iv3 DVD distributions. It is very handy and effective at cloning your root disk. It’s so much more lightweight and with far fewer dependencies and setup than Ignite, too. If you just want to clone your root disk and you’ve got DRD, I’d recommend you use it before any other method.
# Clone your root disk onto c2t0d0. Easy right? drd clone -t /dev/dsk/c2t0d0
What about Good Old DD
Yes, dd works ! However, there are two fairly severe drawbacks. If you use dd then you will need two disks of identical size and geometry. It might work on a larger target disk, but don’t bet on it. It helps if they are just identical and the system is completely quiet while cloning it. In-progress writes are unwelcome while cloning and can result in a corrupt clone.
Example of a dd clone under HPUX 11.11
# Clone a disk at SCSI ID 1 onto a disk at SCSI ID 4 dd if=/dev/dsk/c0t1d0 of=/dev/dsk/c0t4d0 bs=1024k
The other drawback of using dd is that the target disk cannot be used on any other bus address (SCSI ID or LUN ID). It won’t work and won’t be happy in the slightest and you will see an LVM configuration failure kernel panic. Ouch.
You can still get a dd clone on a new ID to boot into maintenance mode, though. If you stop the system at the ISL prompt, then use the hpux stage2 loader with the “-lm” flag you can boot a dd style clone on a different bus address. Then from single-user mode you can do a vgimport on the root volume group followed by executing bcheckrc to fire up the filesystems and switch into another runlevel.
Booting a dd Clone in Maintenance Mode and strapping to runlevel 3
# Force the loader to do maintenance mode. You might need -lqm for a mirror
# set
ISL> hpux -lm
# Whatever you call your root VG... import it
vgimport vg01
# Activate it.
vgchange -a y vg01
# Fire up root writability, /usr, and other important filesystems.
bcheckrc
# You __might__ want to mount all filesystems, if you don't have any
# NFS mounts or things that might hang at this point, then go for it.
mount -a
# Switch to runlevel 3 manually. We are back, son.
init 3
That procedure will boot a crippled system, but the problem is that it’ll also never get back to working order on it’s new home. You’ll be stuck doing that silly procedure every time unless you put the target/clone disk on the exact same bus position as the disk you originally cloned. So, it’s not suitable for any situation where you can’t swap disks around, to get back to the original SCSI bus position.
This leaves you with the implication that dd clones are only good if you have physical access to switch the bus position (SCSI ID) back to the same spot the source/original disk was at when you did the dd cloning. Needless to say, this also means that a dd clone onto a SAN device will forever suffer this crippling bootup procedure, if it works at all. You are better off with the manual cloning procedure.
High Price of MirrorDisk/UX Licenses
I’m sorry it’s so much longer and more complicated. Please, blame HPUX 11.11, not me. Blame them for not giving the LVM mirroring feature, MirrorDisk/UX with the OS by default without greasing them for hundreds and even thousands of dollars to get the feature. This would have eliminated the need for the long procedure as we could have just broke the LVM mirror.
If you happen to have MirrorDisk/UX then you are lucky. You can use LVM mirroring to create a clone onto new disks and migrate easily without as many steps. Just mind that you boot the clone without quorum (ISL> hpux -lq) before either stripping off the mirror or re-mirroring to a new disk. You might want to consider manual cloning instead for two reasons. First, it will teach you a lot about the guts (booting and key config files) of HPUX that might be useful if you are system admin. Secondly, it gives you maximum flexibility for storage migrations such as SAN migrations and an additional degree of safety since you don’t need to alter your root VG at all to clone it manually. Manual cloning is also is file based not block based but that’s just a fact, not really an advantage.
If you have the flexibility to install HPUX from scratch, you might also consider a Veritas Volume Manager root disk setup. It just depends on how you feel about VxVM versus LVM. You can encapsulate and clone an LVM based disk onto a new VxVM based rootdg setup. VxVM can give you a RAID1 like feature. VxVM also requires a license under many of it’s more helpful conditions.
The Manual Cloning Procedure
Below is the procedure I’ve been able to work out. I hope it helps hobbyists or customers still using HPUX version 11.11 on Hewlett Packard HPPA / PA-RISC systems.
The nice thing is you can be kind of stuck without Ignite, DRD, or anything more feature rich: this still works.
Normally what happens when you need to use this procedure is that you have a backup from an older system that has crashed complete or lost it's mirror and is trashed. You want to fully recover the system. Another reason might be because you need to clone an HP-UX system which has a lot of licenses and additional optimizations or user accounts etc… In other words, stuff you don't want to have to reconstruct.
In general you can use any system you want as a source but you'll probably have to restore it as a two step process. Ie.. say you've got a system with two internal disks which you eventually want to use as a mirror-set. What you'll need to do (once you have your backup available is essentially this process:
- Install a vanilla HP-UX OS image using the CDROMs or DVD and just do a standard installation. You'll be overwriting this eventually, but you need it in order to perform the restore, first.
- Leave the second disk alone. Don't mirror on it or allow the installer it use it for a mirror. This will be your clone target.
- Once you boot the system, perform the cloning steps as documented here using the second disk to create a new volume group and filesystems to house your restored data. Watch out for all the LIF and boot data preparitory steps. These will bite you later if you fail to do them as documented.
- Perform the restore by extracting all the needed files you backed up.
- Fix the /etc/fstab, the /stand/bootconf, fix/copy any LVM configuration files.
Prepare the Target Disk
If the clone-target disk you are using isn’t new or came in from a RAID card, you might want to go ahead and wipe the first 100MB or so just to keep LVM, VxVM, RAID cards, or other sub-systems from interfering with the process. We will need to create a layout on our target disk which matches the source disk. For HPUX the OS is always laid out the same way by the installer. Making custom changes can break HPUX scripts, tools, and other places they hardcoded their assumptions. Change as little as you can get away with. However, one upside to this procedure is that it let's you clone the OS onto a disk that isn't the same size as the original.
The typical HPUX layout
Here is the normal layout for an 11.11 disk.
- lvol1 is for /stand and is bootable. It’s the “standalone” boot file system.
- lvol2 is for swap and can serve as a dump device for kernel dumps
- lvol3 is for your root filesystem (/)
- lvol4 is for temporary files (/tmp)
- lvol5 is for user home directories (/home)
- lvol6 is for for optional utilities and 3rd Party system applications (/opt)
- lvol7 is the traditional Unix user filesystem containing optional binaries, man pages, shared documentation, system libraries, and you don’t need to boot the system but are still considered part of the OS (/usr). Still very important since it'll have 'vi' and such.
- lvol8 is the traditional spot for “various” files such as crontabs, mail boxes, printer spools, and system logs. (/var)
Setting the HPUX Loader Boot Auto Action String
Now you need an “auto” action when the system boots. This is what the ISL is going to do when it first starts booting your device automatically. The “hpux” utility is unimaginatively named as the stage2 loader for HPUX. It lives in a very simple filesystem called the LIF. It’s a tool you can execute from the ISL firmware prompt to bootstrap the HPUX OS. If your GSP / console is setup to stop at the ISL prompt, you’ll need to invoke the “hpux” utility yourself to boot your system.
However, be aware that LVM systems which use a mirrored root disk will fail to boot from a single disk. This is because the LVM requires a “quorum” of two disks to operate normally. If a disk fails, it’s not normal, so you need to disable the quorum in order to boot the system. You can do this by passing the “hpux” stage2 loader utility the flags “-lq”. If you stop your system from autobooting it’s possible to do this manually by simply typing “hpux -lq” at the ISL prompt.
However, given that the whole point of using mirroring for LVM root disks is to be resilient in the face of a hardware failure. So, in order to anticipate that failure and provide enhanced functionality, we should use the mkboot utility to disable quorum on the root LVM disks by default. After all, if the partner disk is there and healthy it’ll automatically join the volume group. If it can’t, we’d like the system to boot anyway, not hang and act squirrely.
Boot Sector Related
# Setup the disk by carving out the LIF area and such for LVM
mkboot -l /dev/dsk/cXtXdX
# Add the auto boot command to the boot section (fine for single disk users)
mkboot -a hpux /dev/dsk/cXtXdX
# Add the auto boot command which disables LVM quorum on a root disk mirror (good idea for mirror users)
mkboot -a "hpux -lq" /dev/dsk/cXtXdX
# Copy the existing lif files to the target disk
lifcp -l /dev/dsk/cXdXtX
# Copy the "auto" file into the LIF (a boot archive / library file system)
lifcp cXdXtX:AUTO -
# Check the old and new devices LIF listing and make sure they match.
# Do this on BOTH the source original and destination clone disks.
cd /dev/dsk ; lifls -l cXtXdX ; lifls -l cXtXdX
# After you setup the LIF area, now prep the disk with pvcreate
# You may need to use the -f flag (careful!) if the disk was already
# used by LVM. Make damn sure you don't trash a VG that you need.
pvcreate -B /dev/dsk/cXtXdX
Create the Target Volume Group for Your Clone
Next you will need to create a new volume group to contain the cloned logical volumes. Obviously, you can’t use the name “vg00” because it’s already taken. I just use “vg01” but the name is arbitrary. HPUX is remarkably insensitive to the name of your root volume group. The important thing is you use a unique minor number when executing the mknod command shown below. Look at the “group” file in each of your other volume groups and then choose the next sequential hexadecimal number (1-9, a-f) for a new “group” file.
New Volume Group Creation
HPUX needs you to manually create the control DSF to get a new volume group. This is extremely stupid and could easily be done automatically by system utilities. This shows terrible due-diligence on the part of the system programmers for HPUX but it just is what it is. This kind of thing is why open source Unix variants are now kicking the stuffing out of commercial variants. Unfortunately, it's typical and mundane in HPUX. Enough complaints, here is the procedure.
# create the directory for the new volume group
mkdir /dev/vg01
# second "x" below should be a unique number used by no other volume groups
mknod /dev/vg01/group c 64 0x0x0000
# If your disk is of any size, it'll probably need a larger Physical Extent
# Size to satisfy the LVM VG equation. Here I use a 64MG PE size which
# works for a 300 GB disk.
vgcreate -s 64 vg01 /dev/dsk/cXtXdX
Next Create Logical Volumes on the Clone Volume Group
You now need to create new logical volumes (LVs) to serve as clone targets from your root setup. The first three LVs are used for boot-relate tasks. They should be contiguous. Bad block relocation is not possible on boot volumes. That should be disabled. You will first want to know how large your current logical volumes are on the source side. They do not have to match, but targets have to be as large or larger as the utilized space on their matching source side.
Logical Volume Creation
The goal here is to create the same sized logical volumes, with the same names and layout as the source was using. You'll note the sizes of the source, then use that to create equals on the target group.
# Find the sizes of the source volumes in megabytes
vgdisplay -v vg00 | egrep -i 'LV Name|LV Size'
# XX is the number of megabytes / size of the logical volume. Which you get
# from looking at the original logical volumes on the source volume group
lvcreate -C y -r n -L XX -n lvol1 vg01
lvcreate -C y -r n -L XX -n lvol2 vg01
lvcreate -C y -r n -L XX -n lvol3 vg01
lvcreate -L XX -n lvol4 vg01
lvcreate -L XX -n lvol5 vg01
lvcreate -L XX -n lvol6 vg01
lvcreate -L XX -n lvol7 vg01
lvcreate -L XX -n lvol8 vg01
Create and Mount Cloned Filesystems
HPUX has a quirk insomuch that it uses it’s old file system (HFS) for boot volumes. It’s a long story. Just understand that we’ll be using HFS for the “stand” (/stand) filesystem. That’s required and not negotiable if you want the disk to boot. The other filesystems need to be VXFS with specific options. You can see how the original file systems were created but I will show the specific flags that matter.
# We need HFS on the boot file system (/stand)
newfs -B -F hfs /dev/vg01/rlvol1
# We need VxFS with largefiles disabled on the root filesystem
mkfs -F vxfs -o nolargefiles /dev/vg01/lvol3
# Let's wipe out the first 10 megs of the swap volume
dd if=/dev/zero of=/dev/vg01/lvol2 bs=1024k count=10
# Now zip through the rest of the filesystem creation with a quick loop
# we __do__ want largefiles on the rest of the non-root volumes
for VOL in 4 5 6 7 8 ; do
echo "Formatting logical volume lvol$VOL "
mkfs -F vxfs -o largefiles /dev/vg01/lvol$VOL
done
# Now mount them on a temporary directory structure for root
mkdir /new
mount /dev/vg01/lvol3 /new
# now for /stand. It's HFS but mount will know automagically.
mkdir /new/stand
mount /dev/vg01/lvol1 /new/stand
# Might as well make sure the /tmp dir mounts properly
mkdir /new/tmp
mount /dev/vg01/lvol4 /new/tmp
# Now do the other regular file systems.
mkdir /new/home
mount /dev/vg01/lvol5 /new/home
mkdir /new/opt
mount /dev/vg01/lvol6 /new/opt
mkdir /new/usr
mount /dev/vg01/lvol7 /new/usr
mkdir /new/var
mount /dev/vg01/lvol8 /new/var
Copy Data
Now that you have recreated the layout structure and filesystems on the new disk, you'll need to copy over the data. This can be done with a number of tools.I am going to provide you with no less than THREE working methods simply because 11.11 can be painful and I’m just showing off. Just use one of these to copy the data from source to destination mount points. I will show the /usr file system in my example, but obviously you should do all the filesystem which are critical for boot disks. Typically, these are: /, /stand, /usr, /var, /home, and /opt. You do not need to copy over the contents of the /tmp file system. It’s fine for it to be completely empty (but mkfs’d of course).
Pick one of the methods you like the best below. I like pax for simplicity. I like dump for old-school charm and metadata completeness. All these methods work in 11.11 despite not having nice tools like GNU tar or 'rsync'. The easiest way is if you do have GNU tar and you can backup the filesystems with it then also restore with it. The reason is that GNU Tar has a flag (-C) which makes restoring to a virtual chroot much easier.
I'm just going to use the /usr file system in these examples.
Dump comes with HP-UX. So, this method will work for any system, but has a lot of commands.
# Using dump (For HFS) dump -0f - /usr | (cd /new/usr ; restore -xf -) # Using vxdump (For VxFS) vxdump -0f - /usr | (cd /new/usr ; vxrestore -xf -) # Using the HPUX tar archiver cd /usr ; tar -cvpf - * | (cd /new/usr ; tar -xpf -) # Using the cpio archiver cd /usr ; find . -xdev -depth -print | cpio -pdlmxv /new/usr # Using pax (a rarely used but installed by default tar competitor) cd /usr ; pax -tvrw . /new/usr
Don’t forget to do all OS file systems. Only swap (lvol2) is skippable. It’s just a block device used for virtual memory and kernel dumps, not a filesystem. It needs to exist. That’s the only one you can skip.
# One quick way to go, after mounts are complete on /new
for FS in / /stand /usr /var /opt /home ; do
echo "INFO: About to copy $FS to /new/$FS "
sleep 2
cd $FS
pax -tvrw . /new/${FS}
done
Now if you can get GNU Tar for your system and you care to go through the hassle of FTP'ing it back to the system once you install the OS, you can use it's features to make this a lot easier. I'll show you how to back up and restore below.
# To do a good OS backup in HP-UX try this (using GZIP, too) tar --one-file-system -czvpf /backup/full-system-backup.tar.gz / /opt /usr /var /home # To restore using GNU Tar (not HP-UX tar) you can use this # but only after you have all the target filesystems mounted. tar -xzv -C /new -f full-system-backup.tar.gz
Choose Boot, Swap, Root, and Dump Volumes
HPUX has some magical jiggery pokery it wants done to volumes one intends to boot from. It also imposes upon you as an admin to define it’s root, boot, swap, and dump volumes. Fortunately, there is one tool to do this called lvlnboot. Be aware that if you hork it up, you can use lvrmboot to start over. You need to anoint the volumes you intend to clone.
# Set the boot volume lvlnboot -b /dev/vg01/lvol1 # Set the root volume lvlnboot -r /dev/vg01/lvol3 # Set the swap volume lvlnboot -s /dev/vg01/lvol2 # Set the kernel dump volume (same as swap usually) lvlnboot -d /dev/vg01/lvol2 # Now view the configuration and visually validate it. lvlnboot -v
Update Two Cloned Configuration Files
Now, there are two places you need to update in order for your clone to reflect changes you had to make in order to complete this process. The first is your /stand/bootconf file. It contains a simple textual line of an “l” followed by two spaces, and the physical boot disk device path. Your old disk is there now. Simply edit this file with vi and update to match your new disk.
Edit your /stand/bootconf
# It's got stale data. Old disk was SCSI ID1 cat /new/stand/bootconf l /dev/dsk/c0t1d0 # Edit it with our new disk vi /new/stand/bootconf # Now it's got our new disk, in this specific case SCSI ID4 cat /new/stand/bootconf l /dev/dsk/c0t4d0
You’ll also need to update your /etc/fstab file to reflect the name of your new volume group. In my case, I used “vg01” for the name of my root VG. The file now contains stale data showing “vg00” and needs to be changed. So, I’ll fix it by changing “vg00” to “vg01” in the vi editor.
# The data is stale. Our source VG was vg00, but we use vg01 cat /new/etc/fstab /dev/vg00/lvol3 / vxfs delaylog 0 1 /dev/vg00/lvol1 /stand hfs defaults 0 1 /dev/vg00/lvol4 /tmp vxfs delaylog 0 2 /dev/vg00/lvol5 /home vxfs delaylog 0 2 /dev/vg00/lvol6 /opt vxfs delaylog 0 2 /dev/vg00/lvol7 /usr vxfs delaylog 0 2 /dev/vg00/lvol8 /var vxfs delaylog 0 2 # Fix it by updating with vi (or you can get clever with sed) vi /etc/fstab # Another way in my case (vg00 becomes vg01) cat /etc/fstab | sed s/vg00/vg01/g > /new/etc/fstab # Should be updated with the new VG on our target disk (vg01 in my case) /dev/vg01/lvol3 / vxfs delaylog 0 1 /dev/vg01/lvol1 /stand hfs defaults 0 1 /dev/vg01/lvol4 /tmp vxfs delaylog 0 2 /dev/vg01/lvol5 /home vxfs delaylog 0 2 /dev/vg01/lvol6 /opt vxfs delaylog 0 2 /dev/vg01/lvol7 /usr vxfs delaylog 0 2 /dev/vg01/lvol8 /var vxfs delaylog 0 2
Fix any stale LVM Configurations
Once you restore your data, that's probably going to include LVM related files in /etc such as your lvmtab, file and lvmconf directory. The problem with this, is that your storage layout might have changed. You're old LVM configuration files almost certainly don't include your new volume group. They also may include volume group configurations that are not related to your booting process and you want to pull those back in later. So, to fix this situation, what you need to do is to copy the /etc/lvmtab and any files in /etc/lvmconf from your “skeleton” installation (where you did all the cloning work) and onto the /new/etc directory where your clone is being worked on. This way, you have a valid LVM disk layout when the image boots up and it's not going to be looking, for example, for “vg01” which did not previously exist before you cloned the box.
# Archive your old LVM configuration. Do not delete it. # you might want those files later on for restoring # other non-root LVM groups cd /new/etc mv lvmtab old.lvmtab mv lvmconf old.lvmconf mkdir lvmconf chown root:bin lvmconf chmod 755 lvmconf cp /etc/lvmtab /new/etc cp -r /etc/lvmconf /new/etc/
Also, make sure your target clone disks have the LVM directory for the group you used (probably vg01) available in /dev. Remember that 'mknod' and directory creation we had to do? Well, you're clone didn't get that process and will need that as well. Also check if there are any prototype “lvol” or other logical volume devices there you didn't make. Those might be stale and left over from the backup image. It never hurts to wipe those out and then just copy over the ones from the skeleton system.
cd /new/dev # Check for any left over LV DSF's ls /dev/vg01 # If the directory isn't there, make it and also make the # "group" control file. mkdir vg01 # Make sure the "N" in the hex address is the same # as what you used earlier. Ie.. probably "1". It definitely # cannot be 0, as that is reserved for vg00 only. mknod group c 64 0x0N0000 # Alternatively you can just preserve the old devices and create new ones # that you know will work. Just don't use a stale vg directory. mv vg01 old.vg01 cp -r /dev/vg01 /new/dev/vg01
Bootpath and Final Thoughts
If you were careful and did all the steps correctly and in the proper order, then the system should be bootable off this new disk. You will see a warning in your startup about the old volume group (source VG) if you didn’t do any cleanup of /etc/lvmtab and the /etc/lvmconf directory, which isn’t needed (but can save you the warning if you know what you are doing). This only happens when your original disk is no longer accessible to the clone.
Keep in mind when SAN booting that most ISL environments will not present SAN disk paths to you as bootable, even if in fact they are. This is to save time since bus scanning and LUN enumeration can be time consuming on SAN environments with hundreds of LUNs. So, if you want to change to a SAN device you may need to note down the hardware path (See output from ioscan -FunC disk) which your cloned SAN boot disk resides on. Set this directly as on your HPUX pre-boot firmware (also sometimes called the BCH menu which is the first thing most PA-RISC systems show interactively while booting).
# Refresh your memory about the disks in the system esp the root disks in the chassis # remember the disk path is buried in there and will look something like "0/1/1/0.1.0" ioscan -FunC disk # Now see what the system is setup to boot on currently. setboot # set the primary and alternate boot paths that will appear in the BCH menu # These are just examples, make sure to get your actual info from ioscan output bootpath PRI 4/0/2/0/0.10.0 bootpath ALT 4/0/1/0/0.10.0 # Set the primary boot path setboot -p "0/1/1/0.1.0" # Set the alternate boot path to your skeleton # install in case things go wrong. Then you can still boot the other environment. setboot -a "0/1/1/1.0.0"
Troubleshooting Common Problems
There are a number of common issues when cloning.
- Issues with networking not coming up and weird errors from ifconfig or netstat saying “Operation not supported” or other weirdness when trying to add an IP or normally run ifconfig in HP-UX. What you need to do is check to make sure /usr/bin/autopush is present, not a dead link, and not a zero byte file. For whatever reason this file has often been trashed for me on different HP-UX systems. I have to find it from another box or from a backup image and restore the binary or fix the link. The error you'd see is: ifconfig : socket : Operation not supported
- Weird problems related to links in /opt/star-ncf-prod. This was some kind of patch which was installed on late model 11.11 machines. It's all kinda problematic. It has several soft links to binaries in /usr/bin and libraries in /usr/lib. One of these libraries is called libxti.2 and it's required bin the /bin/login program to get anyone logged into the system. This symlink goes to /opt/star-ncf-prod/ep_patch/usr/lib/libxti.2 and it's easily broken or screwed up. You'll probably need to boot to single user mode and either restore the library to /usr/lib manually, fix the link, or whatever makes “/usr/ccs/bin/ldd /usr/bin/login” show every library without “not found”, especially libxti.2.