Categories
Blog

Oracle Cloud PAAS Machine – Drives Missing – Backup Failing

Problem:

At some point the PAAS machine was rebooted and the drives for the middleware instance did not remount which caused the following error when the backups were taking place:



Further investigation, once connected to the server, you will notice the drives have disappeared from the machine:



And notice errors in the /var/log/messages.



Log onto the machine as root and run lsblk and take note of the vg_binaries with no lvm  :

# lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
xvdb 202:16 0 21G 0 disk
├─xvdb1 202:17 0 500M 0 part /boot
├─xvdb2 202:18 0 4G 0 part [SWAP]
└─xvdb3 202:19 0 16G 0 part /
xvdc 202:32 0 14G 0 disk
├─vg_binaries-lv_tools (dm-1) 252:1 0 10G 0 lvm /u01/app/oracle/tools
├─vg_binaries-lv_mw (dm-2) 252:2 0 2G 0 lvm
└─vg_binaries-lv_jdk (dm-3) 252:3 0 2G 0 lvm
xvdd 202:48 0 6G 0 disk
└─vg_domains-lv_domains (dm-0) 252:0 0 6G 0 lvm

Mount them manually, once the mount point has been established.

# mount /dev/mapper/vg_domains-lv_domains /u01/data/otd-instance 
# mount /dev/mapper/vg_binaries-lv_mw /u01/app/oracle/middleware
# mount /dev/mapper/vg_binaries-lv_jdk /u01/jdk

Start any services you had on the machine.
So currently if the machine reboots, we have to manually mount the drives, until the RCA has been generated.
If you do wish to start OTD manually as the Oracle user :

$ export JAVA_HOME=/u01/jdk
$ /u01/data/otd-instance/admin-server/bin/startserv