FAL[client, ARC0]: Error 12154 for fetching gap sequence

Recently I came across the following error :
ERROR: failed to establish dependency between database ps1 and diskgroup resource ora.FRA.dg
Error 12154 received logging on to the standby
FAL[client, ARC0]: Error 12154 connecting to s1 for fetching gap sequence
Error 12154 received logging on to the standby
FAL[client, ARC0]: Error 12154 connecting to s2 for fetching gap sequence
Lets have a look to see what has happened and resolve.
1) If we log onto the standby and run the following we can see the gap.

SQL> select *
  2  from v$archive_gap;
---------- ------------- --------------
         1         74298          74419
         2         67767          67844

1b) Also as a different example you could log onto the ASM standby machine as Oracle and have a look at the archive logs and see if there is any missing.

cd FRA

If we look at the list we can identify missing log, it is in sequential order ls78live_1 is thread 1, ls78live_2 is thread 2., We can see below ls78live_1_144812_704059285.arc is missing.


2) On Primary database retrieve the missing archive log files based on the information we got from v$archive_gap.

rman target sys/** catalog rcuser/**@rmcatalog
RMAN> run
{allocate channel c1 type disk;
backup archivelog from sequence 74298 until sequence 74299 thread 1 format '/database/backup/s78live/gap/arch_1_%U';
backup archivelog from sequence 67767 until sequence 67768 thread 2 format '/database/backup/s78live/gap/arch_2_%U';

If the archive logs have been already removed you will get the following error.

Starting backup at 22-JUL-16
specification does not match any archived log in the repository
backup cancelled because there are no files to backup
Finished backup at 22-JUL-16.

In this case you will need to look through your backed up archive logs.
3) On Standby – lets restore the archive files

RMAN> run
{catalog start with '/database/backup/s82live/arch/1dayold';
allocate channel c1 type disk ;
allocate channel c2 type disk ;
restore archivelog from sequence 74298 until sequence 74299 thread 1;
restore archivelog from sequence 67767 until sequence 67768 thread 2;

12) We now enable the recovery of the standby.

SQL> sqlplus / as sysdba
SQL> alter database recover managed standby database using current logfile disconnect from session;

We have identified the gap, found the missing archive log’s, copied them to the standby and re-enabled the dataguard.


OTN Appreciation Day: ASM

First thing I have to say, Thanks Tim for this great idea. Tim Hall
I have to say one of my favorite features has to be the Oracle ASM.
Oracle ASM is the storage/file system where you store your data files and other database files. One of the great advantages I like is that you can add or remove disks without any downtime! To give you a great example is that a few months back I had to change the disks that our ASM was using to faster ones, we were able to add these new disks to the ASM and remove the old disks without any system outage.
A blog post I had written earlier show’s how this work’s – Adding New ASM Disks
If you don’t use ASM, I would be asking why!


POUG – 1st Polish International Oracle Conference

Amazing, Awesome some of the words that come to my mind. I wrote this on the plane but I am just posting it;

I have just landed back into the UK and I am still thinking about this conference. My journey began on Thursday, I was flying into Warsaw and my plan was delayed, I really did not want to miss the speaker dinner, a great opportunity to meet everyone. I managed to arrive in sufficient time. I checked in at the hotel and went to meet the rest of the group at the bar.

At the bar was Martin, Neil and Jim, these guys rock ;). Had a quick drink then met the rest of the group at reception?

It was great to meet all the speakers. I have to say we all were well looked after by the POUG organizers, we were taken to a restaurant for an amazing meal and thereafter a little stroll/sightseeing for the few who had lost all direction and sense; we ended up at Irish bar for a night cap.

(Day 1). There was a really good turnout easily I would say 150+ and some had travelled from the borders of Poland. There were 2 tracks running, I went to Jim’s talk DBA, Heal Thyself: Five diseases of IT Organizations. It was my first time at Jim’s talk, it was superbly delivered after I had left I decided to polish up on my slides for later. I missed Heli Helskyaho’s @helifromfinland talk about SQL developer that I really wanted to go to, I heard it was really good. I will catch you next time.

After lunch I went to Pieter’s @vanpupi and Philippe’s @pfierens talk about Oracle VM on Exadata, this was good as well, learnt some things that you will not find in any documents. Straight after this talk was my presentation. I was told it would be HDMI connection, but there was only a VGA cable (Panic, Panic, Panic) – luckily Philippe let me borrow his adapter as well as a ‘Powerpoint Pointer’. Thank You Philippe.

My talk was about extraction of cloud data to a premise BI and monitoring. The room was pretty full, that was a great sign, never really been nervous too much before but I was this time – as the talk progressed we moved to Cloud security and monitoring in the cloud as there was a great interest in this area. After my talk had finished we still had around 5m so we discussed the cloud aspects in more detail. I know there were some more questions on the monitoring side, if anybody did want any info, do get in contact me. This is another great side of user groups, you can go to one talk and ask other topic questions that may come to your mind.

Thereafter I sat into Robin’s @rmoff talk about OBIEE 12c Performance, I learnt some new things that will be great for me at work. I have to say you can never know everything there is always thing to learn.

After this talk we were whisked to the POUG Appreciation event as I would call it. They really looked after the speakers and attendees.

(Day 2) – It was a long night, and it was so amazing to see so many people the next day for Neil’s @chandlerDBA talk on ‘Why has my plan changed’. Being a man with many hats at my work and being a DBA for over 17 years, I enjoyed Neil’s talk as I am always fighting SQL performance looking forward to see his scripts. I would definitely recommend it, if you are at a different conference and he is talking – don’t miss itJ.

After Neil’s talk I decided to stay in this stream for Joze’s talk on ‘Opening the black box called “Cost Based Optimizer’, he’s right it is a black box. Thereafter I moved to Martin’s @MDWildlake talk ‘Tips on Bulk Data Processing with SQL and PL/SQL’ – It was a great insight in bulk processing data and this man like’s his beer 😉

Once we had consumed our lunch we had our final session of the day ‘#DBADEV,Bridging the gap between development and operation table’ This was a panel session with Sabine @oraesque, Martin @MDWidlake, Philippe @pfierens, Neil @ChandlerDBA, Piet @pdevisser and Erik @evrocs_nl. I think most people would be able to say something in this area, I am always fighting with developersJ. What a great way to end the conference.

All my breakfasts, lunch’s were timed with Pieter’s – Absolute Legend J. At the last breakfast Pieter, Eric and myself were talking how good the conference was and how they have set the bar really high, it is very high!. It can only get better ;). I can see this conference growing huge, from talking to some of the guests they would like talks on EM Monitoring, Middleware, APEX. Maybe next year have a few of these 😉

This was a super conference venue, food, equipment and the beer, excellently organized and delivered, it was great to meet all the speakers, organizers and guests. POUG is definitely now on the map. It was bloody marvelous 😉