ora-00600 при накате архивлога на стендбае

Доброго времени суток!

Может кто подскажет что делать - при накате очередного архивлога получаю нижеследующую ошибку
При этом БД в read only открывается без проблем



Wed Sep 16 19:14:06 2015
ORA-279 signalled during: ALTER DATABASE RECOVER standby database ...
ALTER DATABASE RECOVER CONTINUE DEFAULT
Media Recovery Log /oradata6/orcl/archivelog/1_336524_849260061.dbf
Errors in file /oradata1/orcl/diag/rdbms/orcl/orcl/trace/orcl_pr00_48365896.trc (incident=1168361):
ORA-00600: internal error code, arguments: [3635], [362], [], [], [], [], [], [], [], [], [], []
Incident details in: /oradata1/orcl/diag/rdbms/orcl/orcl/incident/incdir_1168361/orcl_pr00_48365896_i1168361.trc


в Трейсе:




*** 2015-09-16 19:16:56.885
*** SESSION ID:(1153.7) 2015-09-16 19:16:56.885
*** CLIENT ID:() 2015-09-16 19:16:56.885
*** SERVICE NAME:() 2015-09-16 19:16:56.885
*** MODULE NAME:() 2015-09-16 19:16:56.885
*** ACTION NAME:() 2015-09-16 19:16:56.885

Started Parallel Media Recovery
*** 2015-09-16 19:16:56.899 4329 krsh.c
Managed Standby Recovery not using Real Time Apply
Dumping database incarnation table:
Resetlogs 0 scn and time: 0x0000.000e20dc 06/03/2014 09:34:21
Recovery target incarnation = 2, activation ID = 1377034138
Influx buffer limit = 100000 min(50% x 1297280, 100000)

*** 2015-09-16 19:17:01.004
Start recovery at thread 1 ckpt scn 9793925534284 logseq 0 block 0
Initial buffer sizes: read 1024K, overflow 832K, change 805K

*** 2015-09-16 19:17:01.027
Media Recovery add redo thread 1
DDE rules only execution for: ORA 312
----- START Event Driven Actions Dump ----
---- END Event Driven Actions Dump ----
----- START DDE Actions Dump -----
Executing SYNC actions
----- START DDE Action: 'DB_STRUCTURE_INTEGRITY_CHECK' (Async) -----
Successfully dispatched
----- END DDE Action: 'DB_STRUCTURE_INTEGRITY_CHECK' (SUCCESS, 0 csec) -----
Executing ASYNC actions
----- END DDE Actions Dump (total 0 csec) -----
ORA-00341: log 7 of thread 1, wrong log # 1 in header
ORA-00312: online log 7 thread 1: '/oradata1/orcl/stb001.log'

*** 2015-09-16 19:17:02.819
krr_open_logfile: adjusting krrxnxr from 0x000000.00000000.0000 to 0x05228c.000d5519.0010

*** 2015-09-16 19:17:02.885
Incident 1168241 created, dump file: /oradata1/orcl/diag/rdbms/orcl/orcl/incident/incdir_1168241/orcl_pr00_61997400_i1168241.trc
ORA-00600: internal error code, arguments: [3635], [362], [], [], [], [], [], [], [], [], [], []

----- Redo read statistics for thread 1 -----
Read rate (ASYNC): 396Kb in 4.00s => 0.10 Mb/sec
Total redo bytes: 396Kb Longest record: 0Kb, moves: 0/1 moved: 0Mb (0%)
Longest LWN: 0Kb, reads: 2
Last redo scn: 0x08e8.5373d24c (9793925534284)
Change vector header moves = 0/1 (0%)
----------------------------------------------

*** 2015-09-16 19:17:05.024
Media Recovery drop redo thread 1

*** 2015-09-16 19:17:05.135
Completed Media Recovery
Checking to start in-flux buffer recovery from SCN 2280.1400099404 to SCN (non-inclusive) 2280.1400099404
ORA-00283: recovery session canceled due to errors
ORA-00600: internal error code, arguments: [3635], [362], [], [], [], [], [], [], [], [], [], []

-bash-4.2$ cat /oradata1/orcl/diag/rdbms/orcl/orcl/trace/orcl_pr00_45023480.trc
Trace file /oradata1/orcl/diag/rdbms/orcl/orcl/trace/orcl_pr00_45023480.trc
Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 - 64bit Production
With the Partitioning, OLAP, Data Mining and Real Application Testing options
ORACLE_HOME = /u01/app/oracle/product/11.2.0/dbhome_1
System name: AIX
Node name: IBMCH-node3
Release: 1
Version: 7
Machine: 00F90D6A4C00
Instance name: orcl
Redo thread mounted by this instance: 1
Oracle process number: 32
Unix process pid: 45023480, image: oracle@IBMCH-node3 (PR00)


*** 2015-09-16 19:28:41.776
*** SESSION ID:(1537.1) 2015-09-16 19:28:41.776
*** CLIENT ID:() 2015-09-16 19:28:41.776
*** SERVICE NAME:() 2015-09-16 19:28:41.776
*** MODULE NAME:() 2015-09-16 19:28:41.776
*** ACTION NAME:() 2015-09-16 19:28:41.776

Started Parallel Media Recovery
*** 2015-09-16 19:28:41.789 4329 krsh.c
Managed Standby Recovery not using Real Time Apply
Dumping database incarnation table:
Resetlogs 0 scn and time: 0x0000.000e20dc 06/03/2014 09:34:21
Recovery target incarnation = 2, activation ID = 1377034138
Influx buffer limit = 100000 min(50% x 1297280, 100000)

*** 2015-09-16 19:28:45.589
Start recovery at thread 1 ckpt scn 9793925534284 logseq 0 block 0
Initial buffer sizes: read 1024K, overflow 832K, change 805K

*** 2015-09-16 19:28:45.642
Media Recovery add redo thread 1
KRR: Increase log read buffers 8->10 for 8Mb LWN
KRR: Increase log read buffers 10->11 for 9Mb LWN
KRR: Increase log read buffers 11->15 for 13Mb LWN
KRR: Increase log read buffers 15->41 for 39Mb LWN
KRR: Increase log read buffers 41->80 for 78Mb LWN

*** 2015-09-16 19:28:46.669
KRR: Increase log read buffers 80->87 for 85Mb LWN
Incident 1200310 created, dump file: /oradata1/orcl/diag/rdbms/orcl/orcl/incident/incdir_1200310/orcl_pr00_45023480_i1200310.trc
ORA-00600: internal error code, arguments: [3635], [362], [], [], [], [], [], [], [], [], [], []

----- Redo read statistics for thread 1 -----
Read rate (ASYNC): 437247Kb in 3.22s => 132.61 Mb/sec
Total redo bytes: 437272Kb Longest record: 32Kb, moves: 426/16925 moved: 13Mb (2%)
Longest LWN: 87964Kb, reads: 248
Last redo scn: 0x08e8.5373d24c (9793925534284)
Change vector header moves = 453/19277 (2%)
----------------------------------------------

*** 2015-09-16 19:28:48.860
Media Recovery drop redo thread 1

*** 2015-09-16 19:28:48.939
Completed Media Recovery
Checking to start in-flux buffer recovery from SCN 2280.1400099404 to SCN (non-inclusive) 2280.1400099404
ORA-00283: recovery session canceled due to errors
ORA-00600: internal error code, arguments: [3635], [362], [], [], [], [], [], [], [], [], [], []
-bash-4.2$

Ora-600 - это внутренняя ошибка
Честно помочь может только oracle support

То что открывается в Read only - нормально, бо проблема при накате standby log

ORA-00341: log 7 of thread 1, wrong log # 1 in header
ORA-00312: online log 7 thread 1: '/oradata1/orcl/stb001.log'

в группе одна копия?
если да, то копируйте другого члена группы вместо битого,
если нет, то Можно попробовать :

1) Сделать backup prod
Остановить standby database

alter database recover managed standby database cancel;

Накатить archive log ( а не standby log ) Если получится
Нужно смотреть что там на primary, че то кажется, это ЖЖЖ не спроста.
Если только не дисковый/сетевой сбой

alter database recover standby database until cancel;

попробовать сделать clear

select group# from v$logfile where name='/oradata1/orcl/stb001.log';
ALTER DATABASE CLEAR LOGFILE GROUP ###

Если не поможет, попробовать пересоздать standby группу

Vadim,
спасибо за совет

но то, что в первом посте ругачка на stb001.log - это я пытался его убрать , потом вновь вернул
и во втором моем посте как раз уже с этим stb001.log нет ошибок

Покажите вывод:

select hxfil file#,fhscn checkpoint_change#, fhafs Absolute_Fuzzy_SCN from x$kcvfh;