Oracle упал

oleg_dov
Дата: 11.12.2007 19:00:31
Возникла критическая проблема:
1. Во время работы при попытки апдейтов начали выскакивать ошибки оракла End of communication channel ну и потом в алертлоге вот что появилось
автор
Tue Dec 11 16:54:14 2007
Errors in file /u01/app/oracle/admin/orcl/udump/orcl_ora_17620.trc:
ORA-00600: internal error code, arguments: [17147], [0xB75F86C4], [], [], [], [], [], []
Tue Dec 11 16:54:14 2007
sdtcs_getexecname: ignored overriding value [oracleorcl]
Tue Dec 11 16:54:17 2007
Errors in file /u01/app/oracle/admin/orcl/udump/orcl_ora_17620.trc:
ORA-00600: internal error code, arguments: [17183], [0xB75F46C4], [], [], [], [], [], []
ORA-00600: internal error code, arguments: [17147], [0xB75F86C4], [], [], [], [], [], []
sdtcs_getexecname: ignored overriding value [oracleorcl]

2. потом оракле через несколько минут практически остановился
автор
Tue Dec 11 17:24:17 2007
Errors in file /u01/app/oracle/admin/orcl/bdump/orcl_pmon_3623.trc:
ORA-00607: Internal error occurred while making a change to a data block
ORA-00602: internal programming exception
ORA-07445: exception encountered: core dump [kdr9ir2rst0()+34] [SIGSEGV] [Address not mapped to object] [0xBF3FFFFC] [] []
Tue Dec 11 17:24:18 2007

и я так понял дамп начал ваять в трейсфайле
3. я попробовал зашатдаунить базу, и убрать трейс файл в который дамп пишется
ну и вот чем закончилось
автор
Tue Dec 11 17:34:31 2007
Errors in file /u01/app/oracle/admin/orcl/bdump/orcl_smon_3633.trc:
ORA-07445: exception encountered: core dump [kdr9ir2rst()+67] [SIGSEGV] [Address not mapped to object] [0xBF3FFFFC] [] []
Tue Dec 11 17:34:31 2007
Stopping background process MMON
Tue Dec 11 17:34:32 2007
Shutting down instance (immediate)
License high water mark = 179
Tue Dec 11 17:34:32 2007
Stopping Job queue slave processes
Tue Dec 11 17:34:32 2007
Job queue slave processes stopped
Tue Dec 11 17:39:31 2007
Active call for process 13114 user 'oracle' program 'oracleorcl@LS202'
Active call for process 17152 user 'oracle' program 'oracleorcl@LS202'
SHUTDOWN: waiting for active calls to complete.
Tue Dec 11 17:44:32 2007
Process 0x0xaf73218c appears to be hung while dumping
Current time = 671612325, process death time = 671552163 interval = 60000
Attempting to kill process 0x0xaf73218c with OS pid = 3633
OSD kill succeeded for process 0xaf73218c
Tue Dec 11 17:44:33 2007
Errors in file /u01/app/oracle/admin/orcl/bdump/orcl_pmon_3623.trc:
ORA-00474: SMON process terminated with error
PMON: terminating instance due to error 474
Instance terminated by PMON, pid = 3623

4. Стартанул опять базу, но результат не утешает
автор
ARCH: Connecting to console port...
Tue Dec 11 17:48:23 2007
ARC1: Creating local archive destination LOG_ARCHIVE_DEST_1: '/var/oracle/archieve1/1_246309_584363323.dbf' (thread 1 sequenc
e 246309)
(orcl)
ARCH: Connecting to console port...
Tue Dec 11 17:48:23 2007
SMON: enabling cache recovery
Tue Dec 11 17:48:23 2007
ARC1: Closing local archive destination LOG_ARCHIVE_DEST_1: '/var/oracle/archieve1/1_246309_584363323.dbf'
(orcl)
Tue Dec 11 17:48:23 2007
ARCH: Connecting to console port...
Committing creation of archivelog '/var/oracle/archieve1/1_246309_584363323.dbf'
Tue Dec 11 17:48:23 2007
ARC1: Completed archiving thread 1 sequence 246309 (682794706-682816080) (orcl)
ARCH: Connecting to console port...
Tue Dec 11 17:48:24 2007
Successfully onlined Undo Tablespace 39.
Tue Dec 11 17:48:24 2007
SMON: enabling tx recovery
Tue Dec 11 17:48:24 2007
Database Characterset is AL32UTF8
Tue Dec 11 17:48:24 2007
Errors in file /u01/app/oracle/admin/orcl/bdump/orcl_p007_22429.trc:
ORA-07445: exception encountered: core dump [kdr9ir2rst()+29] [SIGSEGV] [Address not mapped to object] [0xBF3FFFFC] [] []
Tue Dec 11 17:48:24 2007
Starting background process QMNC
QMNC started with pid=23, OS id=22431
Tue Dec 11 17:48:25 2007
replication_dependency_tracking turned off (no async multimaster replication found)
Tue Dec 11 17:48:25 2007
Starting background process MMON
Starting background process MMNL
MMON started with pid=24, OS id=22433
MMNL started with pid=25, OS id=22435
Tue Dec 11 17:48:25 2007
Completed: ALTER DATABASE OPEN
Tue Dec 11 17:48:28 2007
db_recovery_file_dest_size of 8192 MB is 0.54% used. This is a
user-specified limit on the amount of space that will be used by this
database for recovery-related files, and does not reflect the amount of
space available in the underlying filesystem or ASM diskgroup.

база вроде как стартанула, но orcl_smon_22397.trc растет - дамп делается
Подскажите, чем это грозит и стоит ли пустить пользоватлеей в базу?????
Спасибо.
oleg_dov
Дата: 11.12.2007 19:08:28
Ну сервер сам погас
автор
Process 0x0xaf736d7c appears to be hung while dumping
Current time = 671695469, process death time = 671635468 interval = 60000
Attempting to kill process 0x0xaf736d7c with OS pid = 22429
OSD kill succeeded for process 0xaf736d7c
Doing block recovery for file 17 block 1365399
Doing block recovery for file 17 block 1365399
Doing block recovery for file 17 block 1365399
Doing block recovery for file 17 block 1365399
Doing block recovery for file 17 block 1365399
Doing block recovery for file 17 block 1365399
Doing block recovery for file 17 block 1365399
Doing block recovery for file 17 block 1365399
Block recovery range from rba 246310.80.0 to scn 0.682816356
Tue Dec 11 17:58:28 2007
Recovery of Online Redo Log: Thread 1 Group 2 Seq 246310 Reading mem 0
Mem# 0 errs 0: /disk3/oracle/e810/onlinelog/redo_g21.log
Mem# 1 errs 0: /disk2/ORCL/onlinelog/redo_g22.log
Block recovery completed at rba 246310.1102.16, scn 0.682816357
Tue Dec 11 17:58:28 2007
SMON: slave died unexpectedly, downgrading to serial recovery
Tue Dec 11 17:58:28 2007
Errors in file /u01/app/oracle/admin/orcl/bdump/orcl_smon_22397.trc:
ORA-07445: exception encountered: core dump [kdr9ir2rst()+18] [SIGSEGV] [Address not mapped to object] [0xBF3FFFFC] [] []
Tue Dec 11 18:08:28 2007
Process 0x0xaf73218c appears to be hung while dumping
Current time = 671755891, process death time = 671695877 interval = 60000
Attempting to kill process 0x0xaf73218c with OS pid = 22397
OSD kill succeeded for process 0xaf73218c
Tue Dec 11 18:08:31 2007
Errors in file /u01/app/oracle/admin/orcl/bdump/orcl_pmon_22387.trc:
ORA-00474: SMON process terminated with error
PMON: terminating instance due to error 474
Tue Dec 11 18:08:31 2007
Errors in file /u01/app/oracle/admin/orcl/bdump/orcl_lgwr_22393.trc:
ORA-00474: SMON process terminated with error
Instance terminated by PMON, pid = 22387

Попомиге pls разобраться с проблемой!!!!
Жесть
Дата: 11.12.2007 19:19:30
Операционная система какая? Перегружай сервер.
Жестяная банка
Дата: 11.12.2007 19:25:53
тема ОС останется нераскрытой?
oleg_dov
Дата: 11.12.2007 19:27:36
Да щас пустил ребут системы.
Операционка Red Hat Enterprise Linux 4
Жесть
Дата: 11.12.2007 19:30:03
oleg_dov
Да щас пустил ребут системы.
Операционка Red Hat Enterprise Linux 4

Согласно Note:138580.1, вроде как, какие-то проблемы с памятью, может так и есть.
Жесть
Дата: 11.12.2007 19:35:50
(доставая из кармана пассатижи и шлем для телепатии). А версия БД - не секрет?
oleg_dov
Дата: 11.12.2007 19:41:18
1. к металинку доступа нет :( но очень хочется глянуть что описание
2. версия базы не секрет, просто торможу щас - напрг с базой, думать тяжко :)
10.0.1.0.4
oleg_dov
Дата: 11.12.2007 19:42:29
опять торможу
1. машину рестартанул, базу стартанул - правда не уверен что все ок
хоят в логах нормально и больше трейс файлы с дампом вроде не клепает
2. еще б выяснить что за фигня случилась
Консерва
Дата: 11.12.2007 19:43:14
А у Вас файлы autoextensible, да?