11.10.2008

Parameter file, Password file and Alert log

1. Parameter file
Parameter file Contains a list of initialization parameters that control how certain aspects of the database instance behave. Most parameters can be modified without restarting the instance, while some require that it be shut down and restarted to pick up the new values. Also known as the initialization file.

2. Password file
Password file Stores passwords in an encrypted format for certain privileged users who are allowed to start and stop the database. Normally, when users connect to the database, their privileges are verified from within the database itself. Since the database is unavailable when it is not running, a privileged user who is starting the database must be authenticated by some other means. The password file provides this capability by storing the privileged user’s password outside of the database.

3. Alert log:
Alert log Records all internal errors and information about several database operations. The DBA should check the alert log at least once a day. The alert log entry briefly describes an error and provides the name of a trace file, which contains more detailed information regarding the internal errors.

Control files

The control files record crucial information such as datafile locations, synchronization information, and backup and recovery information. Without these files, the database cannot start up. If anything happens to the control files while the database is running, the database will crash.

Datafiles and Redolog files

The datafiles are files that hold information, whereas the redo log files are files that record all changes made to the data. In the event of file corruption or disk failure.
The DBA can restore datafiles from a backup copy, and then apply the changes listed in the redo log to bring the backup datafiles up to date.

Database components

Oracle Database 10g contains two major components:
1. The physical database and
2. The Oracle instance.

The physical database consists of the files that physically reside on the system, such as datafiles, redo log files, and control files.

The Oracle instance is made up of a large memory area and a set of background processes. The instance provides a means for users to connect to the database to retrieve, add, delete, and modify data.

Some Other Components:
a. Oracle Executables and Library Files
Oracle executables and library files are not considered part of the database.
b. Multiple Instances
Each database will usually have only a single instance, but on some platforms, the DBA can configure multiple instances of a single database.

10.22.2008

AWR from OEM

AWR (Automatic Workload Repository):

What is AWR?

AWR reports collects database performance statistics and metrics for analysis and tuning, shows the exact time spent in the database, and even saves session information.

What is default interval period between two awr report ?

By default, AWR automatically generates snapshots of the performance data once every hour and retains the statistics in the workload repository for 7 days.

GUI ----> OEM (Oracle Enterprise Manager)
Login to OEM ---> Administration Tag ----> Workload Section --->AUTOMATIC WORKLOAD REPOSITORY

The data in the snapshot interval is then analyzed by the Automatic Database Diagnostic Monitor (ADDM).

How to Manage & change interval period for awr report ?

There is two to modify or changes in AWR report.
1. Manually ---> through DBMS_WORKLOAD_REPOSITORY plsql package

2. GUI ----> OEM (Oracle Enterprise Manager)
Login to OEM ---> Administration Tag ----> Workload Section --->AUTOMATIC WORKLOAD REPOSIT

How to Interpreting with AWR report ?

1. Load Profile

Here we know about Logical Read, Physical Read, Soft Parse, Hard Parse

2. Instance Efficiency Percentages
If your instance percentage show BUFFER CAHCE HIT RATIO is 100% it is not sign for your database is very fast and running smootly.


3. Top 5 Timed Events
It is very important section in AWR report.
through this we can know most five wait event is effecting database performance.

4. SQL Statistics
It this section we will know about Execution time, cpu used, logical & physical read for top most sql statement.


5.Advisory Statistics
In this section we will get advice for PGA, BUFFER CACHE, SHARED POOL, JAVA POOL size for better performance.

11. Find the value of Bind variable

Select NAME,POSITION,DATATYPE_STRING,VALUE_STRING from v$sql_bind_capture
where sql_id='d9kf91muzy2wq';

10. Find connected session,login_time in a given time

SELECT
a.sid,a.serial#,a.username,a.osuser,a.machine
,a.service_name
,to_char(a.logon_time, 'hh24:mi dd/mm/yy') login_time
FROM
v$session a,v$process b
WHERE
a.username IS NOT NULL
AND
a.paddr=b.addr
AND a.type='USER'
AND a.logon_time > to_date ('2008-10-18 15:00:00','YYYY-MM-DD HH24:MI:SS')
AND a.logon_time <
to_date ('2008-10-19 15:00:00','YYYY-MM-DD HH24:MI:SS')
Order by a.sid,a.username;

9. Find the SID

Set linesize 200
column USERNAME format a10
column MACHINE format a20
column OSUSER format a10
column SERVICE_NAME format a35



SELECT
a.sid,a.serial#,b.spid,b.pid,a.username,a.osuser,a.machine
FROM
v$session a,v$process b
WHERE
a.username IS NOT NULL
AND
a.paddr=b.addr
Order by a.sid,a.username;

8. Time related DBA queries/scripts

1. Which SQL taking more CPU time

SELECT SQL_TEXT,X.CPU_TIME FROM DBA_HIST_SQLTEXT DHST,
(SELECT DHSS.SQL_ID SQL_ID,SUM(DHSS.CPU_TIME_DELTA) CPU_TIME
FROM DBA_HIST_SQLSTAT DHSS
WHERE DHSS.SNAP_ID IN(SELECT SNAP_ID FROM DBA_HIST_SNAPSHOT
WHERE BEGIN_INTERVAL_TIME>=TO_DATE('10/16/2008','MM/DD/YYYY')
AND END_INTERVAL_TIME<=TO_DATE('10/18/2008','MM/DD/YYYY')) GROUP BY DHSS.SQL_ID) X WHERE X.SQL_ID=DHST.SQL_ID ORDER BY X.CPU_TIME DESC;



7. Wait related DBA queries..



SELECT distinct wait_class#, wait_class FROM v$event_name ORDER BY wait_class#;

SELECT wait_class_id, wait_class#, wait_class, total_waits, time_waited
FROM v$system_wait_class
order by time_waited desc;

6. What object is currently causing the highest resource waits?

SQL> column OBJECT_NAME format a30
SQL> column EVENT format a30

select dba_objects.object_name,
dba_objects.object_type,
active_session_history.event,
sum(active_session_history.wait_time +
active_session_history.time_waited) ttl_wait_time
from v$active_session_history active_session_history,
dba_objects
where active_session_history.sample_time between sysdate - 60/2880 and sysdate
and active_session_history.current_obj# = dba_objects.object_id
group by dba_objects.object_name, dba_objects.object_type, active_session_history.event
order by 4 desc;

5. What SQL is currently using the most resources?

select active_session_history.user_id,
dba_users.username,
sqlarea.sql_text,
sum(active_session_history.wait_time +
active_session_history.time_waited) total_wait_time
from v$active_session_history active_session_history,
v$sqlarea sqlarea,
dba_users
where active_session_history.sample_time between sysdate - 60/2880 and sysdate
and active_session_history.sql_id = sqlarea.sql_id
and active_session_history.user_id = dba_users.user_id
group by active_session_history.user_id,sqlarea.sql_text, dba_users.username
order by 4 desc;

4. What user is waiting the most?

Select sesion.sid,
sesion.username,
sum(active_session_history.wait_time +
active_session_history.time_waited) total_wait_time
from v$active_session_history active_session_history,
v$session sesion
where active_session_history.sample_time between sysdate - 60/2880 and sysdate
and active_session_history.session_id = sesion.sid
group by sesion.sid, sesion.username
order by 3;

3. Last 30 minutes result those resources that are in high demand on your system.

select active_session_history.event,
sum(active_session_history.wait_time +
active_session_history.time_waited) total_wait_time
from v$active_session_history active_session_history
where active_session_history.sample_time between sysdate - 60/2880 and sysdate
group by active_session_history.event
order by 2;

2. Identify heavy SQL (Get the SQL with heavy DISK_READS)

Select sql_text ,executions ,disk_reads ,buffer_gets
from v$sqlarea
where decode(executions ,0,disk_reads,disk_reads/executions)
> (select avg(decode(executions,0,disk_reads,disk_reads/executions))
+ stddev(decode(executions,0,disk_reads,disk_reads/executions))
from v$sqlarea)
and parsing_user_id !=3D;

1. Identify heavy SQL (Get the SQL with heavy BUFFER_GETS)

Select sql_text ,executions ,disk_reads ,buffer_gets
from v$sqlarea
where decode(executions,0,buffer_gets,buffer_gets/executions)
> (select avg(decode(executions,0,buffer_gets,buffer_gets/executions))
+ stddev(decode(executions,0,buffer_gets,buffer_gets/executions))
from v$sqlarea) and parsing_user_id !=3D;

10.20.2008

Difference between Host variable and Bind Variable

HOST VARIABLES:
  • Host variables are the key to communication between your host program and Oracle.
  • Typically, a precompiler program inputs data from a host variable to Oracle, and Oracle outputs data to a host variable in the program.
  • Oracle stores input data in database columns, and stores output data in program host variables.

BIND VARIABLES:
  • A host variable can be any arbitrary C expression that resolves to a scalar type. But, a host variable must also be an lvalue. Bind variables are used in SQL and PL/SQL statements for holding data or result sets.
  • They are commonly used in SQL statements to optimize statement perrformance.
  • A statement with a bind variable may be re-executed multiple times without needing to be re-parsed.
  • Their values can be set and referenced in PL/SQL blocks.
  • They can be referenced in SQL statements bind variable references should be prefixed with a colon

Execute to parse ratio

Execute to parse ratio:

Execute to parse ratio is a measure of how many times you execute a sql statement versus parse it.
This 'ratio' will go towards 100 as the number of executes goes up and up and up, while the number of parses remains the same.
This will go to zero as the number of parses and executes are equal.
This will go negative if you parse more than you execute.

That percentage is computed as: round(100*(1-:prse/:exe),2)

Your developers have all of the control here - we cannot change this ratio without touching the way the application interacts with the database.

When Execute to Parse values is too low:

That means, someone is parsing statements highly, but not executing properly. They are just chewing up your CPU, latching the shared pool, killing your performance.

If the database is parsing every statement that is executing, the parse to execute ratio will be close to 1%.


How could parse be bigger than execution? Or, Why Execute to Parse values become too low?

It is possible that the application is not using shareable SQL, or the database has sub-optimal parameters that are reducing the effectiveness of cursor sharing. A problem like excessive parsing is likely to manifest itself as additional network traffic between the application server and clients. The additional parse activity may also show up as a marked increase in CPU consumption on the database server.

(http://www.dba-oracle.com/m_parse_to_execute_ratio.htm)

There may be tools that parse a statement but never execute it. Or they parse it to discover what inputs/outputs it takes and then later on parse it again to execute it (so the parse/execute is 2:1).

It is commonly referred to as "inefficient coding techniques". They parse like there is no tomorrow -- works great in a single user environment -- starts to stink with two users (or more).

Not using bind variable in query can cause such low value.

Some applications (generic apps typically) parse a "select * from T" to describe a table, never execute the cursor -- their parses exceed their executes.

Oracle does what it is told to do. Oracle is told by you to PARSE. If you do not tell Oracle to execute the statement, it won't. That is how you have parse but no execute.


How I can identify those SQL's, Which PARSE but never EXECUTE:

v$sql -- look at the parse and execute counts.

Example:
select PARSE_CALLS,EXECUTIONS,SQL_TEXT
from v$sql
where executions = 0 and parse_calls > 0
order by parse_calls asc;

Select PARSE_CALLS, EXECUTIONS, SQL_TEXT
from v$sql
where executions <
PARSE_CALLS
order by parse_calls;

How they can be eliminated or efficiently coded ?


By only parsing a statment ONCE per session,

not once per execution
not once to "describe a table"
not once to "describe the ith column in a table"

When Parse is very high and execution is very low, Check in coding --
cursor-sharing=force must be on?
**Make cursor-sharing=similar. That will reduce hard parse.
(http://asktom.oracle.com/pls/asktom/f?p=100:11:2626091586804596::::P11_QUESTION_ID:4032595293314)

SQL> show parameter cursor_sharing

Setting CURSOR_SHARING to EXACT allows SQL statements to share the SQL area only when their texts match exactly. This is the default behavior. Using this setting, similar statements cannot shared; only textually exact statements can be shared.

Some Oracle databases with high ad-hoc query activity (Crystal Reports, Business Objects cannot avoid in-line literals inside the SQL, and that's why Oracle introduced the cursor_sharing parameter. Cursor_sharing=similar allows for bind variable "peeking", and in my opinion, it's too buggy to use until Oracle 11i, when you get adaptive cursor sharing. (http://www.dba-oracle.com/t_cursor_sharing_similar.htm)

If you have a routine that will be executed many times to generate a page, you should definitely parse once per page and execute it over and over, closing the cursor if necessary before your connection is returned to the connection pool.

It is 100% controlled by the client code and cannot be affected by the server settings at all. It is 100% a function of the number of times the client parses a sql statement and how many times they execute it.


How a Statement given for parsing cannot Execute?

Example:
dbms_sql.parse ( l_theCursor, p_query, dbms_sql.native );
dbms_sql.describe_columns ( l_theCursor, l_colCnt, l_descTbl );

That just parsed the query, described it (to find the number of columns, data types, names, sizes, etc).

It never executed it -- If you closed the cursor, you would have a PARSE without an execute.


cursor sharing and parse:

Cursor sharing cannot reduce the NUMBER of parse calls (only your coders can do that! really - that is the ONLY way to reduce parse calls, they developers must call PARSE less often!)
cursor sharing can reduce the NUMBER OF HARD PARSES, that is all.

Hard parse:
A hard parse is expensive because each incoming SQL statement must be re-loaded into the shared pool; with the associated overhead involved in shared pool RAM allocation and memory management. Once loaded, the SQL must then be completely re-checked for syntax & semantics and an executable generated. Excessive hard parsing can occur when your shared_pool_size is too small or when you have non-reusable SQL statements without host variables and bind variables.


http://www.ooug.org/2005slides/0720/ToolsIuse/trace_no.html
http://asktom.oracle.com/pls/asktom/f?p=100:11:2626091586804596::::P11_QUESTION_ID:4032595293314

10.19.2008

Oracle DBA scripts - Partition related

set pages 999 lines 100
col table_name format a40
col high_value format a20
col tablespace_name format a20

1. List partitioned tables

select table_name, partitioning_type type, partition_count partitions
from dba_part_tables
where owner = '&owner'
order by 1;

2. List a tables partitions

select partition_name , tablespace_name , high_value
from dba_tab_partitions
where table_owner = '&owner' and table_name = '&table_name'
order by partition_position;

9.23.2008

ORA-30041

I was connected as sysdba and when I tried to assign quota on temporary tablespace I got an error:

SQL> ALTER USER r_nahar QUOTA UNLIMITED ON TEMP;
ALTER USER r_nahar
*
ERROR at line 1:
ORA-30041: Cannot grant quota on the tablespace

Why this error:
It is is expected behavior in oracle 10.2g.
In version 10.2 oracle does not support quota on temporary tablespaces.
If your version is earlier than 10.2g then this is a bug.

Solution:
Don't try to assign quota on the temporary tablespaces if your database version is 10.2 as it is expected behavior.



9.12.2008

DBA Training Class

DBA (Oracle-10g) Training contains two parts:

9.11.2008

Administration Part - 1

LESSON 1: INSTALLINGORACLE DATABASE 10

A. Install Oracle Database 10g


LESSON 2: EXPLORING THEORACLE DATABASE 10 ARCHITECTURE

A. Understand Database Architecture

B. Identify Database Object Types
  • Database Object Types
  • Database Object Naming Conventions
C. Recognize the Oracle Instance
  • The System Global Area
  • Background Process Types
D. Understand the User Connection Process
  • The User Connection Process
E. Identify Resources for the DBA
  • Data Dictionary
  • Data Dictionary Views
F. Oracle Enterprise Manager Framework
  • OEM Database Control
  • OEM Grid Control
  • OEM Performance Pages
  • Grid Control Performance Pages

LESSON 3: CREATING A DATABASE

A. Create, Configure, and Delete a Database Using the DBCA
  • Logical Design Planning
  • DBCA Management Options
  • Creating a Database with the DBCA
  • Optimal Flexible Architecture
  • Template Management
  • Common Template Sections
B. Use Client and Administration Tools
  • Database Interfaces
  • SQL*Plus
  • iSQL*Plus

LESSON 4: CONFIGURINGORACLENET SERVICES

A. Examine Oracle Net Services
  • Client-Server Architecture
  • N-tier Architecture
  • Oracle Net
  • Oracle Net Features
  • Oracle Net Layered Architecture

B. Perform Basic Oracle Net Server-Side Configuration
  • The Oracle Listener
  • Listener Management
  • Instructor Edition

C. Perform Basic Oracle Net Client-Side Configuration
  • Name Resolution
  • Name Resolution Methods
  • Host Naming Configuration
  • Local Naming Configuration
  • Connection Issue Troubleshooting
  • Tnsping Utility
  • Oracle Net Logging
  • Oracle Net Tracing
D. Configure Oracle Shared Server
  • Shared Server Architecture
  • Shared Server Connection Process
  • Shared Server Configuration
  • Shared Server Monitoring and Tuning
E. Configure Connect-Time Failover
  • Connect-Time Failover

LESSON 5:USING SQL AND PL/SQL

A.Write Simple SQL SELECT Statements
  • Structured Query Language
  • Types of SQL Commands
  • SQL Query Processing Steps Statement
B. Filter and Sort Data
  • The WHERE Clause
  • The ORDER BY Clause
C.Work with SQL Functions
  • Expressions
  • Single-Row Functions
  • Single-Row Function Types
  • Group Functions
  • Group Function Options

Determining Server Requirements

Before installing Oracle Database 10g, the DBA may need to purchase the future database’s server. The DBA needs to estimate a number of factors that will affect server size: expected data capacity of the database, expected user load, and potential future growth. The DBA must also choose an operating system, the number and speed of processors, the size of disk configurations and memory, and the amount of space to hold backups.

Minimum Hardware Requirements:
Oracle Database 10g requires approximately 4 GB of free hard disk space for a complete, basic installation, including 1 GB of swap space and 400 MB of temporary space. A minimum of 256 MB of memory is required for a basic installation, and a total of 512 MB is recommended for using the Oracle Enterprise Manager (OEM). However, for optimal performance, additional memory should be considered.

Sample Database Creation:
The Oracle Database 10g installation gives the option of creating a sample database in addition to installing the database management system. If the DBA chooses not to create a database during installation, databases may be created after installation is complete.

Installation features

After determining server requirements, the DBA needs to become familiar with Oracle’s installation features. Oracle Database 10g supports the industry trend toward grid computing. It includes an automatic hardware and software compatibility check, direct integration of the Oracle MetaLink service into the Oracle Enterprise Manager (OEM), and options for database management, file storage, and backup and recovery.

There are many installation features in Oracle 10g.

a. Database management
The Oracle Enterprise Manager (OEM) Database Control, which is included with Oracle Database 10g, can be used to manage local databases. In order to centrally manage several databases across multiple servers, install the OEM Grid Control from the Oracle Enterprise Manager CD. OEM Grid Control enables the configuration and management of multiple hosts, databases, and other Oracle services.

b. Preinstallation compatibility check
Performing an Oracle Database 10g installation using the Oracle Universal Installer (OUI)invokes a preinstallation system check utility, which ensures that the system meets the minimum requirements for installation, including hardware, free disk space, and operating system version. The utility may be launched separately, either as an independent system check or as part of a manual installation.

c. File storage
Three options are available through the OUI to configure database file storage:

• The File Systems option specifies that database file systems will follow the standard method for managing files and directories as specified by the operating system.

• Automatic Storage Management (ASM) combines the functionality of a Logical Volume Manager (LVM) with the simplicity of a traditional file system. To use the ASM management option, either specify an existing ASM disk group or add a set of disks to be used in a new ASM disk
group.

• The Raw Devices option enables shared storage on raw disk volumes or partitions for databases stored in Real Application Clusters (RACs) when a Cluster File System (CFS) is not present on the system. In order to use the Raw Devices option during installation, a raw device must first be created for each datafile, log file, and control file in the database.

d. Backup and recovery
Automatic backups may be enabled or disabled during installation. In order to enable backups, a backup location and a user’s name and password credentials must be provided.

e. User passwords
The SYS, SYSTEM, DBSNMP, and SYSMAN database management schemas may use a single password specified during installation, or different passwords may be created for each account.

f. Cluster Ready Services(CRSs)
Provide Oracle Database 10g administrators with a native tool for RAC and platform services management.

g. Enterprise Configuration Management Tool(ECMT)
Available only on systems using the OEM Grid Control. ECMT simplifies the tasks of collecting host and database information and performing enterprise configuration.

h. Oracle MetaLink integration
Administrators with valid Oracle MetaLink accounts may store their account credentials within the OEM to receive automatic software patch advisories, manage downloaded patches, and administer patch applications.

i. Cloning
The Oracle Database 10g OEM provides a Clone Database utility to safely and easily duplicate existing databases. In addition, OEM Grid Control users can duplicate an existing Oracle Database 10g database on one or more servers, greatly simplifying the process of cloning an entire software installation, or Oracle Home directory, to multiple locations, while automatically adjusting critical environment variables, such as host names and IP addresses.

Installation Performance Capabilities

The DBA needs only the Oracle Database 10g installation CD in order to perform a basic server installation, and can complete it in under 30 minutes. Minimum hardware requirements include 4 GB of disk space and 256 MB of memory.

Additional CD-ROMs:

To install supplementary applications, such as OEM Grid Control and HTTP Server, additional CD-ROMs are required:

• Oracle Database 10g Companion CD
• Oracle Database 10g Products CD
• Oracle Database 10g Client CD
• Oracle Database 10g Documentation CD
• Oracle Database 10g Enterprise Manager CD
• Oracle Database 10g Demos CD
All content on these CD-ROMs is available on a single DVD-ROM.

Instance Configuration

An instance may be configured and started during the basic installation process or by using a simple sequence of commands. Oracle Database 10g promotes the use of central areas for backup and metadata storage.

There are several instance configuration features. Such as-

a. Database Configuration Assistant (DBCA)features:

The creation or configuration of a database using the DBCA in Oracle Database 10g automatically creates and implements a new, auxiliary system tablespace, SYSAUX, which supplements the SYSTEM database as a central location for the storage of database metadata. The flash recovery area is an optional disk location used to store files related to recovery, such as control file and online redo log copies, archived logs, and flashback logs. When the DBCA is used to create a database, it automatically creates the flash recovery area.

b. Policy-based database configuration framework:

Oracle Database 10g includes a stock set of recommended database policies, which can be disabled via the OEM Database Control interface. These policies aid database monitoring and administration by implementing a default set of best practices-the violation of which results in Warning or Critical alerts in the OEM Database Control.

c. Initialization parameters:

Basic initialization parameters, such as PROCESSES, SESSIONS, and CONTROL_FILES, are separated from more advanced parameters and defined automatically. Advanced parameters, such as CURSOR_SHARING, should be modified only to improve performance or troubleshoot problems.

9.10.2008

Examples

SELECT log_mode FROM gv$database;

SQL> ALTER SESSION SET NLS_DATE_FORMAT = 'DD-MON-YYYY HH24:MI:SS';
SQL> col NAME format a70

SQL> col SQL_REDO format a30
SQL> col SQL_UNDO format a30
SQL> column USERNAME format a15
you must have the SELECT ANY TRANSACTION privilege to query V$LOGMNR_CONTENTS.

Examples Using LogMiner:

Example 1: Finding All Modifications in the Last Archived Redo Log File

Step 1:
Determine which redo log file was most recently archived.

This example assumes that you know that you want to mine the redo log file that was most recently archived.

SELECT NAME FROM V$ARCHIVED_LOG
WHERE FIRST_TIME = (SELECT MAX(FIRST_TIME) FROM V$ARCHIVED_LOG);


NAME
-------------------------------------------
/oracle/DATABASE_NAME/archivelog/2008_08_31/o1_mf_1_8554_4cngdomq_.arc

Step 2:

Specify the list of redo log files to be analyzed.

Specify the redo log file that was returned by the query in Step 1. The list will consist of one redo log file.

BEGIN DBMS_LOGMNR.ADD_LOGFILE
('/oracle/database_name/archivelog/2008_08_31/o1_mf_1_8554_4cngdomq_.arc');
END;
/

EXECUTE DBMS_LOGMNR.ADD_LOGFILE( -
LOGFILENAME => '/usr/oracle/data/db1arch_1_16_482701534.dbf', -
OPTIONS => DBMS_LOGMNR.NEW);

Step 3:

Start LogMiner.

SQL>EXECUTE DBMS_LOGMNR.START_LOGMNR(-
OPTIONS => DBMS_LOGMNR.DICT_FROM_ONLINE_CATALOG);


PL/SQL procedure successfully completed.

Step 4:

Query the V$LOGMNR_CONTENTS view.


Note that there are four transactions (two of them were committed within the redo log file being analyzed, and two were not).

The output shows the DML statements in the order in which they were executed; thus transactions interleave among themselves.

SELECT username AS USR, (XIDUSN || '.' || XIDSLT || '.' ||  XIDSQN) AS XID,
SQL_REDO, SQL_UNDO FROM V$LOGMNR_CONTENTS WHERE username IN ('HR', 'OE');

USR XID SQL_REDO SQL_UNDO
---- --------- --------------------- -------------------------------
HR 1.11.1476 set transaction read write;

HR 1.11.1476 insert into "HR"."EMPLOYEES"( delete from "HR"."EMPLOYEES"
"EMPLOYEE_ID","FIRST_NAME", where "EMPLOYEE_ID" = '306'
"LAST_NAME","EMAIL", and "FIRST_NAME" = 'Nandini'
"PHONE_NUMBER","HIRE_DATE", and "LAST_NAME" = 'Shastry'
"JOB_ID","SALARY", and "EMAIL" = 'NSHASTRY'
"COMMISSION_PCT","MANAGER_ID", and "PHONE_NUMBER" = '1234567890'
"DEPARTMENT_ID") values and "HIRE_DATE" = TO_DATE('10-JAN-2003
('306','Nandini','Shastry', 13:34:43', 'dd-mon-yyyy hh24:mi:ss')
'NSHASTRY', '1234567890', and "JOB_ID" = 'HR_REP' and
TO_DATE('10-jan-2003 13:34:43', "SALARY" = '120000' and
'dd-mon-yyyy hh24:mi:ss'), "COMMISSION_PCT" = '.05' and
'HR_REP','120000', '.05', "DEPARTMENT_ID" = '10' and
'105','10'); ROWID = 'AAAHSkAABAAAY6rAAO';


------------------------------------------------------------------------------------

SQL> col SQL_REDO format a30
SQL> col SQL_UNDO format a30
SQL> column USERNAME format a15

SELECT username,session# SID, (XIDUSN || '.' || XIDSLT || '.' || XIDSQN) AS XID,
SQL_REDO, SQL_UNDO,
to_char(timestamp,'mm/dd/yy hh24:mi:ss')timestamp
FROM V$LOGMNR_CONTENTS WHERE username IN ('PROD7');


Step 5 End the LogMiner session.

SQL> EXECUTE DBMS_LOGMNR.END_LOGMNR();


-------------------------------------------------
Example 2: Grouping DML Statements into Committed Transactions
Step 1:
SELECT NAME FROM V$ARCHIVED_LOG
WHERE FIRST_TIME = (SELECT MAX(FIRST_TIME) FROM V$ARCHIVED_LOG);


NAME
-------------------------------------------
/oracle/database_name/archivelog/2008_08_31/o1_mf_1_8554_4cngdomq_.arc

Step 2:
BEGIN DBMS_LOGMNR.ADD_LOGFILE(
LOGFILENAME => '
/oracle/database_name/archivelog/2008_08_31/o1_mf_1_8554_4cngdomq_.arc',
OPTIONS => DBMS_LOGMNR.NEW);
END;
/

Step 3:

Start LogMiner by specifying the dictionary to use and the COMMITTED_DATA_ONLY option.
EXECUTE DBMS_LOGMNR.START_LOGMNR(-
OPTIONS => DBMS_LOGMNR.DICT_FROM_ONLINE_CATALOG + -
DBMS_LOGMNR.COMMITTED_DATA_ONLY);

Step 4
:
SELECT username AS USR, (XIDUSN || '.' || XIDSLT || '.' || XIDSQN) AS XID, SQL_REDO,
SQL_UNDO FROM V$LOGMNR_CONTENTS WHERE username IN ('HR', 'OE');
;

Example 4: Using the LogMiner Dictionary in the Redo Log Files

This example shows how to use the dictionary that has been extracted to the redo log files.

When you use the dictionary in the online catalog, you must mine the redo log files in the same database that generated them.

Using the dictionary contained in the redo log files enables you to mine redo log files in a different database.


This example assumes that you know that you want to mine the redo log file that was most recently archived.

SELECT NAME, SEQUENCE# FROM V$ARCHIVED_LOG
WHERE FIRST_TIME = (SELECT MAX(FIRST_TIME) FROM V$ARCHIVED_LOG);

NAME SEQUENCE#
------------------------------------------------------------------------------------------------------ ----------------
/oracle/database_name/archivelog/2008_08_31/o1_mf_1_8568_4cpdcgcw_.arc 8568


SELECT NAME, SEQUENCE#, DICTIONARY_BEGIN d_beg, DICTIONARY_END d_end
FROM V$ARCHIVED_LOG
WHERE SEQUENCE# = (SELECT MAX (SEQUENCE#) FROM V$ARCHIVED_LOG
WHERE DICTIONARY_END = 'YES' and SEQUENCE# <= 8568);

Example 6: Filtering Output by Time Range

ALTER SESSION SET NLS_DATE_FORMAT = 'DD-MON-YYYY HH24:MI:SS';

To filter out redo records based on timestamp values is by specifying the time range in the
DBMS_LOGMNR.START_LOGMNR procedure call.

If you want to see the archive log file name from V$ARCHIVED_LOG
SELECT NAME,to_char(FIRST_TIME,'mm/dd/yy hh24:mi:ss') Time FROM V$ARCHIVED_LOG
WHERE FIRST_TIME like ('01-SEP-08%');
Step 1: Create a list of redo log files to mine.

The subsequent SQL EXECUTE statement calls the procedure and specifies the starting date 01-SEP-2008:
-- my_add_logfiles
-- Add all archived logs generated after a specified start_time.

CREATE OR REPLACE PROCEDURE my_add_logfiles (in_start_time IN DATE) AS
CURSOR c_log IS
SELECT NAME FROM V$ARCHIVED_LOG
WHERE FIRST_TIME >= in_start_time;

count pls_integer := 0;
my_option pls_integer := DBMS_LOGMNR.NEW;

BEGIN
FOR c_log_rec IN c_log
LOOP
DBMS_LOGMNR.ADD_LOGFILE(LOGFILENAME => c_log_rec.name,
OPTIONS => my_option);
my_option := DBMS_LOGMNR.ADDFILE;
DBMS_OUTPUT.PUT_LINE('Added logfile ' || c_log_rec.name);
END LOOP;
END;
/

EXECUTE my_add_logfiles(in_start_time => '01-sep-2008');

Step 2:

Query the V$LOGMNR_LOGS to see the list of redo log files.

SQL> col NAME format a70

SQL> SELECT FILENAME name, LOW_TIME start_time, FILESIZE bytes FROM V$LOGMNR_LOGS;

Query the V$LOGMNR_LOGS to see the list of redo log files.

i.
EXECUTE DBMS_LOGMNR.START_LOGMNR(-
OPTIONS => DBMS_LOGMNR.DICT_FROM_ONLINE_CATALOG + -
DBMS_LOGMNR.COMMITTED_DATA_ONLY + -
DBMS_LOGMNR.PRINT_PRETTY_SQL);

Or

ii.

SQL> ALTER SESSION SET NLS_DATE_FORMAT = 'DD-MON-YYYY HH24:MI:SS';


EXECUTE DBMS_LOGMNR.START_LOGMNR(-
STARTTIME => '01-sep-2008 15:00:00', -
ENDTIME => '02-sep-2008 16:00:00', -
OPTIONS => DBMS_LOGMNR.DICT_FROM_ONLINE_CATALOG + -
DBMS_LOGMNR.COMMITTED_DATA_ONLY + -
DBMS_LOGMNR.PRINT_PRETTY_SQL);


To avoid the need to specify the date format in the call to the PL/SQL DBMS_LOGMNR.START_LOGMNR procedure, you can use the SQL ALTER SESSION SET NLS_DATE_FORMAT statement first, as shown in the following example.

ALTER SESSION SET NLS_DATE_FORMAT = 'DD-MON-YYYY HH24:MI:SS';

EXECUTE DBMS_LOGMNR.START_LOGMNR( -
DICTFILENAME => '/oracle/database/dictionary.ora', -
STARTTIME => '01-Jan-1998 08:30:00', -
ENDTIME => '01-Jan-1998 08:45:00'-
OPTIONS => DBMS_LOGMNR.CONTINUOUS_MINE);
----------------------

select username, timestamp,scn_to_timestamp(SCN),
seg_type_name, seg_name, table_space, session# SID, serial# , operation
from v$logmnr_contents
where table_name = 'LOGIN' and
(scn_to_timestamp(SCN))> TO_date ('27-AUG-08 00:22:39', 'dd-mon-yyyy hh24:mi:ss');--run saturn

select username,scn_to_timestamp(SCN) as DataTime ,
seg_type_name, seg_name, session# SID, serial# , operation
from v$logmnr_contents
where table_name = 'LOGIN' and
(scn_to_timestamp(SCN))> TO_date ('27-AUG-08 00:22:39', 'dd-mon-yyyy hh24:mi:ss');

-----------------------------------

SQL> set lines 200
SQL> column USERNAME format a20
SQL> column SEG_NAME format a15
SQL> column SEG_TYPE_NAME format a15
SQL> column TABLE_SPACE format a15

select username,scn_to_timestamp(SCN)as DataTime,
seg_type_name, seg_name, session# SID, operation
from v$logmnr_contents
where table_name = 'LOGIN'
and (scn_to_timestamp(SCN))> TO_date ('27-AUG-08 01:00:00', 'dd-mon-yyyy hh24:mi:ss')
and (scn_to_timestamp(SCN))<>


select username,scn_to_timestamp(SCN)as DataTime,timestamp,
seg_type_name, seg_name, session# SID, operation
from v$logmnr_contents
where
username in ('prod7');

select username,timestamp,seg_type_name, seg_name, session# SID, operation
from v$logmnr_contents
where table_name = 'LOGIN';

----test

select username,timestamp,seg_type_name, seg_name, session# SID, operation
from v$logmnr_contents
where table_name = 'LOGIN'
and timestamp > TO_date ('01-SEP-2008 20:00:00', 'dd-mon-yyyy hh24:mi:ss')
and timestamp <>

Remote Mining

Source database mining in production database you might avoid. So remote mining comes.
Remote mining means the database that analyze of redo log files or archived redo log files is not same as of source database- that generate redo/archived log files.

The system of using Logminer in remote DB is
-transfer the analysis redo or archived redo log files as well as
-transfer the mining dictionary to the remote database.

Step by step:

Prerequisites:

-To extract a LogMiner dictionary to the redo log files, the DB must be open and in ARCHIVELOG mode and archiving must be enabled.


Scenario: Online redo log file redo01 need to be analyzed in remote machine neptune(local).
Suppose Production db or Source DB is Saturn.

Step1:

Extract logminer dictionary: (On Source Database)


In source that is in production server build the dictionary by,

SQL> EXECUTE DBMS_LOGMNR_D.BUILD(OPTIONS=> DBMS_LOGMNR_D.STORE_IN_REDO_LOGS);
PL/SQL procedure successfully completed.

The process of extracting the dictionary to the redo log files does consume database resources, but if you limit the extraction to off-peak hours, this should not be a problem, and it is faster than extracting to a flat file.
Depending on the size of the dictionary, it may be contained in multiple redo log files.

After executing this procedure in DB, this procedure adds the dictionary information to the online redo logs, which, in turn, go into the archived logs.
This redo log file must have been created before the redo log file that you want to analyze, but should be as recent as possible.

If the relevant redo log files have been archived, you can find out which redo log files contain the start and end of an extracted dictionary.
To do so, query the V$ARCHIVED_LOG view, by,

SQL> SELECT NAME FROM V$ARCHIVED_LOG WHERE DICTIONARY_BEGIN='YES';

NAME
--------------------------------------------------------------------------------
/oracle/database_name/archivelog/2008_09_03/o1_mf_1_8601_4cw3q3t8_.arc

SQL> SELECT NAME FROM V$ARCHIVED_LOG WHERE DICTIONARY_END='YES';

NAME
--------------------------------------------------------------------------------
/oracle/database_name/archivelog/2008_09_03/o1_mf_1_8601_4cw3q3t8_.arc


Specify the names of the start and end redo log files, and possibly other logs in between them, with the ADD_LOGFILE procedure when you are preparing to begin a LogMiner session.

Step 2:

Transfer the logminer dictionary and log for analysis into the mining database.

SQL> !scp /oracle/database_name/archivelog/2008_09_03/o1_mf_1_8601_4cw3q3t8_.arc oracle@neptune:

Password:
o1_mf_1_8601_4cw3q3t 100% |*********************************************************************| 10836 KB 00:01


Then also transfer the redo log. Based on your requirement you can transfer archived log or online redo log. To see a defined time archived log query by select NAME,FIRST_NAME from v$archived_log where completion_time >SYSDATE-1;


In this example I will analysis online redo log file.

SQL> SELECT distinct member LOGFILENAME FROM V$LOGFILE;

LOGFILENAME
--------------------------------------------------------------------------------
/oradata1/
database_name/datafiles/database_name/redo03.log
/oradata1/
database_name/datafiles/database_name/redo01.log
/oradata1/
database_name/datafiles/database_name/redo02.log

SQL> !scp /oradata1/database_name/datafiles/database_name/redo01.log oracle@neptune:
Password:
redo03.log 100% |*********************************************************************| 51200 KB 00:04


Step 3: Specify the files for analysis.(In mining/remote database) (Neptune)

Here specify the logfile that need to mine plus specify the dictionary files.
SQL>EXECUTE DBMS_LOGMNR.ADD_LOGFILE( LOGFILENAME => '/export/home/oracle/redo01.log', OPTIONS => DBMS_LOGMNR.ADDFILE);

PL/SQL procedure successfully completed.

SQL>EXECUTE DBMS_LOGMNR.ADD_LOGFILE( LOGFILENAME => '/export/home/oracle/o1_mf_1_1284_4c23hcd6_.arc', OPTIONS => DBMS_LOGMNR.ADDFILE);

PL/SQL procedure successfully completed.

Step 4:
Start the logminer session(In mining database)

Don't specify here DICTFILENAME option. Execute the following,

SQL> EXECUTE DBMS_LOGMNR.START_LOGMNR(OPTIONS => DBMS_LOGMNR.DICT_FROM_REDO_LOGS);

Step 5:
Query V$LOGMNR_CONTENTS

Step 6:
End the LogMiner Session:
EXECUTE DBMS_LOGMNR.END_LOGMNR;



LogMiner Setting

1. Using the Online Catalog: (Set LogMiner in Source Database)

Supplemental logging places additional column data into the redo log file whenever an UPDATE operation is performed.

Step-1:
Ensure that you have on at a minimal level supplemental logging.
To work with logMiner you must have database supplemental logging on of the source database at a minimum level.
By default, Oracle DB does not provide any supplemental logging, that means, by default LogMiner is not usable.

You can check your supplemental logging on of off by following commands,
SQL> select SUPPLEMENTAL_LOG_DATA_MIN from v$database;
SUPPLEME
--------
NO

In order to on it at a minimal level,
SQL> ALTER DATABASE ADD SUPPLEMENTAL LOG DATA;
Database altered.

Step-2:


Install the DBMS_LOGMNR package.
The next step is to install DBMS_LOGMNR package. If you have already installed then ignore this steps. You can install this package by running $ORACLE_HOME/rdbms/admin/dbmslm.sql script. If you create your database with dbca then this script run automatically. So you can ignore this step. However if you create database manually with CREATE DATABASE ... command then you must run the script before using logminer. That is ,
SQL>@$ORACLE_HOME/rdbms/admin/dbmslm.sql

Step-3:


Grant the EXECUTE_CATALOG_ROLE role.
The user who will do the mining task give him EXECUTE_CATALOG_ROLE. Here user is nahar.
SQL>GRANT EXECUTE_CATALOG_ROLE TO nahar;

Step-4:

Create the synonym. nahar creates a public synonym:
CREATE PUBLIC SYNONYM dbms_logminer_nahar FOR SYS.DBMS_LOGMNR;

--All above four steps are needed just for once.

************************************************************** kisukkhon por por abar ekhan theke shuru

Step-5:

Specify the scope of the mining.
Now you decide on which file you will do the analysis. You may have interest over archived redo log files or online redo log files based on your scenario.

i. Analysis Redo logs:

Suppose you have recent problem in your database and so you might show interest of your online redo log files. You can see
current online redo logs by,
SQL> SELECT distinct member LOGFILENAME FROM V$LOGFILE;
LOGFILENAME
--------------------------------------------------------------------------------

/oracle/app/oradata/FolderName/redo02.log
/oracle/app/oradata/FolderName/redo03.log
/oracle/app/oradata/FolderName/redo01.log


SQL>BEGIN DBMS_LOGMNR.ADD_LOGFILE

('/oracle/app/oradata/FolderName/redo01.log');
END;
/

PL/SQL procedure successfully completed.


BEGIN DBMS_LOGMNR.ADD_LOGFILE

('/oracle/app/oradata/FolderName/redo02.log');
END;
/

PL/SQL procedure successfully completed.


BEGIN DBMS_LOGMNR.ADD_LOGFILE

('/oracle/app/oradata/FolderName/redo03.log');
END;
/

PL/SQL procedure successfully completed.


Practice:

To add multiple log file:

SQL>BEGIN DBMS_LOGMNR.ADD_LOGFILE
('/oradata2/data1/dbase/redo01.log');
DBMS_LOGMNR.ADD_LOGFILE
('/oradata2/data1/dbase/redo03.log');
END;
/


ii. Analysis Archive logs:

Suppose you have past problem in your database and so you might show interest of your archive log files.

Sometimes, you want to mine the redo log file that was most recently archived. You can see --
To check recent archive log:

SQL> SELECT NAME FROM V$ARCHIVED_LOG
WHERE FIRST_TIME = (SELECT MAX(FIRST_TIME) FROM V$ARCHIVED_LOG);

NAME
--------------------------------------------------------------------------------
/oracle/database_name/archivelog/2008_08_27/o1_mf_1_8516_4cb52pvt_.arc


SQL> BEGIN DBMS_LOGMNR.ADD_LOGFILE
('/oracle/database_name/archivelog/2008_08_27/o1_mf_1_8516_4cb52pvt_.arc');
END;
/


Practice:

SQL>column NAME format a70

SELECT NAME,to_char(FIRST_TIME,'mm/dd/yy hh24:mi:ss') Time FROM V$ARCHIVED_LOG
WHERE FIRST_TIME like ('27-AUG-08%'); --to get the 26 august's archive log


Step-6:

To direct LogMiner to use the dictionary currently in use for the database, specify the online catalog as your dictionary source -

Start the LogMiner session and specify a dictionary.
To start the LogMiner session:

BEGIN
DBMS_LOGMNR.START_LOGMNR
(options =>
dbms_logmnr.dict_from_online_catalog);
END;
/


PL/SQL procedure successfully completed.

Using the
OPTIONS parameter, it is specified that Oracle Database read the dictionary information to convert the object names from the online catalog while starting LogMiner.

Step-7: ***
Request the redo data

Check contents from V$LOGMNR_CONTENTS view.
To get information and when DML or DDL happened in the V$LOGMNR_CONTENTS about table STUDENT we can issue

SQL> set lines 200
SQL> set pages 0 -------to delete the column header
sys@DatabaseName>column USERNAME format a20
sys@
DatabaseName>column SEG_NAME format a15
sys@
DatabaseName>column SEG_TYPE_NAME format a15
sys@
DatabaseName>column TABLE_SPACE format a15

SQL> select username, to_char(timestamp,'mm/dd/yy hh24:mi:ss') timestamp,
seg_type_name, seg_name, table_space, session# SID, serial# , operation
from v$logmnr_contents
where table_name = 'STUDENT';


We can get SQL_UNDO and SQL_REDO information by,

SQL>
col SQL_REDO format a30
SQL> col SQL_UNDO format a30

SQL> select sql_undo, sql_redo
from v$logmnr_contents
where table_name = 'STUDENT' and OPERATION='UPDATE';


Request the archive data:

select username, to_char(timestamp,'mm/dd/yy hh24:mi:ss') timestamp,
seg_type_name, seg_name, table_space, session# SID, serial# , operation
from v$logmnr_contents
where table_name = 'LOGIN';

select username, to_char(timestamp,'mm/dd/yy hh24:mi:ss') timestamp,
seg_type_name, seg_name, table_space, session# SID, serial# , operation
from v$logmnr_contents
where table_name = 'LOGIN'
and timestamp like ('27-AUG-08%'); --to get the info of a specific date


step-8:

End the LogMiner session.

Use the DBMS_LOGMNR.END_LOGMNR procedure.


SQL>

BEGIN

DBMS_LOGMNR.END_LOGMNR;
END;
/

PL/SQL procedure successfully completed.


LogMiner

v$logmnr_contents

desc v$logmnr_contents

The type of change made to the database: INSERT, UPDATE, DELETE, or DDL (OPERATION column).

The SCN at which a change was made (SCN column).

The SCN at which a change was committed (COMMIT_SCN column).

The transaction to which a change belongs (XIDUSN, XIDSLT, and XIDSQN columns).

The table and schema name of the modified object (SEG_NAME and SEG_OWNER columns).

The name of the user who issued the DDL or DML statement to make the change (USERNAME column).

If the change was due to a SQL DML statement, the reconstructed SQL statements showing SQL DML that is equivalent (but not necessarily identical) to the SQL DML used to generate the redo records (SQL_REDO column).

If a password is part of the statement in a SQL_REDO column, the password is encrypted. SQL_REDO column values that correspond to DDL statements are always identical to the SQL DDL used to generate the redo records.

If the change was due to a SQL DML change, the reconstructed SQL statements showing the SQL DML statements needed to undo the change (SQL_UNDO column).

LogMining

We know that any changes to database is recored in online redo logfiles. If your database archival mode on then online redo log files are archived which contains the database changes. With these logfile we can analysis any changes in database.


In simple we can say
an undo entry provides the values of data stored before a change and the redo entry provides the values of data stored after a change. So we can get them from online redo logs and then to archived logs.

  • So from online redo logs and from archived redo logs we can get database redo and undo information. But online and archived logs have an unpublished format and are not human-readable. With the DBMS_LOGMNR package we can analysis redo log files and can get back undo and redo information in a human readable format.

  • Another scenario of use of logminer is to investigate database past in time. With Flashback Query we can get prior values of the record in the table at some point in the past but is limited to UNDO_RETENTION parameter (which is often as short as 30 minutes for an OLTP database.).So in order to analysis past activity on the database logminer is a good choice.


Definition:

  • Every change made to an Oracle database by default generates undo and redo information which is accumulated in Oracle redo log files. LogMiner is an integrated feature of the Oracle Database that provides DBA's and auditors with the infrastructure required for relational access to Oracle's redo stream.

  • The Oracle LogMiner utility enables you to query redo logs through a SQL interface. Redo logs contain information about the history of activity on a database.


Why LogMiner:

You can use the Oracle LogMiner utility to enable you to query redo log files through a SQL interface. Database administrators can use LogMiner to:

  • Identify the time of a database-event

  • Isolate transactions carried out in error by users

  • determine steps needed for the recovery of inadvertent (done unintentionally) changes to data

  • assemble data on actual usage for use in performance-tuning and capacity-planning

  • auditing the operation of any commands run against the database

Note that LogMiner uses Oracle logs to reconstruct exactly how data changed, whereas the complementary utility Oracle Flashback addresses, reconstructs and presents the finished results of such changes, giving a view of the database at some point in time.



Before start using LogMiner:

  • Log in with the sys account using sysdba role

  • Never escape from current session because the data viewing of LogMiner will no be available from other session.



LogMiner Restrictions:

The following are not supported:

  • Data types LONG and LOB
  • Simple and nested abstract data types ( ADTs)
  • Collections (nested tables and VARRAYs)
  • Object Refs
  • Index Organized Tables (IOTs)

LogMiner Configuration

There are four basic objects in a LogMiner configuration that you should be familiar with: the source database, the mining database, the LogMiner dictionary, and the redo log files containing the data of interest:


  • The redo log files contain the changes made to the database or database dictionary.

  • The source database is the database that produces all the redo log files that you want LogMiner to analyze.

  • The mining database is the database that LogMiner uses when it performs the analysis.

  • The LogMiner dictionary allows LogMiner to provide table and column names, instead of internal object IDs, when it presents the redo log data that you request. LogMiner uses the dictionary to translate internal object identifiers and datatypes to object names and external data formats. Without a dictionary, LogMiner returns internal object IDs and presents data as binary data.

For example, consider the following the SQL statement: INSERT INTO HR.JOBS(JOB_ID, JOB_TITLE, MIN_SALARY, MAX_SALARY) VALUES('IT_WT','Technical Writer', 4000, 11000);

Without the dictionary, LogMiner will display: insert into "UNKNOWN"."OBJ#45522"("COL 1","COL 2","COL 3","COL 4") values(HEXTORAW('45465f4748'),HEXTORAW('546563686e6963616c20577269746572'),HEXTORAW('c229'),HEXTORAW('c3020b'))


LogMiner Dictionary Options:

LogMiner requires a dictionary to translate object IDs into object names when it returns redo data to you. LogMiner gives you three options for

supplying the dictionary.

1. Using the Online Catalog
:

Oracle recommends that you use this option when you will have access to the source database from which the redo log files were created and when no changes to the column definitions in the tables of interest are anticipated. This is the most efficient and easy-to-use option.
The online catalog contains the latest information about the database and may be the fastest way to start your analysis. Because DDL operations that change important tables are somewhat rare, the online catalog generally contains the information you need for your analysis. Remember, however, that the online catalog can only reconstruct SQL statements that are executed on the latest version of a table. As soon as a table is altered, the online catalog no longer reflects the previous version of the table. This means that LogMiner will not be able to reconstruct any SQL statements that were executed on the previous version of the table.


2. Extracting a LogMiner Dictionary to the Redo Log Files:

Oracle recommends that you use this option when you do not expect to have access to the source database from which the redo log files were created, or if you anticipate that changes will be made to the column definitions in the tables of interest.


3. Extracting the LogMiner Dictionary to a Flat File:

This option is maintained for backward compatibility with previous releases. This option does not guarantee transactional consistency. Oracle recommends that you use either the online catalog or extract the dictionary from redo log files instead.


8.27.2008

Common Mistakes in Database Design

Common mistakes:

1. Poor design/planning
2. Ignoring normalization
3. Poor naming standards
4. Lack of documentation
5. One table to hold all domain values
6. Using identity/guid columns as your only key
7. Not using SQL facilities to protect data integrity
8. Not using stored procedures to access data
9. Lack of testing

Poor design/planning

The database is the cornerstone of pretty much every business project, if you don't take the time to map out the needs of the project and how the database is going to meet them, then the chances are that the whole project will veer off course and lose direction. Furthermore, if you don't take the time at the start to get the database design right, then you'll find that any substantial changes in the database structures that you need to make further down the line could have a huge impact on the whole project, and greatly increase the likelihood of the project time-line slipping.

Admittedly it is impossible to predict every need that your design will have to fulfill and every issue that is likely to arise, but it is important to mitigate against potential problems as much as possible, by careful planning.

Ignoring Normalization

Normalization defines a set of methods to break down tables to their constituent parts until each table represents one and only one "thing", and its columns serve to fully describe only the one "thing" that the table represents.

The concept of normalization has been around for 30 years and is the basis on which SQL and relational databases are implemented. In other words, SQL was created to work with normalized data structures.

Normalizing your data is essential to good performance, and ease of development, but the question always comes up: "How normalized is normalized enough?" Still now, Upto 3rd Normal Form is essential, but 4th and 5th Normal Forms are really useful and, once you get a handle on them, quite easy to follow and well worth the time required to implement them. In reality, however, it is quite common that not even the first Normal Form is implemented correctly.

Poor naming standards

Names, are the first and most important line of documentation for your application. The names you choose are not just to enable you to identify the purpose of an object, but to allow all future programmers, users, and so on to quickly and easily understand how a component part of your database was intended to be used, and what data it stores. No future user of your design should need to wade through a 500 page document to determine the meaning of some wacky name.

A name such as tblCustomer or colVarcharAddress might seem useful from a development perspective, but to the end user it is just confusing. As a developer, you should rely on being able to determine that a table name is a table name by context in the code or tool, and present to the users clear, simple, descriptive names, such as Customer and Address.

As a practice I strongly advise against is the use of spaces and quoted identifiers in object names. You should avoid column names such as "Part Number" or, in Microsoft style, [Part Number], therefore requiring you users to include these spaces and identifiers in their code. It is annoying and simply unnecessary.

Acceptable alternatives would be PART_NUMBER, part_number, partNumber or PartNumber. Again, consistency is key. If you choose PartNumber then that's fine – as long as the column containing invoice numbers is called InvoiceNumber, and not one of the other possible variations.

Lack of documentation

By carefully naming your objects, columns, and so on, you can make it clear to anyone what it is that your database is modeling.

Documentation contain definitions on its tables, columns, relationships, and even default and check constraints, so that it is clear to everyone how they are intended to be used. In many cases, you may want to include sample values, where the need arose for the object, and anything else that you may want to know in a year or two when "future you" has to go back and make changes to the code.
Where this documentation is stored is largely a matter of corporate standards and/or convenience to the developer and end users. It could be stored in the database itself, using extended properties. Alternatively, it might be in maintained in the data modeling tools. It could even be in a separate data store, such as Excel or another relational database.

Your goal should be to provide enough information that when you turn the database over to a support programmer, they can figure out your minor bugs and fix them.


I know there is an old joke that poorly documented code is a synonym for "job security." While there is a hint of truth to this, it is also a way to be hated by your coworkers and never get a raise. And no good programmer I know of wants to go back and rework their own code years later. It is best if the bugs in the code can be managed by a junior support programmer while you create the next new thing. Job security along with raises is achieved by being the go-to person for new challenges.

One table to hold all domain values

Relational databases are based on the fundamental idea that every object represents one and only one thing. There should never be any doubt as to what a piece of data refers to. By tracing through the relationships, from column name, to table name, to primary key, it should be easy to examine the relationships and know exactly what a piece of data means.

The big myth is that the more tables there are, the more complex the design will be. So, conversely, shouldn't condensing multiple tables into a single "catch-all" table simplify the design? It does sound like a good idea but this idea is wrong in large application. This may seem a very clean and natural way to design a table for all but the problem is that it is just not very natural to work with in SQL. And in this situation, the SQL query will take long time and may arise performance issue.

The point of this tip is simply that it is better to do the work upfront, making structures solid and maintainable, rather than trying to attempt to do the least amount of work to start out a project. By keeping tables down to representing one "thing" it means that most changes will only affect one table, after which it follows that there will be less rework for you down the road.

Using identity/guid columns as your only key

First Normal Form dictates that all rows in a table must be uniquely identifiable. Hence, every table should have a primary key.

SQL Server allows you to define a numeric column as an IDENTITY column, and then automatically generates a unique value for each row.

Alternatively, you can use NEWID() (or NEWSEQUENTIALID()) to generate a random, 16 byte unique value for each row. These types of values, when used as keys, are what are known as surrogate keys. The word surrogate means "something that substitutes for" and in this case, a surrogate key should be the stand-in for a natural key.


The problem is that too many designers use a surrogate key column as the only key column on a given table. The surrogate key values have no actual meaning in the real world; they are just there to uniquely identify each row.

Now, consider the following Part table, whereby PartID is an IDENTITY column and is the primary key for the table:

PartID

PartNumber

Description

1

XXXXXXXX

The X part

2

XXXXXXXX

The X part

3

YYYYYYYY

The Y part


How many rows are there in this table? Well, there seem to be three, but are rows with PartIDs 1 and 2 actually the same row, duplicated? Or are they two different rows that should be unique but were keyed in incorrectly?

The rule of thumb I use is simple. If a human being could not pick which row they want from a table without knowledge of the surrogate key, then you need to reconsider your design. This is why there should be a key of some sort on the table to guarantee uniqueness, in this case likely on PartNumber.

In summary: as a rule, each of your tables should have a natural key that means something to the user ,and can uniquely identify each row in your table. In the very rare event that you cannot find a natural key (perhaps, for example, a table that provides a log of events), then use an artificial/ surrogate key.

Not using SQL facilities to protect data integrity (For SQL server users)

All fundamental, non-changing business rules should be implemented by the relational engine. The base rules of nullability, string length, assignment of foreign keys, and so on, should all be defined in the database.


Not using stored procedures to access data

Stored procedures are your friend. Use them whenever possible as a method to protect the database layer from the users of the data. Stored procedures make database development much cleaner, and encourage collaborative development between your database and functional programmers. A few of the other interesting reasons that stored procedures are important include the following.

Maintainability

Stored procedures provide a known interface to the data, this is probably the largest draw. Stored procedures give the database professional the power to change characteristics of the database code without additional resource involvement, making small changes, or large upgrades (for example changes to SQL syntax) easier to do.

Encapsulation

Stored procedures allow you to "encapsulate" any structural changes that you need to make to the database so that the knock on effect on user interfaces is minimized. For example, say you originally modeled one phone number, but now want an unlimited number of phone numbers. You could leave the single phone number in the procedure call, but store it in a different table as a stopgap measure, or even permanently if you have a "primary" number of some sort that you always want to display. Then a stored procedure could be built to handle the other phone numbers. In this manner the impact to the user interfaces could be quite small, while the code of stored procedures might change greatly.

Security

Stored procedures can provide specific and granular access to the system. For example, you may have 10 stored procedures that all update table X in some way. If a user needs to be able to update a particular column in a table and you want to make sure they never update any others, then you can simply grant to that user the permission to execute just the one procedure out of the ten that allows them perform the required update.


Lack of testing

As database professionals know, the first thing to get blamed when a business system is running slow is the database. Why? First because it is the central piece of most any business system, and second because it also is all too often true.


By gaining deep knowledge of the system we have created and understanding its limits through testing.
Testing is the first thing to go in a project plan when time slips a bit. And what suffers the most from the lack of testing? Functionality? Maybe a little, but users will notice and complain if the "Save" button doesn't actually work and they cannot save changes to a row they spent 10 minutes editing. What really gets the shaft in this whole process is deep system testing to make sure that the design you (presumably) worked so hard on at the beginning of the project is actually implemented correctly.


Initially, major bugs come in thick and fast, especially performance related ones. If the first time you have tried a full production set of users, background process, work flow processes, system maintenance routines, ETL, etc, is on your system launch day, you are extremely likely to discover that you have not anticipated all of the locking issues that might be caused by users creating data while others are reading it, or hardware issues cause by poorly set up hardware.


Once the major bugs are squashed, the fringe cases (which are pretty rare cases, like a user entering a negative amount for hours worked) start to raise their ugly heads. What you end up with at this point is software that irregularly fails in what seem like weird places (since large quantities of fringe bugs will show up in ways that aren't very obvious and are really hard to find.)


Now, it is far harder to diagnose and correct because now you have to deal with the fact that users are working with live data and trying to get work done. Plus you probably have a manager or two sitting on your back saying things like "when will it be done?" every 30 seconds, even though it can take days and weeks to discover the kinds of bugs that result in minor (yet important) data aberrations. Had proper testing been done, it would never have taken weeks of testing to find these bugs, because a proper test plan takes into consideration all possible types of failures, codes them into an automated test, and tries them over and over. Good testing won't find all of the bugs, but it will get you to the point where most of the issues that correspond to the original design are ironed out.

If everyone insisted on a strict testing plan as an integral and immutable part of the database development process, then maybe someday the database won't be the first thing to be fingered when there is a system slowdown.

Summary

Database design and implementation is the cornerstone of any data centric project (99.9% of business applications) and should be treated as such when you are developing. Some of the tips, like planning properly, using proper normalization, using a strong naming standards and documenting your work– these are things that even the best DBAs and data architects have to fight to make happen.