This article looks at After Image Journaling and audit trails in Recital using SET JOURNAL and associated commands.
Overview
After Image Journaling, used in conjunction with a structured backup policy is an effective disaster recovery solution. Any transaction that takes place on a table that is being journaled is logged in the journal file. In the event of a disk crash or similar event in which the table is lost, the journaled transactions can be reapplied to the latest backup copy of the table. Alternatively or additionally, the journal can be used to provide an audit trail to all modifications made to the table data.
NOTE: Recital also provides Before Image Journaling via BEGIN TRANSACTION / END TRANSACTION blocks, allowing unsuccessful transactions to be rolled back to a set saved state.
SET JOURNAL and RECOVER
Regular backups are an essential routine for any system, but in high-transaction environments restoration of the latest backup can still mean a major loss of data. After image journaling can successfully be used as part of your disaster recovery strategy to minimize data loss and down time. Recital after image journaling functionality is based on the use of the SET JOURNAL and RECOVER commands.
SET JOURNAL
SET JOURNAL TO [<.dbj filename> | ()] SET JOURNAL ON | OFF | ( )
The SET JOURNAL command is used to enable the After Image Journaling and audit trail for the active table. The TO <.dbj filename> clause associates the specified transaction journal file with the active table. If the journal file does not exist, it will be created. The filename can be substituted with a <expC>, enclosed in round brackets, which returns a valid filename. If no file extension is specified, ‘.dbj’ is used. When specifying a journal file, it is recommended that the journal file is stored on a different disk than that which the table is stored on, so that if a fatal disk error occurs, then the journal file will not be lost along with the table.
//Enable journaling for the southwind!orders table open database southwind use orders set journal to /journals/ord_journ
The <.dbj filename> is a standard table. It contains seven
fields that are specific to a journal file, followed by the first
249 fields of the associated table.
The first seven fields in the journal are:
|
Field |
Type |
Display |
Storage |
Description |
|
AUD_DATE |
Date |
8 | 10 * |
4 |
The date on which the transaction was performed. |
|
AUD_TIME |
Character |
8 |
8 |
The time at which the transaction was performed, in the format HH:MM:SS. |
|
AUD_TERM |
Character |
12 |
12 |
The name of the terminal from which the transaction was performed |
|
AUD_UID |
Short |
5 |
2 |
The ID of the user who performed the transaction. |
|
AUD_GID |
Short |
5 |
2 |
The group ID of the user who performed the transaction. |
|
AUD_CMD |
Short |
4 |
2 |
The command number of the transaction performed from the command table below |
|
AUD_RECNO |
Integer |
7 |
4 |
The record number in the associated table which the transaction was performed on. |
* Dependent on SET CENTURY setting.
The AUD_CMD Command Reference Numbers are as follows:
|
Command |
Number |
|
DELETE |
14 |
|
RECALL |
36 |
|
REPLACE |
41 |
|
BROWSE |
6 |
|
CHANGE |
8 |
|
EDIT |
17 |
|
INSERT |
26 |
|
APPEND |
5 |
|
READ |
35 |
Since journal files are standard Recital tables, you can use standard Recital commands such as the REPORT command to print audit trails, transaction logs, etc.
//Enable journaling for the southwind!orders table open database southwind use orders set journal to /journals/ord_journ //.. transactions close data //View journaled records use /journals/ord_journ.dbj
Click image to display full size
Fig 1: Journal Record Example.
The SET JOURNAL TO command without a <.dbj filename> specified closes the active journal file and no further journaling will take place on the active table until the SET JOURNAL TO <.dbj filename> is reissued.
The journaling features are mainly used with shared tables. It should be noted that there is an overhead in enabling transaction journaling, as records updated in a table are also written to the journal file. When records are appended into a journal file, locking is automatically performed so that multiple users can update the journal concurrently. The associated table must be opened shareable for this to occur. Each table can have a journal file associated with it.
The SET JOURNAL ON | OFF command enables or disables transaction journaling. This command is primarily used in applications where journaling can be disabled for a certain class of operations. By default, SET JOURNAL is ON, but no journal files are set.
NOTE: Only the first 249 fields of a table can be journaled: subsequent fields are ignored. The maximum number of fields in a Recital table is 256.
RECOVER
RECOVER FROM <.dbj filename> | ()
The RECOVER command uses the journal file to reapply lost transactions to a previous backup of the data after a fatal error such as a disk head crash. The FROM clause specifies the journal file to use. The file name can be substituted with an <expC>, enclosed in round brackets, which returns a valid filename. If no file extension is specified, then ‘.dbj’ is assumed.
Regular backups are essential to the successful use of After Image Journaling. It is also very important to reinitialize the journal file after each backup: either open the journal file as you would a normal table and use the ZAP command, or delete the file completely. If a fatal error occurs, such as a disk head crash, the table and index files must be restored from a backup, then the RECOVER command executed. RECOVER will reapply' all of the transactions in the journal file to the table, and update the indexes. After the RECOVER command has completed, you can continue with normal processing.
//Create a backup of the southwind!orders table //...backup table and associated files //Reinitialize the journal file erase /journals/ord_journ.dbj //Enable journaling for the southwind!orders table open database southwind use orders set journal to /journals/ord_journ //.. transactions //Restore the backup of the southwind!orders table //...restore //Open the restored backup open database southwind use orders //Reapply the transactions using the journal recover from /journals/ord_journ.dbj //Now, enable the journal file again or //restart with a new backup
Journaling Memo Fields
By default, memo fields - variable length text fields - are not journaled due to the possible storage overhead of multiple copies of potentially large blocks of text. But, if memo journaling is required, the SET MEMOJOURNAL ON command can be used to enable this.
SET MEMOJOURNAL
SET MEMOJOURNAL ON | OFF | ()
The SET MEMOJOURNAL command causes memo fields to be journaled when journaling is set on a table. This command allows the optional logical expression <expL> to be evaluated. If a value of .T. is returned, MEMOJOURNAL is set ON. If a value of .F. is returned, MEMOJOURNAL is set OFF. By default SET MEMOJOURNAL is OFF.
Like a normal Recital table, the journal holds only a pointer to a data block in an associated memo file, not the actual memo data itself. The journal's memo file has a file extension of .dbm rather than the standard Recital .dbt. Therefore, if the journal is being opened as a table, in order to view the journal's memo data, the SET MEMOEXT command should be used.
//Enable journaling for the southwind!suppliers table open database southwind use suppliers set journal to /journals/sup_journ //.. transactions close data //Set filename extension for memo file set memoext to '.dbm' //View journaled records use /journals/sup_journ.dbj
Summary
The After Image Journaling enabled by the SET JOURNAL and RECOVER commands can be used in conjunction with a strict backup regime to minimize data loss in cases where tables become damaged or irretrievable. Journal files can be accessed like standard Recital tables and provide detailed information about the transactions applied to a table, so can be used for auditing purposes.
DB_SAMBA=YES ;export DB_SAMBA
preserve case = nodefault case = lowermangle case = yes
oplocks = False
share modes = no
APPEND FROM <table-name>Before when appending into a shared Recital table each new row was locked along with the table header, then unlocked after it was inserted. This operation has now been enhanced to lock the table once, complete inserting all the rows from the table and then unlock the table. The performance of this operation has been increased by using this method. All the database and table constraints are still enforced.
If when your attempt to create device meta-data fails this is drbd preventing you from corrupting a file system present on the target partition.
$ drbdadm create-md drbd0
v08 Magic number not found
md_offset 30005817344
al_offset 30005784576
bm_offset 30004867072
Found ext2 filesystem which uses 190804004 kB
current configuration leaves usable 29301628 kB
Device size would be truncated, which
would corrupt data and result in
'access beyond end of device' errors.
You need to either
* use external meta data (recommended)
* shrink that filesystem first
* zero out the device (destroy the filesystem)
Operation refused.
Command 'drbdmeta /dev/drbd0 v08 /dev/sda4 internal create-md' terminated with exit code 40
drbdadm aborting
Once you have confirmed that the file system present on the target partition is no longer required at the prompt type the following:
Replace /dev/sdaX with the block device you are targeting.
dd if=/dev/zero of=/dev/sdaX bs=1M count=128
Once this has completed the drbdadm create-md drbd0 command will complete with a "success."
$ drbdadm create-md drbd0
v08 Magic number not found
v07 Magic number not found
v07 Magic number not found
v08 Magic number not found
Writing meta data...
initialising activity log
NOT initialized bitmap
New drbd meta data block successfully created.
success
$
Key features of the Recital database include:
- SQL-92 and a broad subset of ANSI SQL 99, as well as extensions
- Cross-platform support
- Stored procedures
- Triggers
- Cursors
- Updatable Views
- System Tables
- Query caching
- High-performance
- Single-User and Multi-User
- Multi-Process
- ACID Transactions
- Referential Integrity
- Cascading Updates and Deletes
- Multi-table Joins
- Row-level Locking
- BLOBs (Binary Large Objects)
- UDFs (User Defined Functions)
- OLTP (On-Line Transaction Processing)
- Drivers for ODBC, JDBC, and .NET
- Sub-SELECTs (i.e. nested SELECTs)
- Embedded database library
- Database timelines providing data undo functionality
- Fault tolerant clustering support
- Hot backup
All temporary files created by Recital are stored in the directory specified by the environment variable DB_TMPDIR.
mkdir /opt/recital/tmp
mount -t tmpfs -o size=1g recitaltmpfs /usr/recital/tmp
In this article Barry Mavin, CEO and Chief Software Architect for Recital, details how to work with Triggers in the Recital Database Server.
Overview
A trigger is a special kind of stored procedure that runs when you modify data in a specified table using one or more of the data modification operations: UPDATE, INSERT, or DELETE.
Triggers can query other tables and can include complex SQL statements. They are primarily useful for enforcing complex business rules or requirements. For example, you can control whether to allow a new order to be inserted based on a customer's current account status.
Triggers are also useful for enforcing referential and data integrity.
Triggers can be used with any data source that is handled natively by the Recital Database Engine. This includes Recital, FoxPro, FoxBASE, Clipper, dBase, CISAM, and RMS data,
Creating and Editing Triggers
To create a new Trigger, right-click the Procedures node in the Databases tree of the Project Explorer and choose Create. To modify an existing Trigger select the Trigger in the Databases Tree in the Project Explorer by double-clicking on it, or select Modify from the context menu. By convertion we recommend that you name your Stored Procedures beginning with "sp_xxx_", user-defined functions with "f_xxx_", and Triggers with "dt_xxx_", where xxx is the name of the table that they are associated with.
Associating Triggers with a Table
Once you have written your Triggers as detailed above you can associate them with the operations performed on a Table by selecting the Table tab.
The Tables tab allows you to select a Trigger procedure by clicking on the small button at the right of the Text field.
Types of Triggers
As can be seen from the Tables tab detailed below, The Recital Database Server handles 6 distinct types of Triggers.
Open Trigger
The Open Trigger is called after is a table is opened but before any operations are performed on it. You can use this trigger to record a log of table usage or provide a programmable means of checing security. If the Trigger procedure returns .F. (false), then the table is not opened. You can use a TRY...CATCH block around the associated command to inform the user.
Close Trigger
The Close Trigger is called just prior to a table being closed. In this trigger you may find it useful to get transaction counts by using the IOSTATS() built-in 4GL function, and record these values in a transaction log.
Update Trigger
The Update Trigger is called prior to a record update operation being performed. You can use this trigger to perform complex application or data specific validation. If the Trigger procedure returns .F. (false), then the record is not updated. You can use inform the user from within the Trigger procedure the reason that the data cannot be updated.
Delete Trigger
The Delete Trigger is called prior to a record delete operation being performed. You can use this trigger to perform complex application or data specific validation such as cross-table lookups e.g. attempting to delete a customer recortd when there are still open orders for that specific customer. If the Trigger procedure returns .F. (false), then the record is not deleted.
Insert Trigger
The Insert Trigger is called prior to a record insert (append) operation being performed. You can use this trigger to perform such tasks as setting up default values of columns within the record. If the Trigger procedure returns .F. (false), then the record is not inserted.
Rollback Trigger
The RollbackTrigger is called prior to a rollback operation being performed from within a form. If the Trigger procedure returns .F. (false), then the record is not rolled back to its original state.
Testing the Trigger
To test run the Trigger, select the Trigger in the Databases Tree in the Project Explorer by double-clicking on it. Once the Database Administrator is displayed, click the Run button to run the Trigger.
Recital provides a wide variety of connectivity solutions to external data sources. This article provides an overview.