The goal of the SCPlugin project is to integrate Subversion into the Mac OS X Finder.
- Support for Subversion.
- Access to commonly used source control operations via contextual menu [screenshot]
- Dynamic icon badging for files under version control. Shows the status of your files visually. [ screenshot ]
This article looks at After Image Journaling and audit trails in Recital using SET JOURNAL and associated commands.
Overview
After Image Journaling, used in conjunction with a structured backup policy is an effective disaster recovery solution. Any transaction that takes place on a table that is being journaled is logged in the journal file. In the event of a disk crash or similar event in which the table is lost, the journaled transactions can be reapplied to the latest backup copy of the table. Alternatively or additionally, the journal can be used to provide an audit trail to all modifications made to the table data.
NOTE: Recital also provides Before Image Journaling via BEGIN TRANSACTION / END TRANSACTION blocks, allowing unsuccessful transactions to be rolled back to a set saved state.
SET JOURNAL and RECOVER
Regular backups are an essential routine for any system, but in high-transaction environments restoration of the latest backup can still mean a major loss of data. After image journaling can successfully be used as part of your disaster recovery strategy to minimize data loss and down time. Recital after image journaling functionality is based on the use of the SET JOURNAL and RECOVER commands.
SET JOURNAL
SET JOURNAL TO [<.dbj filename> | ()] SET JOURNAL ON | OFF | ( )
The SET JOURNAL command is used to enable the After Image Journaling and audit trail for the active table. The TO <.dbj filename> clause associates the specified transaction journal file with the active table. If the journal file does not exist, it will be created. The filename can be substituted with a <expC>, enclosed in round brackets, which returns a valid filename. If no file extension is specified, ‘.dbj’ is used. When specifying a journal file, it is recommended that the journal file is stored on a different disk than that which the table is stored on, so that if a fatal disk error occurs, then the journal file will not be lost along with the table.
//Enable journaling for the southwind!orders table open database southwind use orders set journal to /journals/ord_journ
The <.dbj filename> is a standard table. It contains seven
fields that are specific to a journal file, followed by the first
249 fields of the associated table.
The first seven fields in the journal are:
|
Field |
Type |
Display |
Storage |
Description |
|
AUD_DATE |
Date |
8 | 10 * |
4 |
The date on which the transaction was performed. |
|
AUD_TIME |
Character |
8 |
8 |
The time at which the transaction was performed, in the format HH:MM:SS. |
|
AUD_TERM |
Character |
12 |
12 |
The name of the terminal from which the transaction was performed |
|
AUD_UID |
Short |
5 |
2 |
The ID of the user who performed the transaction. |
|
AUD_GID |
Short |
5 |
2 |
The group ID of the user who performed the transaction. |
|
AUD_CMD |
Short |
4 |
2 |
The command number of the transaction performed from the command table below |
|
AUD_RECNO |
Integer |
7 |
4 |
The record number in the associated table which the transaction was performed on. |
* Dependent on SET CENTURY setting.
The AUD_CMD Command Reference Numbers are as follows:
|
Command |
Number |
|
DELETE |
14 |
|
RECALL |
36 |
|
REPLACE |
41 |
|
BROWSE |
6 |
|
CHANGE |
8 |
|
EDIT |
17 |
|
INSERT |
26 |
|
APPEND |
5 |
|
READ |
35 |
Since journal files are standard Recital tables, you can use standard Recital commands such as the REPORT command to print audit trails, transaction logs, etc.
//Enable journaling for the southwind!orders table open database southwind use orders set journal to /journals/ord_journ //.. transactions close data //View journaled records use /journals/ord_journ.dbj
Click image to display full size
Fig 1: Journal Record Example.
The SET JOURNAL TO command without a <.dbj filename> specified closes the active journal file and no further journaling will take place on the active table until the SET JOURNAL TO <.dbj filename> is reissued.
The journaling features are mainly used with shared tables. It should be noted that there is an overhead in enabling transaction journaling, as records updated in a table are also written to the journal file. When records are appended into a journal file, locking is automatically performed so that multiple users can update the journal concurrently. The associated table must be opened shareable for this to occur. Each table can have a journal file associated with it.
The SET JOURNAL ON | OFF command enables or disables transaction journaling. This command is primarily used in applications where journaling can be disabled for a certain class of operations. By default, SET JOURNAL is ON, but no journal files are set.
NOTE: Only the first 249 fields of a table can be journaled: subsequent fields are ignored. The maximum number of fields in a Recital table is 256.
RECOVER
RECOVER FROM <.dbj filename> | ()
The RECOVER command uses the journal file to reapply lost transactions to a previous backup of the data after a fatal error such as a disk head crash. The FROM clause specifies the journal file to use. The file name can be substituted with an <expC>, enclosed in round brackets, which returns a valid filename. If no file extension is specified, then ‘.dbj’ is assumed.
Regular backups are essential to the successful use of After Image Journaling. It is also very important to reinitialize the journal file after each backup: either open the journal file as you would a normal table and use the ZAP command, or delete the file completely. If a fatal error occurs, such as a disk head crash, the table and index files must be restored from a backup, then the RECOVER command executed. RECOVER will reapply' all of the transactions in the journal file to the table, and update the indexes. After the RECOVER command has completed, you can continue with normal processing.
//Create a backup of the southwind!orders table //...backup table and associated files //Reinitialize the journal file erase /journals/ord_journ.dbj //Enable journaling for the southwind!orders table open database southwind use orders set journal to /journals/ord_journ //.. transactions //Restore the backup of the southwind!orders table //...restore //Open the restored backup open database southwind use orders //Reapply the transactions using the journal recover from /journals/ord_journ.dbj //Now, enable the journal file again or //restart with a new backup
Journaling Memo Fields
By default, memo fields - variable length text fields - are not journaled due to the possible storage overhead of multiple copies of potentially large blocks of text. But, if memo journaling is required, the SET MEMOJOURNAL ON command can be used to enable this.
SET MEMOJOURNAL
SET MEMOJOURNAL ON | OFF | ()
The SET MEMOJOURNAL command causes memo fields to be journaled when journaling is set on a table. This command allows the optional logical expression <expL> to be evaluated. If a value of .T. is returned, MEMOJOURNAL is set ON. If a value of .F. is returned, MEMOJOURNAL is set OFF. By default SET MEMOJOURNAL is OFF.
Like a normal Recital table, the journal holds only a pointer to a data block in an associated memo file, not the actual memo data itself. The journal's memo file has a file extension of .dbm rather than the standard Recital .dbt. Therefore, if the journal is being opened as a table, in order to view the journal's memo data, the SET MEMOEXT command should be used.
//Enable journaling for the southwind!suppliers table open database southwind use suppliers set journal to /journals/sup_journ //.. transactions close data //Set filename extension for memo file set memoext to '.dbm' //View journaled records use /journals/sup_journ.dbj
Summary
The After Image Journaling enabled by the SET JOURNAL and RECOVER commands can be used in conjunction with a strict backup regime to minimize data loss in cases where tables become damaged or irretrievable. Journal files can be accessed like standard Recital tables and provide detailed information about the transactions applied to a table, so can be used for auditing purposes.
SE Linux is a feature of the Linux kernel that provides mandatory access control. This policy based access control system grants far greater control over the resources on a machine than standard Linux access controls such as permissions.
Many modern Linux distributions are shipping with SELinux enabled by default, Fedora 14 and Rhel 6 both install with it enabled.
When you run Recital Web on a SELinux enabled machine and navigate to the default.rsp page you will see something similar to the screen shot below.
If you launch the SELinux troubleshooter you will see the following problem.
SELinux is blocking the apache server from accessing the Recital server running on port 8001.
To manage you SELinux policy you must have the policycoreutils package group installed. The policycoreutils contains the policy core utilities that are required for basic operation of a SELinux system.
If you wish to use a GUI tool, you must install the policycoreutils-gui package.
At the command prompt execute the following:
As root
$ yum install policycoreutils
$ semanage port -a -t http_port_t -p tcp 8001
$ service recital restart
$ service httpd restart
We use the semanage command here to allow the http server access to port 8001. Once you have completed the steps detailed above you can go and navigate back to the default.rsp page in your borwser, where you will find the permission denied message is now replaced by the default.rsp page.
SELinux does a great job of restricting services and daemons so rather than simply disabling it, why not work with it!
When it comes to security, every little bit helps...
The getUIComponentBitmapData method can create bitmapdata for a given IUIComponent. Pass any UIcomponent to get its respective bitmapdata.
public static function getUIComponentBitmapData(target:IUIComponent):BitmapData {
var resultBitmapData:BitmapData = new BitmapData(target.width, target.height);
var m:Matrix = new Matrix();
resultBitmapData.draw(target, m);
return resultBitmapData;
}
Now convert the bitmapdata to a jpeg bytearray.
private static function encodeToJPEG(data:BitmapData, quality:Number = 75):ByteArray {
var encoder:JPGEncoder = new JPGEncoder(quality);
return encoder.encode(data);
}
Now encode the ByteArray into Base64.
public static function base64Encode(data:ByteArray):String {
var encoder:Base64Encoder = new Base64Encoder();
encoder.encodeBytes(data);
return encoder.flush();
}
Upload the base64 encoded ByteArray to the server.
public static uploadData():void {
var url:String = "saveFile.php";
var urlRequest:URLRequest = new URLRequest(url);
urlRequest.method = URLRequestMethod.POST;
var urlLoader:URLLoader = new URLLoader();
var urlVariables:URLVariables = new URLVariables();
urlVariables.file = jpgEncodedFile; // as returned from base64Encode()
urlLoader.data = urlVariables;
urlLoader.load(urlRequest);
}
The saveFile.php file on the server.
DRBD:
DRBD (Distributed Replicated Block Device) forms the storage redundancy portition of a HA cluster setup. Explained in basic terms DRBD provides a means of achieving RAID 1 behavoir over a network, where whole block devices are mirrored accross the network.
To start off you will need 2 indentically sized raw drives or partitions. Many how-to's on the internet assume the use of whole drives, of course this will be better performance, but if you are simply getting familar with the technology you can repartition existing drives to allow for two eqaully sized raw partitions, one on each of the systems you will be using.
There are 3 DRBD replication modes:
• Protocol A: Write I/O is reported as completed as soon as it reached local disk and local TCP send buffer
• Protocol B: Write I/O is reported as completed as soon as it reached local disk and remote TCP buffer cache
• Protocol C: Write I/O is reported as completed as soon as it reached both local and remote disks.
If we were installing the HA cluster on a slow LAN or if the geogrphical seperation of the systems involved was great, then I recommend you opt for asyncronous mirroring (Protocol A) where the notifcation of a completed write operation occurs as soon as the local disk write is performed. This will greatly improve performance.
As we are setting up our HA cluster connected via a fast LAN, we will be using DRBD in fully syncronous mode, protocol C.
Protocol C involves the file system on the active node only being notified that the write operation was finished when the block is written to both disks of the cluster. Protocol C is the most commonly used mode of DRBD.
/etc/drbd.conf
global { usage-count yes; }
common { syncer { rate 10M; } }
resource r0 {
protocol C;
net {
max-buffers 2048;
ko-count 4;
}
on bailey {
device /dev/drbd0;
disk /dev/sda4;
address 192.168.1.125:7789;
meta-disk internal;
}
on giskard {
device /dev/drbd0;
disk /dev/sda3;
address 192.168.1.127:7789;
meta-disk internal;
}
}
drbd.conf explained:
Global section, usage-count. The DRBD project keeps statistics about the usage of DRBD versions. They do this by contacting a HTTP server each time a new DRBD version is installed on a system. This can be disabled by setting usage-count no;.
The common seciton contains configurations inhereted by all resources defined.
Setting the syncronisation rate, this is accoimplished by going to the syncer section and then assigning a value to the rate setting. The syncronisation rate refers to rate in which the data is being mirrored in the background. The best setting for the syncronsation rate is related to the speed of the network with which the DRBD systems are communicating on. 100Mbps ethernet supports around 12MBps, Giggabit ethernet somewhere around 125MBps.
in the configuration above, we have a resource defined as r0, the nodes are configured in the "on" host subsections.
"Device" configures the path of the logical block device that will be created by DRBD
"Disk" configures the block device that will be used to store the data.
"Address" configures the IP address and port number of the host that will hold this DRBD device.
"Meta-disk" configures the location where the metadata about the DRBD device will be stored.
You can set this to internal and DRBD will use the physical block device to store the information, by recording the metadata within the last sections of the disk.
Once you have created your configuration file, you must conduct the following steps on both the nodes.
Create device metadata.
$ drbdadm create-md r0
v08 Magic number not found
Writing meta data...
initialising activity log
NOT initialized bitmap
New drbd meta data block sucessfully created.
success
Attach the backing device.
$ drbdadm attach r0
Set the syncronisation parameters.
$ drbdadm syncer r0
Connect it to the peer.
$ drbdadm connect r0
Run the service.
$ service drbd start
Heartbeat:
Heartbeat provides the IP redundancy and the service HA functionailty.
On the failure of the primary node the VIP is assigned to the secondary node and the services configured to be HA are started on the secondary node.
Heartbeat configuration:
/etc/ha/ha.conf
## /etc/ha.d/ha.cf on node1
## This configuration is to be the same on both machines
## This example is made for version 2, comment out crm if using version 1
// replace the node variables with the names of your nodes.
crm no
keepalive 1
deadtime 5
warntime 3
initdead 20
bcast eth0
auto_failback yes
node bailey
node giskard
/etc/ha.d/authkeys
// The configuration below set authentication off, and encryption off for the authentication of nodes and their packets.
//Note make sure the authkeys file has the correct permisisions chmod 600
## /etc/ha.d/authkeys
auth 1
1 crc
/etc/ha.d/haresources
//192.168.1.40 is the VIP (Virtual IP) assigned to the cluster.
//the "smb" in the configuration line represents the service we wish to make HA
// /devdrbd0 represents the resource name you configured in the drbd.conf
## /etc/ha.d/haresources
## This configuration is to be the same on both nodes
bailey 192.168.1.40 drbddisk Filesystem::/dev/drbd0::/drbdData::ext3 smb
When installing nomachine on redhat 5.3 64-bit be sure to:
- Make sure you have installed the 64-bit packages as the 32-bit ones will not work.
- add the hostname to /etc/hosts
- Check "Disable encryption of all traffic" (in configuration / advanced tab)
- add the hostname to /etc/hosts
- make sure the host IP is not specified as 127.0.0.1 line
- Uncheck "Disable encryption of all traffic" (in configuration / advanced tab)
The following example connects to a Recital ODBC datasource, executes a query then outputs all the results from the resultset.
<?php
$sql = "select country from customers limit 10";
$conn = odbc_connect('Recital ODBC test', '?', '?');
$rs = odbc_exec($conn, $sql);
odbc_result_all($rs);
odbc_close($conn);
?>
Output:
<table><tr><th>Country</th></tr>
<tr><td>Germany </td></tr>
<tr><td>Mexico </td></tr>
<tr><td>Mexico </td></tr>
<tr><td>UK </td></tr>
<tr><td>Sweden </td></tr>
<tr><td>Germany </td></tr>
<tr><td>France </td></tr>
<tr><td>Spain </td></tr>
<tr><td>France </td></tr>
<tr><td>Canada </td></tr>
</table>
For information on installing and configuring the Recital Universal ODBC Driver and creating and modifying datasources, please see the Documentation section of this web site.
Note: Use of ? for the username and password on the local server is dependent on DB_LOCAL_LOGIN being enabled.
Privacy Policy
We respect and are committed to protecting your privacy.
We require that you provide some personal information, to allow us to provide the services we do to the users of recitalsoftware.com, and to improve recitalsoftware.com. The type of information we collect, how we use it, and what choices you have, are detailed in this policy.
Information Collection and Use
We collect the information you provide when you register for an account or complete an information request form.
We use this information to satisfy your requests for further information, to customize our responses and our future communication with you, and to contact you, regarding development and events in the projects or areas of recitalsofware.com that you have expressed interest in, or in recitalsoftware.com, in general.
We make every effort to allow you to opt-in and opt-out of receiving messages from recitalsoftware.com. However, if you are receiving messages from us and cannot find a way to unsubscribe, please contact us at This email address is being protected from spambots. You need JavaScript enabled to view it..
Information Sharing
We will not release your personal information to anyone by any method, including selling, renting, or sharing, unless:
- you grant us permission
- we are required to do so by law
We will not share personal identification information data, either for single individuals or groups, with any parties, including those affiliated with recitalsoftware.com, such as members or sponsors.
How We Use Cookies
This website uses cookies. A cookie is a small amount of text data, sent from our webserver to your browser, and stored on your device. The cookie is sent back to the webserver each time the browser connects to this site. We use cookies to personalize the site and to streamline your interaction with the site.
It may be possible to configure your browser to refuse cookies, or to ask you to accept each time a cookie is offered. If you choose not to accept cookies, areas of this site may have reduced functionality or performance.
Software on our servers, or third party web statics services, may store your IP address, and other information passed on by your browser (such as browser version, operating system, screen size, language, etc). recitalsoftware.com and/or third party services will aggregate this information to provide usages statistics for this website. We use this information to optimize the design, structure, and performance of this site. In particular, we use Google Analytics to provide usage statistics. For more information, read the Google Analytics Privacy Policy.
Data Security
recitalsoftware.com is also committed to the security of your personal information. We train those who work on recitalsoftware.com on this privacy policy. On our site, we use SSL (Secure Sockets Layer) to protect your personal information, by encrypting your information when you send it to recitalsoftware.com.
Public Forum Content
recitalsoftware.com makes available to its users communication forums, such as mail lists, blogs, and others. Be aware that any information or messages you share in these forums becomes public information immediately. Exercise caution in determining whether to disclose any of your personal information. recitalsoftware.com reserves the right to act as necessary to preserve the integrity of the site and its forums, including removing any and all posts deemed vulgar or inappropriate.
Children's Online Privacy
Regarding children under the age of 13, recitalsoftware.com does not knowingly:
- accept personal information from them
- allow them to become registered members of our web site
Updates to this Privacy Policy
We may update this policy. We will contact you if we make any substantial changes in how we use your personal information.
This privacy policy was last updated on July 1, 2010.
Contact Information
If you have any questions about this privacy policy itself, or on how we use personal information on recitalsoftware.com, please contact us at This email address is being protected from spambots. You need JavaScript enabled to view it..