Recital

Login Register

DRBD:
DRBD (Distributed Replicated Block Device) forms the storage redundancy portition of a HA cluster setup. Explained in basic terms DRBD provides a means of achieving RAID 1 behavoir over a network, where whole block devices are mirrored accross the network.

To start off you will need 2 indentically sized raw drives or partitions. Many how-to's on the internet assume the use of whole drives, of course this will be better performance, but if you are simply getting familar with the technology you can repartition existing drives to allow for two eqaully sized raw partitions, one on each of the systems you will be using.

There are 3 DRBD replication modes:
• Protocol A: Write I/O is reported as completed as soon as it reached local disk and local TCP send buffer
• Protocol B: Write I/O is reported as completed as soon as it reached local disk and remote TCP buffer cache
• Protocol C: Write I/O is reported as completed as soon as it reached both local and remote disks.

If we were installing the HA cluster on a slow LAN or if the geogrphical seperation of the systems involved was great, then I recommend you opt for asyncronous mirroring (Protocol A) where the notifcation of a completed write operation occurs as soon as the local disk write is performed. This will greatly improve performance.

As we are setting up our HA cluster connected via a fast LAN, we will be using DRBD in fully syncronous mode, protocol C.
Protocol C involves the file system on the active node only being notified that the write operation was finished when the block is written to both disks of the cluster. Protocol C is the most commonly used mode of DRBD.

/etc/drbd.conf

global { usage-count yes; }
common { syncer { rate 10M; } }
resource r0 {
protocol C;
net {
max-buffers 2048;
ko-count 4;
}
on bailey {
device    /dev/drbd0;
disk      /dev/sda4;
address   192.168.1.125:7789;
meta-disk internal;
}
on giskard {
device    /dev/drbd0;
disk      /dev/sda3;
address   192.168.1.127:7789;
meta-disk internal;
}
}

drbd.conf explained:

Global section, usage-count. The DRBD project keeps statistics about the usage of DRBD versions. They do this by contacting a HTTP server each time a new DRBD version is installed on a system. This can be disabled by setting usage-count no;.

The common seciton contains configurations inhereted by all resources defined.
Setting the syncronisation rate, this is accoimplished by going to the syncer section and then assigning a value to the rate setting. The syncronisation rate refers to rate in which the data is being mirrored in the background. The best setting for the syncronsation rate is related to the speed of the network with which the DRBD systems are communicating on. 100Mbps ethernet supports around 12MBps, Giggabit ethernet somewhere around 125MBps.

in the configuration above, we have a resource defined as r0, the nodes are configured in the "on" host subsections.
"Device" configures the path of the logical block device that will be created by DRBD
"Disk" configures the block device that will be used to store the data.
"Address" configures the IP address and port number of the host that will hold this DRBD device.
"Meta-disk" configures the location where the metadata about the DRBD device will be stored.
You can set this to internal and DRBD will use the physical block device to store the information, by recording the metadata within the last sections of the disk.
Once you have created your configuration file, you must conduct the following steps on both the nodes.

Create device metadata.

$ drbdadm create-md r0
v08 Magic number not found
Writing meta data...
initialising activity log
NOT initialized bitmap
New drbd meta data block sucessfully created.
success

Attach the backing device.
$ drbdadm attach r0

Set the syncronisation parameters.
$ drbdadm syncer r0

Connect it to the peer.
$ drbdadm connect r0

Run the service.
$ service drbd start

Heartbeat:

Heartbeat provides the IP redundancy and the service HA functionailty.
On the failure of the primary node the VIP is assigned to the secondary node and the services configured to be HA are started on the secondary node.

Heartbeat configuration:

/etc/ha/ha.conf

## /etc/ha.d/ha.cf on node1
## This configuration is to be the same on both machines
## This example is made for version 2, comment out crm if using version 1
// replace the node variables with the names of your nodes.

crm no
keepalive 1
deadtime 5
warntime 3
initdead 20
bcast eth0
auto_failback yes
node bailey
node giskard

/etc/ha.d/authkeys
// The configuration below set authentication off, and encryption off for the authentication of nodes and their packets.
//Note make sure the authkeys file has the correct permisisions chmod 600

## /etc/ha.d/authkeys
auth 1
1 crc

/etc/ha.d/haresources
//192.168.1.40 is the VIP (Virtual IP) assigned to the cluster.
//the "smb" in the configuration line represents the service we wish to make HA
// /devdrbd0 represents the resource name you configured in the drbd.conf

## /etc/ha.d/haresources
## This configuration is to be the same on both nodes

bailey 192.168.1.40 drbddisk Filesystem::/dev/drbd0::/drbdData::ext3 smb

Published in Blogs
Read more...

RTOS()

Syntax

RTOS( [ <workarea> ] )

Description

The RTOS() function returns all the fields in the current row as a string. The string will begin with the unique row identifier and then the deleted flag, followed by the data in the record. An optional workarea can be specified, otherwise the current workarea will be used

Example

use backup in 0
use accounts in 0
nrecs=reccount()
for i = 1 to nrecs
  if rtos(accounts) != rtos(backup)
     debug("record "+recno()+" don't match")
  endif
next

Published in Blogs
Read more...
Awk is an powerful text processing language that allows you to manipulate files containing columns of data and strings. Awk is extremely useful, both for general operation of Unix commands, and for data transformation.
 
Introduced in Recital 10 is the PIPETOSTR() function (as well as backticks `` inside strings) which can be used  in conjunction with awk to transform recital data.
 
The following links provide good tutorials on awk:
Published in Blogs
Read more...

This article looks at After Image Journaling and audit trails in Recital using SET JOURNAL and associated commands.

Overview

After Image Journaling, used in conjunction with a structured backup policy is an effective disaster recovery solution. Any transaction that takes place on a table that is being journaled is logged in the journal file. In the event of a disk crash or similar event in which the table is lost, the journaled transactions can be reapplied to the latest backup copy of the table. Alternatively or additionally, the journal can be used to provide an audit trail to all modifications made to the table data.

NOTE: Recital also provides Before Image Journaling via BEGIN TRANSACTION / END TRANSACTION blocks, allowing unsuccessful transactions to be rolled back to a set saved state.

SET JOURNAL and RECOVER

Regular backups are an essential routine for any system, but in high-transaction environments restoration of the latest backup can still mean a major loss of data. After image journaling can successfully be used as part of your disaster recovery strategy to minimize data loss and down time. Recital after image journaling functionality is based on the use of the SET JOURNAL and RECOVER commands.

SET JOURNAL
SET JOURNAL TO [<.dbj filename> | ()]
SET JOURNAL ON | OFF | ()

The SET JOURNAL command is used to enable the After Image Journaling and audit trail for the active table. The TO <.dbj filename> clause associates the specified transaction journal file with the active table.  If the journal file does not exist, it will be created.  The filename can be substituted with a <expC>, enclosed in round brackets, which returns a valid filename.  If no file extension is specified, ‘.dbj’ is used. When specifying a journal file, it is recommended that the journal file is stored on a different disk than that which the table is stored on, so that if a fatal disk error occurs, then the journal file will not be lost along with the table. 

//Enable journaling for the southwind!orders table
open database southwind
use orders
set journal to /journals/ord_journ

The <.dbj filename> is a standard table.  It contains seven fields that are specific to a journal file, followed by the first 249 fields of the associated table.

The first seven fields in the journal are:


Field

Type

Display

Storage

Description

AUD_DATE

Date

8 | 10 *

4

The date on which the transaction was performed.

AUD_TIME

Character

8

8

 The time at which the transaction was performed, in the format HH:MM:SS.

AUD_TERM

Character

12

12

The name of the terminal from which the transaction was performed

AUD_UID

Short

5

2

The ID of the user who performed the transaction.

AUD_GID

Short

5

2

The group ID of the user who performed the transaction.

AUD_CMD

Short

4

2

The command number of the transaction performed from the command table below

AUD_RECNO

Integer

7

4

The record number in the associated table which the transaction was performed on.


* Dependent on SET CENTURY setting.

The AUD_CMD Command Reference Numbers are as follows:


Command

Number

DELETE

14

RECALL

36

REPLACE

41

BROWSE

6

CHANGE

8

EDIT

17

INSERT

26

APPEND

5

READ

35


Since journal files are standard Recital tables, you can use standard Recital commands such as the REPORT command to print audit trails, transaction logs, etc.

//Enable journaling for the southwind!orders table
open database southwind
use orders
set journal to /journals/ord_journ
//.. transactions
close data
//View journaled records
use /journals/ord_journ.dbj

Click image to display full size

Fig 1: Journal Record Example.

The SET JOURNAL TO command without a <.dbj filename> specified closes the active journal file and no further journaling will take place on the active table until the SET JOURNAL TO <.dbj filename> is reissued.

The journaling features are mainly used with shared tables.  It should be noted that there is an overhead in enabling transaction journaling, as records updated in a table are also written to the journal file. When records are appended into a journal file, locking is automatically performed so that multiple users can update the journal concurrently.  The associated table must be opened shareable for this to occur.  Each table can have a journal file associated with it. 

The SET JOURNAL ON | OFF command enables or disables transaction journaling.  This command is primarily used in applications where journaling can be disabled for a certain class of operations. By default, SET JOURNAL is ON, but no journal files are set.

NOTE: Only the first 249 fields of a table can be journaled: subsequent fields are ignored. The maximum number of fields in a Recital table is 256.

RECOVER
RECOVER FROM <.dbj filename> | ()

The RECOVER command uses the journal file to reapply lost transactions to a previous backup of the data after a fatal error such as a disk head crash. The FROM clause specifies the journal file to use. The file name can be substituted with an <expC>, enclosed in round brackets, which returns a valid filename.  If no file extension is specified, then ‘.dbj’ is assumed. 

Regular backups are essential to the successful use of After Image Journaling.   It is also very important to reinitialize the journal file after each backup: either open the journal file as you would a normal table and use the ZAP command, or delete the file completely. If a fatal error occurs, such as a disk head crash, the table and index files must be restored from a backup, then the RECOVER command executed. RECOVER will reapply' all of the transactions in the journal file to the table, and update the indexes.  After the RECOVER command has completed, you can continue with normal processing. 

//Create a backup of the southwind!orders table
//...backup table and associated files
//Reinitialize the journal file
erase /journals/ord_journ.dbj
//Enable journaling for the southwind!orders table
open database southwind
use orders
set journal to /journals/ord_journ
//.. transactions
//Restore the backup of the southwind!orders table
//...restore
//Open the restored backup
open database southwind
use orders
//Reapply the transactions using the journal
recover from /journals/ord_journ.dbj
//Now, enable the journal file again or
//restart with a new backup

Journaling Memo Fields

By default, memo fields - variable length text fields - are not journaled due to the possible storage overhead of multiple copies of potentially large blocks of text. But, if memo journaling is required, the SET MEMOJOURNAL ON command can be used to enable this.

SET MEMOJOURNAL
SET MEMOJOURNAL ON | OFF | ()

The SET MEMOJOURNAL command causes memo fields to be journaled when journaling is set on a table. This command allows the optional logical expression <expL> to be evaluated.  If a value of  .T. is returned, MEMOJOURNAL is set ON.  If a value of .F. is returned, MEMOJOURNAL is set OFF.  By default SET MEMOJOURNAL is OFF.

Like a normal Recital table, the journal holds only a pointer to a data block in an associated memo file, not the actual memo data itself. The journal's memo file has a file extension of .dbm rather than the standard Recital .dbt. Therefore, if the journal is being opened as a table, in order to view the journal's memo data, the SET MEMOEXT command should be used.

//Enable journaling for the southwind!suppliers table
open database southwind
use suppliers
set journal to /journals/sup_journ
//.. transactions
close data
//Set filename extension for memo file
set memoext to '.dbm'
//View journaled records
use /journals/sup_journ.dbj

Summary

The After Image Journaling enabled by the SET JOURNAL and RECOVER commands can be used in conjunction with a strict backup regime to minimize data loss in cases where tables become damaged or irretrievable. Journal files can be accessed like standard Recital tables and provide detailed information about the transactions applied to a table, so can be used for auditing purposes.

Published in Blogs
Read more...
TIP
The Compatibility Dialog settings are written to the compat.db file in <path>/conf - please ensure that the user setting the compatibility settings has write access to this file and directory.  Once these settings are written, the dialog will not be displayed unless SET COMPATIBLE is issued.

Published in Blogs
Read more...

In this article Barry Mavin, CEO and Chief Software Architect for Recital, details on how to use the Client Drivers provided with the Recital Database Server to work with local or remote server-side JDBC data sources.

Overview

The Recital Universal .NET Data Provider provides connectivity to the Recital Database Server running on any supported platform (Windows, Linux, Unix, OpenVMS) using the RecitalConnection object.

The Recital Universal JDBC Driver provides the same functionality for java applications.

The Recital Universal ODBC Driver provides the same functionality for applications that use ODBC.

Each of the above Client Drivers use a connection string to describe connections parameters.

The basic format of a connection string consists of a series of keyword/value pairs separated by semicolons. The equals sign (=) connects each keyword and its value.

The following table lists the valid names for keyword/values.


Name Default Description

Data Source
-or-
Server
-or-Nodename

  The name or network address of the instance of the Recital Database Server which to connect to.
Directory   The target directory on the remote server where data to be accessed resides. This is ignored when a Database is specified.

Encrypt
-or-
Encryption

false When true, DES3 encryption is used for all data sent between the client and server.
Initial Catalog
-or-
Database
  The name of the database on the remote server.
Password
-or-
Pwd
  The password used to authenticate access to the remote server.
User ID   The user name used to authenticate access to the remote server.

Connection Pooling
-or-
Pool

false Enable connection pooling to the server. This provides for one connection to be shared.
Logging false Provides for the ability to log all server requests for debugging purposes
Rowid true When Rowid is true (the default) a column will be post-fixed to each SELECT query that is a unique row identifier. This is used to provide optimised UPDATE and DELETE operations. If you use the RecitalSqlGrid, RecitalSqlForm, or RecitalSqlGridForm components then this column is not visible but is used to handle updates to the underlying data source.
Logfile   The name of the logfile for logging
Gateway  

Opens an SQL gateway(Connection) to a foreign SQL data source on the remote server.
Using Gateways, you can transparently access the following local or remote data sources:

  • Recital
  • Oracle
  • ODBC (Server-side ODBC data sources)
  • JDBC (Server-side JDBC data sources)
  • ADO (Use this to connect to SQL Server and other Native Windows OLEDB data sources)
  • MySQL
  • PostgreSQL
The gateway can be specified in several formats:
servertype@nodename:username/password-database
e.g.
oracle@nodename:username/password-database
mysql@nodename:username/password-database
postgresql@nodename:username/password-database
-or-
odbc:odbc_data_source_name_on_server
oledb:oledb_connection_string_on_server
jdbc:jdbc_driver_path_on_server;jdbc:Recital:args

To connect to a server-side JDBC data source, you ue the gateway=value key/value pair in the following way.

gateway=jdbc:jdbc_driver_path_on_server;jdbc:Recital:args

You can find examples of connection strings for most ODBC and OLE DB data sources by clicking here.

Example in C# using the Recital Universal .NET Data Provider:
////////////////////////////////////////////////////////////////////////
// include the references below
using System.Data;
using Recital.Data;

////////////////////////////////////////////////////////////////////////
// The following code example creates an instance of a DataAdapter that 
// uses a Connection to the Recital Database Server, and a gateway to
// Recital Southwind database. It then populates a DataTable 
// in a DataSet with the list of customers via the JDBC driver. 
// The SQL statement and Connection arguments passed to the DataAdapter 
// constructor are used to create the SelectCommand property of the
// DataAdapter.
public DataSet SelectCustomers()
{
	string gateway = "jdbc:/usr/java/lib/RecitalJDBC/Recital/sql/RecitalDriver;"+
			"jdbc:Recital:Data Source=localhost;database=southwind";       
	RecitalConnection swindConn = new
			RecitalConnection("Data Source=localhost;gateway=\""+gateway+"\");
	RecitalCommand selectCMD = new
			RecitalCommand("SELECT CustomerID, CompanyName FROM Customers", swindConn);
	selectCMD.CommandTimeout = 30;
	RecitalDataAdapter custDA = new RecitalDataAdapter();    
	custDA.SelectCommand = selectCMD;    
	swindConn.Open();    
	DataSet custDS = new DataSet();
	custDA.Fill(custDS, "Customers");    
	swindConn.Close();
	return custDS;
}
Example in Java using the Recital Universal JDBC Driver:
////////////////////////////////////////////////////////////////////////
// standard imports required by the JDBC driver
import java.sql.*;
import java.io.*;
import java.net.URL;
import java.math.BigDecimal;
import Recital.sql.*;

////////////////////////////////////////////////////////////////////////
// The following code example creates a Connection to the Recital // Database Server, and a gateway to the Recital Southwind database. // It then retrieves all the customers via the JDBC driver. public void SelectCustomers() { // setup the Connection URL for JDBC String gateway = "jdbc:/usr/java/lib/RecitalJDBC/Recital/sql/RecitalDriver;"+ "jdbc:Recital:Data Source=localhost;database=southwind"; String url = "jdbc:Recital:Data Source=localhost;gateway=\""+gateway+"\";
// load the Recital Universal JDBC Driver new RecitalDriver(); // create the connection Connection con = DriverManager.getConnection(url); // create the statement Statement stmt = con.createStatement(); // perform the SQL query ResultSet rs = stmt.executeQuery("SELECT CustomerID, CompanyName FROM Customers"); // fetch the data while (rs.next()) { String CompanyID = rs.getString("CustomerID"); String CompanyName = rs.getString("CompanyName"); // do something with the data... } // Release the statement stmt.close(); // Disconnect from the server con.close(); }
Published in Blogs
Read more...
Recital 10 introduced the ECHO command. This command operates in the same way as the PHP ECHO command. It does not append a newline to the output but rather operates in the same way as the existing ?? command in Recital. The string being output can contain C-style string escapes \n \t or \r (newline, tab and carriage return respectively) e.g.
echo "Hello world\n"
Published in Blogs
Read more...
Recital 10.0 introduced the SET DATADIR TO [<directory>] command.. The full syntax is;
SET DATADIR TO [ <directory> ] 
This command is used to specify a  directory where database tables, memos, indexes, and dictionary  files are located. When a table is being opened this directory is searched first before the current directory and the file search path to locate the table and its associated files. This allows the database tables to be relocated to a different file system without the need to change an existing application.
Published in Blogs
Read more...

Recital is a dynamic programming language with an integrated high performance database particularly well suited for the development and deployment of high transaction throughput applications.  Recital 10 further enhances Recital with extensive features and functionality to facilitate its use in fault tolerant high availability systems. Much of the development of Recital 10 was concentrated on performance optimizations and cluster aware functionality to provide an application platform that can be scaled as needed without any application changes. 

Key features of Recital 10 include:

  • Cluster aware database engine that works transparently with drbd, heartbeat, glusterfs and samba
  • High degree of fault tolerance with self healing indexes
  • Massive performance improvements 
  • Extensive internals overall and modernization with superior object-oriented capabilities
  • Chronological data versioning with database timelines
  • SmartQuery caching
  • Database Administration Tools
  • Code and Data Profiling
  • Better integration with unix/linux command shell
  • Incorporates a range of new built-in functions compatible with those in the PHP core libraries
  • Built-in support for outputting data in HTML, XML, and JSON format
  • Seamless SQL command integration into the Recital scripting language
  • Much improved Microsoft FoxPRO language compatibility
  • Numerous extensions and improvements (see below for details)
  • Very large file support (2^63)
Published in Blogs
Read more...

There's a nice article on IBM developerworks describing how to package software using RPM. You can read it here.

Published in Blogs
Read more...
Twitter

Copyright © 2024 Recital Software Inc.

Login

Register

User Registration
or Cancel