A number of people have complained about data loss after a system crash when using Ext4.  A bug report was logged. According to multiple posts by a developer of Ext4, Theodore Tso, this is due to differences in approach to security and performance issues between the two file systems.

Ext3 commits writes to disk within approximately 5 seconds - Ext4 can take from 40-150 seconds.  In addition, if a system is using Ext3 and crashes before the commit takes place you will still have the previous contents of a file where under Ext4 the file will be empty.  Theodore Tso feels that this is a failure at the application level and that the file system is behaving as designed and as specified by the POSIX spec (which apparently does not specify what is supposed to happen when a system is not shut down cleanly).  His solution to the issue is to suggest proper use of fsync() and lists various scenarios/examples in post 54 of the bug report (linked above).  In addition he wrote a patch that recognize the rename() situation mentioned in his post 54 yet retains the normal Ext4 behaviors and performance in the majority of cases.  Also a more "proper" solution has been provided which allows the behavior of Ext3 to be retained under Ext4 by mounting it with alloc_on_commit.

A somewhat related topic is the use of on-board caching by hard drives.  This behavior can be modified on most drives by using hdparm.

If you have a live linux site that is running vmware virtual machines, you can pause the virtual machines when you want to perform a backup, I use acronis for linux which does incremental backups and you can specify a command to run before and after the backup, this allows backups to be performed at scheduled times daily without any intervention. In my examples below, the virtual machine i want to control is in /root/vmware/Recital/Recital.vmx

To pause the virtual machine on linux:

# vmrun pause /root/vmware/Recital/Recital.vmx

and then to unpause after the backup has completed:

# vmrun unpause /root/vmware/Recital/Recital.vmx

That is basicallty it, no need to attend the backup and it can be perfomed at a suitable time so no users are affected.

Incidentally, vmrun lets you startup a virtual machine at system startup too, without needing the GUI:

# vmrun start /root/vmware/Recital/Recital.vmx

Or alternatively stop it:

# vmrun stop /root/vmware/Recital/Recital.vmx

There are lots of other capabilities of this command, running applications inside the virtual machine etc etc. For full details just type vmrun in a terminal window.

In this article Barry Mavin, CEO and Chief Software Architect for Recital debunks the myths and misrepresentations surrounding XBase and explains how Recital, an enterprise-class XBase platform, has overcome all the shortfalls and weaknesses of early XBase implementations.

This article looks at the range of client access mechanisms for Windows that can be used with the Recital C-ISAM Bridge and details bridge configuration and usage.

Overview

Just because the format of data is regarded as 'legacy' does not make that data in any way obsolete. Modern client interfaces can not only extend the life of long-term data, but also provide different ways to analyse and gain advantage from that data.

Recital Corporation provides a range of solutions to interface to Informix compliant C-ISAM data on Linux or UNIX from Windows clients.

.NET

Click image to display full size

Fig 1: Recital Mirage .NET application accessing the C-ISAM Demo table.

Recital offers two alternative ways to access C-ISAM data using Microsoft .NET:

The Recital .NET Data Provider is a managed Data Provider written in C# that provides full compatibility with the Microsoft SQLserver and OLE DB data providers that ship with the .NET framework. It is fully integrated with the Visual Studio .NET IDE supporting data binding and automatic code generation using the form designer. The Recital .NET Data Provider works in conjunction with the Recital Database Server for Linux or UNIX to access C-ISAM data.

Recital Mirage .NET is a complete solution for migrating, developing and deploying 4GL database applications. Recital Mirage .NET works in conjunction with the Recital Mirage .NET Application Server for Linux or UNIX to access C-ISAM data.

JDBC

Click image to display full size

Fig 2: Java™ Swing JTable accessing the C-ISAM Demo table via the Recital JDBC Driver.

The Recital JDBC Driver is an all Java Type 4 JDBC 3.0 Driver, allowing you to access C-ISAM data from Java applets and applications. The Recital JDBC Driver works in conjunction with the Recital Database Server for Linux or UNIX to access C-ISAM data.

ODBC

Click image to display full size

Fig 3: Microsoft® Office Excel 2003 Pivot Chart and Pivot Table accessing the C-ISAM Demo table via the Recital ODBC Driver.

The Recital ODBC Driver is an ODBC 3.0 Driver, allowing you to access C-ISAM data from your preferred ODBC based Windows applications. You can develop your own applications in languages such as C++ or Visual Basic, manipulate the data in a spreadsheet package or word processor document and design charts, graphs and reports. The Recital ODBC Driver works in conjunction with the Recital Database Server for Linux or UNIX to access C-ISAM data.

Configuring the Recital C-ISAM Bridge

Data access is achieved through a C-ISAM Bridge.  This requires the creation of an empty Recital table that has the same structure as the C-ISAM file and of a RecitalC-ISAM Bridge file.

On Linux and UNIX, Recital Terminal Developer and the Recital Database Server come complete with an example C-ISAM data file, C-ISAM index and Recital C-ISAM bridge that can be used for testing and as a template for configuring your own C-ISAM bridges.  The Recital Database Server also includes a bridge creation ini file.

Step 1:

Create a Recital table with the same structure as the C-ISAM file.  The fields/columns in the structure file must exactly match the data type and length of those in the C-ISAM file.  The Recital table will have one byte more in total record length due to the Recital record deletion marker.

To create the table, use the SQL CREATE TABLE command or the Recital Terminal Developer CREATE worksurface.  The SQL CREATE TABLE command can be called directly:

SQLExecDirect:
  In:      hstmt = 0x00761BE8,
    szSqlStr = "CREATE TABLE cisamdemo.str (DD Char(4)
              DESCRIPTION "Dd...", cbSqlStr = -3
  Return:  SQL_SUCCESS=0

or be included in a 4GL program:

// createtab.prg
CREATE TABLE cisamdemo.str;
    (DD Char(4) DESCRIPTION "Dd",;
     CONFIRM Char(6) DESCRIPTION "Confirm",;
     PROCDATE Char(6) DESCRIPTION "Procdate",;
     CONTROL Char(5) DESCRIPTION "Control",;
     DOLLARS Decimal(13,2) DESCRIPTION "Dollars",;
     DEALER Char(5) DESCRIPTION "Dealer",;
     TERRITORY Char(2) DESCRIPTION "Territory",;
     WOREP Char(12) DESCRIPTION "Worep",;
     CURRTRAN Char(3) DESCRIPTION "Currtran",;
     TRADDATE Char(6) DESCRIPTION "Traddate",;
     CITY Char(10) DESCRIPTION "City",;
     ACCOUNT Char(11) DESCRIPTION "Account",;
     PRETRAN Char(2) DESCRIPTION "Pretran",;
     AFSREP Char(14) DESCRIPTION "Afsrep",;
     REPKEY Char(9) DESCRIPTION "Repkey",;
     BRANCH Char(3) DESCRIPTION "Branch",;
     WODEALER Char(5) DESCRIPTION "Wodealer",;
     BANKCODE Char(2) DESCRIPTION "Bankcode",;
     COMMRATE Decimal(6,4) DESCRIPTION "Commrate",;
     NEWREP Char(1) DESCRIPTION "Newrep",;
     SETTLE Char(1) DESCRIPTION "Settle",;
     POSTDATE Char(6) DESCRIPTION "Postdate")
if file("cisamdemo.str")
    return .T.
else
    return .F.
endif
// end of createtab.prg

Server-side 4GL programs can be called by all clients, e.g. from a Java class with a JDBC connection:

//---------------------------------
//-- create_str.java --
//---------------------------------
import java.sql.*;
import java.io.*;
import Recital.sql.*;

public class create_str {

  public static void main(String argv[]) {
    try {
      new RecitalDriver();
      String url = "jdbc:Recital:" +
        "SERVERNAME=cisamserver;" +
        "DIRECTORY=/usr/recital/data/southwind;" +
        "USERNAME=user;" +
        "PASSWORD=password";
      Connection con = DriverManager.getConnection(url, "user", "pass");
      Statement stmt = con.createStatement();

      CallableStatement sp = con.prepareCall("{call createtab}");
      boolean res = sp.execute();
      String outParam = sp.getString(1);
      System.out.println("Returned "+outParam);
      sp.close();
      con.close();
    } catch (Exception e) {
      System.out.flush();
      System.err.flush();
      DriverManager.println("Driver exception: " + e.getMessage());
      e.printStackTrace();
    }
    try {
      System.out.println("Press any key to continue...");
      System.in.read();
    } catch(IOException ie) {
      ;
    }
  }
}

The table should be given a ‘.str’ file extension (rather than the default ‘.dbf’) to signify that this is a structure file only.

Please see the end of this article for information on matching Informix and Recital data types

Fig 4: Recital CREATE/MODIFY STRUCTURE worksurface for character mode table creation.

Step 2: Creating the Bridge File

If you have Recital installed on the server platform, the Bridge File can be created using the CREATE BRIDGE worksurface.  The corresponding command to modify the bridge file is MODIFY BRIDGE <bridge file>.  This is the cisamdemo.dbf bridge file in the CREATE/MODIFY BRIDGE WORKSURFACE:

> modify bridge cisamdemo.dbf

Fig 5: Recital CREATE BRIDGE/MODIFY BRIDGE worksurface for bridge creation.

For Recital Database Server clients, the Bridge File can be created using the Recital/SQL CREATE BRIDGE command:

Recital/SQL CREATE BRIDGE:

CREATE BRIDGE cisamdemo.dbf;
  TYPE "CISAM";
  EXTERNAL "cisamdemo.dat";
  METADATA "cisamdemo.str";
  ALIAS "cisamdemo"

or:

CREATE BRIDGE cisamdemo.dbf;
  AS "type=CISAM;external=cisamdemo.dat;metadata=cisamdemo.str;alias=cisamdemo"

The examples above assume that the C-ISAM file, the bridge file and the Recital structure file are all in the current working directory.  Full path information can be specified for the <externalname> and the <databasename>.  For added flexibility, environment variables can be used to determine the path at the time the bridge is opened.  Environment variables can be included for either or both the <externalname> and the <databasename>.  A colon should be specified between the environment variable and the file name.

e.g.

  CREATE BRIDGE cisamdemo.dbf;
  TYPE "CISAM";
  EXTERNAL "DB_DATADIR:cisamdemo";
  METADATA "DB_MIRAGE_PATH:cisamdemo.str";
  ALIAS "cisamdemo"

Recital CREATE BRIDGE/MODIFY BRIDGE worksurface:

Fig 6: Recital CREATE BRIDGE/MODIFY BRIDGE worksurface - using environment variables.

Using the Bridge

The Bridge can now be used.  To access the C-ISAM file, use the ‘alias’ specified in the Bridge definition.

SQL:

SELECT * FROM cisamdemo

Recital/4GL:

use cisamdemo

Indexes

The cisamdemo.dat file included in the Recital distributions for Linux and UNIX has three associated index keys in the cisamdemo.idx file:

Select area: 1
Database in use: cisamdemo
Alias: cisamdemo
Number of records: 4
Current record: 2
File Type: CISAM (C-ISAM)
Master Index: [cisamdemo.idx key #1]
Key: DD+CONFIRM+PROCDATE+CONTROL
Type: Character
Len: 21   (Unique)
Index: [cisamdemo.idx key #2]
Key: DD+SUBSTR(CONFIRM,2,5)+TRADDATE+STR(DOLLARS,13,2)
     +CURRTRAN+ACCOUNT
Type: Character
Len: 42
Index: [cisamdemo.idx key #3]
Key: DEALER+BRANCH+AFSREP+SUBSTR(PROCDATE,5,2)
     +SUBSTR(CONTROL,2,4)
Type: Character
Len: 28

The Recital C-ISAM bridge makes full use of the C-ISAM indexes.  SQL SELECT statements with WHERE clauses are optimized based on any of the existing indexes when possible.  The following ODBC SELECT call makes use of key #3 rather than sequentially searching through the data file.

SQLExecDirect:
  In:      hstmt = 0x00761BE8,
    szSqlStr = "select * from cisamdemo
      where dealer+branch+afsrep=' 00161   595-7912",
      cbSqlStr = -3
  Return:  SQL_SUCCESS=0

Get Data All:

"DD", "CONFIRM", "PROCDATE", "CONTROL", "DOLLARS", "DEALER", 
"TERRITORY", "WOREP", "CURRTRAN", "TRADDATE", "CITY", "ACCOUNT", 
"PRETRAN", "AFSREP", "REPKEY", "BRANCH", "WODEALER", "BANKCODE", 
"COMMRATE", "NEWREP", "SETTLE", "POSTDATE"

"0159", " 15522", "930312", "13356", 4992.60, "00161", "19", 
"", "210", "930305", "", "70000100009", "", "595-7912", 
"930315791", "", "", "59", 0.0000, "1", "", "930315"
1 row fetched from 22 columns.

Using the Recital/4GL, the primary index is set as the master index when the bridge is first opened.  Any secondary indexes can be selected using the SET ORDER TO <expN> command.  The Recital/4GL SEEK or FIND commands and SEEK() function can be used to search in the current master index.

> SET ORDER TO 3
Master index: [cisamdemo.idx key #3]
> SEEK “00161   595-7912”

Appendix 1: Data Types

Informix	Recital
Byte	Numeric
Char	Character
Character	Character
Date	Date
Datetime	Character
Decimal	Numeric
Double Precision	Float
Float	Real
16 Bit Integer	Short
Integer	Numeric
Interval	Character
32 Bit Long	Integer
Money	Numeric
Numeric	Numeric
Real	Numeric
Smallfloat	Numeric
Smallint	Numeric
Text	Unsupported
Varchar	Character

Appendix 2: C-ISAM RDD Error Messages

The following errors relate to the use of the Recital CISAM Replaceable Database Driver (RDD).  They can be received as an ‘errno <expN>’ on Recital error messages:

ERRNO()	Error Description
100	Duplicate record
101	File not open
102	Invalid argument
103	Invalid key description
104	Out of file descriptors
105	Invalid ISAM file format
106	Exclusive lock required
107	Record claimed by another user
108	Key already exists
109	Primary key may not be used
110	Beginning or end of file reached
111	No match was found
112	There is no “current” established
113	Entire file locked by another user
114	File name too long
115	Cannot create lock file
116	Memory allocation request failed
117	Bad custom collating
118	Duplicate primary key allowed
119	Invalid transaction identifier
120	Exclusively locked in a transaction
121	Internal error in journaling
122	Object not locked