In this article Barry Mavin, CEO and Chief Software Architect for Recital provides details on how the Recital Database Server can be used to provide a solution for Universal Data Integration.
Overview
The Recital Database Server handles universal cross-platform data access to a wide range of data sources. The database server natively handles full remote SQL data access to Recital, Visual FoxPro, FoxPro, FoxBASE, Clipper and older dBase data. Using Bridges, it handles full remote SQL data access to C-ISAM and OpenVMS RMS. Using gateway connections, it handles full remote SQL data access to Oracle, MySQL, PostgreSQL, SQL Server, server-side ODBC, server-side JDBC and server-side OLE DB data sources. With its ability to access data using server-side ODBC, JDBC and OLE DB drivers from clients on all supported operating systems (Windows, Linux, Unix, OpenVMS), the Recital Database Server is an ideal Data Integration Solution for applications of all sizes and complexity.
Universal Data Integration Solutions
There are several ways in which data may be accessed by the Database Server.
Table 1:
Client Universal Data Access solutions for accessing local or remote data.
Client | Solution |
---|---|
Recital | Use remote gateway connections |
Visual FoxPro | Use the Universal ODBC Driver |
Java (all platforms) | Use the Universal JDBC Driver |
.NET Framework | Use the Universal .NET Data Provider |
Microsoft Office | Use the Universal ODBC Driver |
Windows Mobile | Use the Universal Compact Framework .NET Data Provider |
PHP on Linux | Use the Universal ODBC Driver for Linux |
Mono on Linux | Use the Universal .NET Data Provider |
Others | If the data source you want to access is not in the list above, then you can use a remote ODBC, JDBC or OLE DB gateway. You can find examples of connection strings for most ODBC and OLE DB data sources by clicking here. |
Table 2:
Windows Server Universal Data Access solutions accessible from any remote client running on Windows, Linux, Unix or OpenVMS:
Data Source | Solution |
---|---|
Recital | Native support (See table 1) |
Visual FoxPro | Native support (See table 1) |
FoxPro | Native support (See table 1) |
FoxBASE | Native support (See table 1) |
Clipper | Native support (See table 1) |
dBase | Native support (See table 1) |
C-ISAM | Use a bridge (See table 1) |
Access | Use a gateway connection gateway="oledb:Provider=Microsoft.Jet.OLEDB.4.0;Data Source=\somepath\mydb.mdb;User Id=admin;Password=;" |
Exchange | Use a gateway connection gateway="oledb:Provider=ExOLEDB.DataSource;Data Source=http://servername/publicstore" |
Excel | Use a gateway connection gateway="oledb:Provider=Microsoft.Jet.OLEDB.4.0;Data Source=C:\MyExcel.xls;" |
Oracle | Use a gateway connection gateway="oledb:Provider=msdaora;Data Source=TheOracleDB;User Id=xxxxx;Password=xxxxx;" |
SQL Server | Use a gateway connection gateway="oledb:Provider=sqloledb;Data Source=Aron1;Initial Catalog=pubs;User Id=sa;Password=asdasd;" |
MySQL | Use a gateway connection gateway="oledb:Provider=MySQLProv;Data Source=mydb;User Id=xxxxx;Password=xxxxx;" |
IBM DB2 | Use a gateway connection gateway="oledb:Provider=DB2OLEDB;Network Transport Library=TCPIP;Network Address=XXX.XXX.XXX.XXX;Initial Catalog=MyCtlg;Package Collection=MyPkgCol;Default Schema=Schema;User ID=MyUser;Password=MyPW" |
Sybase ASA | Use a gateway connection gateway="oledb:Provider=ASAProv;Data source=myASA" |
Sybase ASE | Use a gateway connection gateway="oledb:Provider=Sybase.ASEOLEDBProvider;Srvr=myASEserver,5000;Catalog=myDBname;User Id=username;Password=password" |
IBM Informix | Use a gateway connection gateway="oledb:Provider=Ifxoledbc.2;password=myPw;User ID=myUser;Data Source=dbName@serverName;Persist Security Info=true" |
Ingres | Use a gateway connection gateway="odbc:dsn=data_source_name" |
Firebird | Use a gateway connection gateway="odbc:dsn=data_source_name" |
IBM AS400 iSeries | Use a gateway connection gateway="oledb:PROVIDER=IBMDA400; DATA SOURCE=MY_SYSTEM_NAME;USER ID=myUserName;PASSWORD=myPwd" |
Interbase | Use a gateway connection gateway="oledb:provider=sibprovider;location=localhost:;data source=c:\databases\gdbs\mygdb.gdb;user id=xxxxx;password=xxxxx" |
Others |
If the data source you want to access is not in the list above, then you can use server-side ODBC, JDBC or OLE DB. |
Table 3:
Linux and Unix Server Universal Data Access solutions accessible from any remote client running on Windows, Linux, Unix or OpenVMS:
Data Source | Solution |
---|---|
Recital | Native support (See table 1) |
Visual FoxPro | Native support (See table 1) |
FoxPro | Native support (See table 1) |
FoxBASE | Native support (See table 1) |
Clipper | Native support (See table 1) |
dBase | Native support (See table 1) |
C-ISAM | Use a bridge (See table 1) |
Oracle | Use a gateway connection gateway="oracle:Connection_String" |
MySQL | Use a gateway connection gateway="mysql:Connection_String" |
IBM DB2 | Use a gateway connection gateway="db2:Connection_String" |
PostgreSQL | Use a gateway connection gateway="postgres:Connection_String" |
Others |
If the data source you want to access is not in the list above, then you can use a server-side JDBC driver. |
Table 4:
OpenVMS Server Universal Data Access solutions accessible from any remote client running on Windows, Linux, Unix or OpenVMS:
Data Source | Solution |
---|---|
Recital | Native support (See table 1) |
Visual FoxPro | Native support (See table 1) |
FoxPro | Native support (See table 1) |
FoxBASE | Native support (See table 1) |
Clipper | Native support (See table 1) |
dBase | Native support (See table 1) |
RMS | Use a bridge (See table 1) |
Others |
If the data source you want to access is not in the list above, then you can use a server-side JDBC driver. |
Supported Data Sources
Native Data Access
The Recital Database Server has native built-in support for the following data sources:
- Recital
- Visual FoxPro
- FoxPro
- FoxBASE
- Clipper
- dBase
You can setup tables to work with using the Database Administration Tool in Recital Enterprise Studio.
Bridges
Using Bridges, you can access the following data sources as if they were standard Recital/FoxPro tables:
- CISAM
- OpenVMS RMS
You can setup bridges using the Database Administration Tool in Recital Enterprise Studio.
Gateways/Connections
Using Gateways, you can transparently access the following local or remote data sources:
- Recital
- Oracle
- ODBC (Server-side ODBC data sources)
- JDBC (Server-side JDBC data sources)
- OLEDB Use this to connect to SQL Server and other Windows OLE DB data sources)
- MySQL
- PostgreSQL
Remote Data Object functions
Recital 10 includes a complete and robust set of data source independent functions for accession MySQL, Oracle, DB2 and Postgres. This article explains how to use them.
Client Data Access drivers
Included with the Recital Database Server are three Client drivers. These Client drivers can access any data sources supported by the Recital Database Server. They are not restricted to accessing only Recital data. They can be used to access server-side ODBC, JDBC and OLE DB data sources also.
Recital Universal .NET Data Provider
Use this client driver when building .NET applications with Visual Studio .NET. A data provider in the .NET Framework serves as a bridge between an application and a data source. A data provider is used to retrieve data from a data source and to reconcile changes to that data back to the data source.
Key features of the Recital Universal .NET Data Provider:
- Fully Internet enabled
The Recital Universal .NET Data Provider works across the internet providing access to a wide range of data sources located on remote servers running Windows, Linux, Unix and OpenVMS. - SQL Server compatible
The Recital Universal .NET Data Provider is plug compatible with the .NET Framework SQL Server Data Provider. - Cross-platform Data Integration
Using the Recital Universal .NET Data Provider, you can connect to remote Windows, Linux, Unix or OpenVMS servers and access any data source supported by the Recital Database Server. - Managed code
The Recital Universal .NET Data Adaptor written in C# is 100% .NET Framework managed code. - Runs on Windows Mobile
The Recital Universal .NET Data Adaptor runs under the .NET Compact Framework on Windows Mobile.
Recital Universal JDBC Driver
The JDBC API is the industry standard for database-independent connectivity between the Java programming language and a wide range of databases. The JDBC API provides a call-level API for SQL-based database access. JDBC technology allows you to use the Java programming language to exploit "Write Once, Run Anywhere" capabilities for applications that require access to enterprise data.
Key features of the Recital Universal JDBC Driver:
- Fully Internet enabled
The Recital Universal JDBC driver works across the internet providing access to a wide range of data sources located on remote servers running Windows, Linux, Unix and OpenVMS. - JDBC 3.0 API
The Recital Universal JDBC driver supports the JDBC 3.0 API. - Pure Java Type 3 Driver
The Recital Universal JDBC driver is a 100% pure Java Type 3 driver. - Full Access to Metadata
The JDBC API provides metadata access that enables the development of sophisticated applications that need to understand the underlying facilities and capabilities of a specific database connection. - Cross-platform Data Integration
Using the Recital Universal JDBC driver, you can connect to remote Windows, Linux, Unix or OpenVMS servers and access any data source supported by the Recital Database Server. - No Installation
A pure JDBC technology-based driver does not require special installation; it is automatically downloaded as part of the applet that makes the JDBC calls. The Recital Universal JDBC Driver is 100% java.
Recital Universal ODBC Driver
Connect to remote data from Microsoft Office or other applications that support ODBC data access. The Recital Universal ODBC Driver is also available for Linux and Unix.
Key features of the Recital Universal ODBC Driver:
- Fully Internet enabled
The Recital Universal ODBC driver works across the internet providing access to a wide range of data sources located on remote servers running Windows, Linux, Unix and OpenVMS. - Works with Crystal Reports
The Recital Universal ODBC driver supports the SQL syntax generated by Crystal Reports. - Works with Microsoft Office
The Recital Universal ODBC driver works with Microsoft Office products. - Works with PHP on Linux
The Recital Universal ODBC driver is available for Linux and works with PHP.
To access the menu bar in Recital, press the / key.
Full details on Recital Function Keys can be found in the Key Assist section of the Help menu, or in our documentation wiki here.
Usually, you do not need to setup an email server under Linux. Most GUI email clients support Gmail POP3 and IMAP configurations. But, how do you send mail via the standard /usr/bin/mail user agents in a shell script? Programs such as sendmail / postfix / exim can be configured as a gmail smarthost but they are largely overkill for this use. The ssmtp program is a neat utility that does just that for you via gmail.
This article looks at After Image Journaling and audit trails in Recital using SET JOURNAL and associated commands.
Overview
After Image Journaling, used in conjunction with a structured backup policy is an effective disaster recovery solution. Any transaction that takes place on a table that is being journaled is logged in the journal file. In the event of a disk crash or similar event in which the table is lost, the journaled transactions can be reapplied to the latest backup copy of the table. Alternatively or additionally, the journal can be used to provide an audit trail to all modifications made to the table data.
NOTE: Recital also provides Before Image Journaling via BEGIN TRANSACTION / END TRANSACTION blocks, allowing unsuccessful transactions to be rolled back to a set saved state.
SET JOURNAL and RECOVER
Regular backups are an essential routine for any system, but in high-transaction environments restoration of the latest backup can still mean a major loss of data. After image journaling can successfully be used as part of your disaster recovery strategy to minimize data loss and down time. Recital after image journaling functionality is based on the use of the SET JOURNAL and RECOVER commands.
SET JOURNAL
SET JOURNAL TO [<.dbj filename> | ()] SET JOURNAL ON | OFF | ( )
The SET JOURNAL command is used to enable the After Image Journaling and audit trail for the active table. The TO <.dbj filename> clause associates the specified transaction journal file with the active table. If the journal file does not exist, it will be created. The filename can be substituted with a <expC>, enclosed in round brackets, which returns a valid filename. If no file extension is specified, ‘.dbj’ is used. When specifying a journal file, it is recommended that the journal file is stored on a different disk than that which the table is stored on, so that if a fatal disk error occurs, then the journal file will not be lost along with the table.
//Enable journaling for the southwind!orders table open database southwind use orders set journal to /journals/ord_journ
The <.dbj filename> is a standard table. It contains seven
fields that are specific to a journal file, followed by the first
249 fields of the associated table.
The first seven fields in the journal are:
Field |
Type |
Display |
Storage |
Description |
AUD_DATE |
Date |
8 | 10 * |
4 |
The date on which the transaction was performed. |
AUD_TIME |
Character |
8 |
8 |
The time at which the transaction was performed, in the format HH:MM:SS. |
AUD_TERM |
Character |
12 |
12 |
The name of the terminal from which the transaction was performed |
AUD_UID |
Short |
5 |
2 |
The ID of the user who performed the transaction. |
AUD_GID |
Short |
5 |
2 |
The group ID of the user who performed the transaction. |
AUD_CMD |
Short |
4 |
2 |
The command number of the transaction performed from the command table below |
AUD_RECNO |
Integer |
7 |
4 |
The record number in the associated table which the transaction was performed on. |
* Dependent on SET CENTURY setting.
The AUD_CMD Command Reference Numbers are as follows:
Command |
Number |
DELETE |
14 |
RECALL |
36 |
REPLACE |
41 |
BROWSE |
6 |
CHANGE |
8 |
EDIT |
17 |
INSERT |
26 |
APPEND |
5 |
READ |
35 |
Since journal files are standard Recital tables, you can use standard Recital commands such as the REPORT command to print audit trails, transaction logs, etc.
//Enable journaling for the southwind!orders table open database southwind use orders set journal to /journals/ord_journ //.. transactions close data //View journaled records use /journals/ord_journ.dbj
Click image to display full size
Fig 1: Journal Record Example.
The SET JOURNAL TO command without a <.dbj filename> specified closes the active journal file and no further journaling will take place on the active table until the SET JOURNAL TO <.dbj filename> is reissued.
The journaling features are mainly used with shared tables. It should be noted that there is an overhead in enabling transaction journaling, as records updated in a table are also written to the journal file. When records are appended into a journal file, locking is automatically performed so that multiple users can update the journal concurrently. The associated table must be opened shareable for this to occur. Each table can have a journal file associated with it.
The SET JOURNAL ON | OFF command enables or disables transaction journaling. This command is primarily used in applications where journaling can be disabled for a certain class of operations. By default, SET JOURNAL is ON, but no journal files are set.
NOTE: Only the first 249 fields of a table can be journaled: subsequent fields are ignored. The maximum number of fields in a Recital table is 256.
RECOVER
RECOVER FROM <.dbj filename> | ()
The RECOVER command uses the journal file to reapply lost transactions to a previous backup of the data after a fatal error such as a disk head crash. The FROM clause specifies the journal file to use. The file name can be substituted with an <expC>, enclosed in round brackets, which returns a valid filename. If no file extension is specified, then ‘.dbj’ is assumed.
Regular backups are essential to the successful use of After Image Journaling. It is also very important to reinitialize the journal file after each backup: either open the journal file as you would a normal table and use the ZAP command, or delete the file completely. If a fatal error occurs, such as a disk head crash, the table and index files must be restored from a backup, then the RECOVER command executed. RECOVER will reapply' all of the transactions in the journal file to the table, and update the indexes. After the RECOVER command has completed, you can continue with normal processing.
//Create a backup of the southwind!orders table //...backup table and associated files //Reinitialize the journal file erase /journals/ord_journ.dbj //Enable journaling for the southwind!orders table open database southwind use orders set journal to /journals/ord_journ //.. transactions //Restore the backup of the southwind!orders table //...restore //Open the restored backup open database southwind use orders //Reapply the transactions using the journal recover from /journals/ord_journ.dbj //Now, enable the journal file again or //restart with a new backup
Journaling Memo Fields
By default, memo fields - variable length text fields - are not journaled due to the possible storage overhead of multiple copies of potentially large blocks of text. But, if memo journaling is required, the SET MEMOJOURNAL ON command can be used to enable this.
SET MEMOJOURNAL
SET MEMOJOURNAL ON | OFF | ()
The SET MEMOJOURNAL command causes memo fields to be journaled when journaling is set on a table. This command allows the optional logical expression <expL> to be evaluated. If a value of .T. is returned, MEMOJOURNAL is set ON. If a value of .F. is returned, MEMOJOURNAL is set OFF. By default SET MEMOJOURNAL is OFF.
Like a normal Recital table, the journal holds only a pointer to a data block in an associated memo file, not the actual memo data itself. The journal's memo file has a file extension of .dbm rather than the standard Recital .dbt. Therefore, if the journal is being opened as a table, in order to view the journal's memo data, the SET MEMOEXT command should be used.
//Enable journaling for the southwind!suppliers table open database southwind use suppliers set journal to /journals/sup_journ //.. transactions close data //Set filename extension for memo file set memoext to '.dbm' //View journaled records use /journals/sup_journ.dbj
Summary
The After Image Journaling enabled by the SET JOURNAL and RECOVER commands can be used in conjunction with a strict backup regime to minimize data loss in cases where tables become damaged or irretrievable. Journal files can be accessed like standard Recital tables and provide detailed information about the transactions applied to a table, so can be used for auditing purposes.
iptables -I INPUT -j ACCEPT -p tcp --destination-port 8001 -i lo
iptables -A INPUT -j DROP -p tcp --destination-port 8001 -i eth0
APPEND FROM TYPE CSV <file-name.csv>The TYPE keyword has now been enhanced to support a comma separated values (CSV) format
$hdiutil create /tmp/tmp.dmg -ov -volname "RecitalInstall" -fs HFS+ -srcfolder "/tmp/macosxdist/"
$hdiutil convert /tmp/tmp.dmg -format UDZO -o RecitalInstall.dmg
Recital is a dynamic programming language with an integrated high performance database particularly well suited for the development and deployment of high transaction throughput applications. Recital 10 further enhances Recital with extensive features and functionality to facilitate its use in fault tolerant high availability systems. Much of the development of Recital 10 was concentrated on performance optimizations and cluster aware functionality to provide an application platform that can be scaled as needed without any application changes.
Key features of Recital 10 include:
- Cluster aware database engine that works transparently with drbd, heartbeat, glusterfs and samba
- High degree of fault tolerance with self healing indexes
- Massive performance improvements
- Extensive internals overall and modernization with superior object-oriented capabilities
- Chronological data versioning with database timelines
- SmartQuery caching
- Database Administration Tools
- Code and Data Profiling
- Better integration with unix/linux command shell
- Incorporates a range of new built-in functions compatible with those in the PHP core libraries
- Built-in support for outputting data in HTML, XML, and JSON format
- Seamless SQL command integration into the Recital scripting language
- Much improved Microsoft FoxPRO language compatibility
- Numerous extensions and improvements (see below for details)
- Very large file support (2^63)
Here's how to set up field validation based on dynamic values from another table.
Using the products.dbf table from the southwind sample database, validation can be added to the categoryid field to ensure it matches an existing categoryid from the categories.dbf table.
open database southwindThe rlookup() function checks whether an expression exists in the index (master or specified) of the specified table . An attempt to update categoryid with a value not in the list will give an error: Validation on field 'CATEGORYID' failed.
alter table products add constraint;
(categoryid set check rlookup(products.categoryid,categories))
If you have access to the Recital Workbench, you can use the modify structure worksurface to add and alter your dictionary entries, including a customized error message if required.
