In this article Barry Mavin, CEO and Chief Software Architect for Recital provides details on how to use the Recital Universal .NET Data Provider with the Recital Database Server.
Overview
A data provider in the .NET Framework serves as a bridge between an application and a data source. A data provider is used to retrieve data from a data source and to reconcile changes to that data back to the data source.
Each .NET Framework data provider has a DataAdapter object: the .NET Framework Data Provider for OLE DB is the OleDbDataAdapter object, the .NET Framework Data Provider for SQL Server is the SqlDataAdapter object, the .NET Framework Data Provider for ODBC is the OdbcDataAdapter object, and the .NET Framework Data Provider for the Recital Database Server is the RecitalDataAdapter object.
The Recital Universal .NET Data Provider can access any data sources supported by the Recital Database Server. It is not restricted to only access Recital data. It can be used to access server-side ODBC, JDBC and OLE DB data sources also.
Core classes of the Data Provider
The Connection, Command, DataReader, and DataAdapter objects represent the core elements of the .NET Framework data provider model. The Recital Universal .NET Data Provider is plug compatible with the .NET Framework Data Provider for SQL Server. All SQL Server classes are prefixed with "Sql" e.g. SqlDataAdaptor. To use the Recital Universal Data Adaptor, simply change the "Sql" prefix to "Recital" e.g. RecitalDataAdaptor.
The following table describes these objects.
| Object | Description |
|---|---|
| RecitalConnection | Establishes a connection to a specific data source. |
| RecitalCommand | Executes a command against a data source. |
| RecitalDataReader | Reads a forward-only, read-only stream of data from a data source. |
| RecitalDataAdapter | Populates a DataSet and resolves updates with the data source. |
Along with the core classes listed in the preceding table, a .NET Framework data provider also contains the classes listed in the following table.
| Object | Description |
|---|---|
| RecitalTransaction | Enables you to enlist commands in transactions at the data source. |
| RecitalCommandBuilder | A helper object that will automatically generate command properties of a DataAdapter or will derive parameter information from a stored procedure and populate the Parameters collection of a Command object. |
| RecitalParameter | Defines input, output, and return value parameters for commands and stored procedures. |
The Recital Universal .NET Data Provider provides connectivity to the Recital Database Server running on any supported platform (Windows, Linux, Unix, OpenVMS) using the RecitalConnection object. The Recital Universal .NET Data Provider supports a connection string format that is similar to the SQL Server connection string format.
The basic format of a connection string consists of a series of keyword/value pairs separated by semicolons. The equal sign (=) connects each keyword and its value.
The following table lists the valid names for keyword values within the ConnectionString property of the RecitalConnection class.
| Name | Default | Description |
|---|---|---|
| Data Source -or- Server -or- Servername -or- Nodename |
The name or network address of the instance of the Recital Database Server which to connect to. | |
| Directory | The target directory on the remote server where data to be accessed resides. This is ignored when a Database is specified. | |
| Encrypt -or- Encryption |
false | When true, DES3 encryption is used for all data sent between the client and server. |
| Initial Catalog -or- Database |
The name of the database on the remote server. | |
| Password -or- Pwd |
The password used to authenticate access to the remote server. | |
| User ID -or- uid -or- User -or- Username |
The user name used to authenticate access to the remote server. | |
| Connection Pooling -or- Pool |
false | Enable connection pooling to the server. This provides for one connection to be shared. |
| Logging | false | Provides for the ability to log all server requests for debugging purposes |
| Rowid | true | When Rowid is true (the default) a column will be post-fixed to each SELECT query that is a unique row identifier. This is used to provide optimised UPDATE and DELETE operations. If you use the RecitalSqlGrid, RecitalSqlForm, or RecitalSqlGridForm components then this column is not visible but is used to handle updates to the underlying data source. |
| Logfile | The name of the logfile for logging | |
| Gateway |
Opens an SQL gateway(Connection) to a foreign SQL data source on the remote server.
The gateway can be specified in several formats: |
Populating a DataSet from a DataAdaptor
The ADO.NET DataSet is a memory-resident representation of data that provides a consistent relational programming model independent of the data source. The DataSet represents a complete set of data including tables, constraints, and relationships among the tables. Because the DataSet is independent of the data source, a DataSet can include data local to the application, as well as data from multiple data sources. Interaction with existing data sources is controlled through the DataAdapter.
A DataAdapter is used to retrieve data from a data source and populate tables within a DataSet. The DataAdapter also resolves changes made to the DataSet back to the data source. The DataAdapter uses the Connection object of the .NET Framework data provider to connect to a data source and Command objects to retrieve data from and resolve changes to the data source.
The SelectCommand property of the DataAdapter is a Command object that retrieves data from the data source. The InsertCommand, UpdateCommand, and DeleteCommand properties of the DataAdapter are Command objects that manage updates to the data in the data source according to modifications made to the data in the DataSet.
The Fill method of the DataAdapter is used to populate a DataSet with the results of the SelectCommand of the DataAdapter. Fill takes as its arguments a DataSet to be populated, and a DataTable object, or the name of the DataTable to be filled with the rows returned from the SelectCommand.
The Fill method uses the DataReader object implicitly to return the column names and types used to create the tables in the DataSet, as well as the data to populate the rows of the tables in the DataSet. Tables and columns are only created if they do not already exist; otherwise Fill uses the existing DataSet schema.
Examples in C#:
////////////////////////////////////////////////////////////////////////
// include the references below
using System.Data;
using Recital.Data;
////////////////////////////////////////////////////////////////////////
// The following code example creates an instance of a DataAdapter that
// uses a Connection to the Recital Database Server Southwind database
// and populates a DataTable in a DataSet with the list of customers.
// The SQL statement and Connection arguments passed to the DataAdapter
// constructor are used to create the SelectCommand property of the DataAdapter.
public DataSet SelectCustomers()
{
RecitalConnection swindConn = new
RecitalConnection("Data Source=localhost;Initial Catalog=southwind");
RecitalCommand selectCMD = new
RecitalCommand("SELECT CustomerID, CompanyName FROM Customers", swindConn);
selectCMD.CommandTimeout = 30;
RecitalDataAdapter custDA = new RecitalDataAdapter();
custDA.SelectCommand = selectCMD;
swindConn.Open();
DataSet custDS = new DataSet();
custDA.Fill(custDS, "Customers");
swindConn.Close();
return custDS;
}
////////////////////////////////////////////////////////////////////////
// The following example uses the RecitalCommand, RecitalDataAdapter and
// RecitalConnection, to select records from a database, and populate a
// DataSet with the selected rows. The filled DataSet is then returned.
// To accomplish this, the method is passed an initialized DataSet, a
// connection string, and a query string that is a SQL SELECT statement
public DataSet SelectRecitalRows(DataSet dataset, string connection, string query)
{
RecitalConnection conn = new RecitalConnection(connection);
SqlDataAdapter adapter = new RecitalDataAdapter();
adapter.SelectCommand = new RecitalCommand(query, conn);
adapter.Fill(dataset);
return dataset;
} This article talks about the log files available in Recital products and how to enable logging when required.
Overview
When discussing a problem with the Recital Support Team, one of the most common requests is that you enable logging and send them the log files produced. Log files along with error files are designed to provide detailed information about Recital processes and the prevailing environment and can be a fast-track to resolving a problem.
Log Files
There are three main types of log file:
- System Log
- Client/Server Communication Logs
- Custom Logs
System Log
The System log is a system-wide all product log. It tracks all login and logout operations from either Recital or the Recital Server. Logout details include the exit code: 0 for an error-free, 'normal' exit and the error number and message when an error has occurred. It also shows the licenses that have been loaded and any license error codes and messages. The system log filename is recital.log.
Client/Server Communication Logs
The Client/Server communication logs track the requests and responses between the Recital Server and its clients. The log files are as follows:
| Filename | Type | Description |
|---|---|---|
|
dbserver.log |
System-wide |
The Recital Server startup log. This logs any problems with the Recital Server startup. |
|
port.log |
System-wide |
The port listener log. The port listener (or portserver) listens on port 8001 for client connection requests and spawns the appropriate server process. |
|
net.log |
Connection |
The netserver log. The netserver is the Recital Server database and 4GL engine. |
|
rsi.log |
Connection |
The Recital Server Interface (RSI) Gateway log. This logs communication with the Database Gateways to SQL databases. |
|
rec.log |
Connection |
The Recital Database Gateway log. The Recital Database Gateway (or recserver) is the SQL database engine for Recital Gateway data access. |
|
mys.log |
Connection |
The MySQL Database Gateway log. |
|
ora.log |
Connection |
The Oracle Database Gateway log. |
|
inf.log |
Connection |
The Informix Database Gateway log. |
|
ing.log |
Connection |
The Ingres Database Gateway log. |
|
pos.log |
Connection |
The PostgreSQL Database Gateway log. |
|
jdb.log |
Connection |
The JDBC Driver Database Gateway log. |
Custom Logs
The Recital/4GL USERLOG() function can be used to log information to a user-specific log file for debugging or audit trail purposes. For full information on this function, please see the USERLOG() documentation.
Enabling Log Files
For instructions on enabling log files for individual products, please follow these links:
- Recital Server for Windows
- Recital Server for Linux
- Recital Server for UNIX
- Recital Server for OpenVMS
- Recital for Linux
- Recital for UNIX
- Recital for OpenVMS
Enabling Log Files: Recital Server for Windows
To enable the system log file for the Recital Universal Application Server for Windows, include the following command in the UAS\config.db file:
set syslogging on
The Recital Server Manager System Logging tab allows for the viewing and resetting of the System log.
Section
Item
Description
System Logging
DateTime
Date and time stamp of the action.
Name
Login name used by connection
Action
Action logged: Login, Logoff, Errot
Details
Details of action
Buttons
Purge
Allows the log file to be reset
Refresh
Refreshes the display
To set up Client/Server logging, use the Recital Server Manager Settings tab to update the server's Registry entries:
The following Log file settings can be configured:
|
Item |
Description |
|---|---|
|
Log files Directory Path |
Enter the directory in which log files will be created. The default is the UAS\log directory. |
|
Enabled |
Check to enable log file creation. |
|
Versions |
Check to enable log file versioning. |
|
Listener |
Click to view the current port listener log file |
|
Server |
Click to view the current netserver log file |
|
Purge |
Click to purge all log files |
Enabling Log Files: Recital Server for Linux
To enable the system log file for the Recital Server for Linux, include the following command in the conf/config.db file:
set syslogging on
To set up Client/Server logging, the Recital Server can be started with the 'logging' parameter, in which case, all relevant logging will take place.
# service startup logging<
Alternatively, one or more of the following environment variables can be added to the dbserver.conf file or set at the Operating System prompt. The Recital Server must be restarted before environment variable changes will be recognized. Each environment variable should be set to the name of a log file.
Environment Variable
Logs Activity of...
UASLOG_PORT
Port Server (db_rsiserver)
UASLOG_NET
(Net) Server (db_netserver)
UASLOG_ORA
Oracle Server (db_oraserver)
UASLOG_INF
Informix Server (db_infserver)
UASLOG_ING
Ingres Server (db_ingserver)
UASLOG_JDB
JDBC Server (db_jdbserver)
UASLOG_REC
Recital Server (db_recserver)
Extract from recital.conf:
UASLOG_PORT="port.log" ; export UASLOG_PORT UASLOG_NET="net.log" ; export UASLOG_NET UASLOG_ORA="ora.log" ; export UASLOG_ORA UASLOG_INF="inf.log" ; export UASLOG_INF UASLOG_ING="ing.log" ; export UASLOG_ING UASLOG_JDB="jdb.log" ; export UASLOG_JDB UASLOG_REC="rec.log" ; export UASLOG_REC
DB_LOGDIR
If the environment variable DB_LOGDIR is set to an existing directory, all log files will be written to this directory. If not, the log files will be created in the bin directory.
DB_LOGDIR is set in the conf/recital.conf file. By default it is set to the log directory:
DB_LOGDIR=${ROI_ROOT}log/ ; export DB_LOGDIR
DB_LOGVER
If the environment variable DB_LOGVER is greater than 0, version numbers are added to the file names. For example, the activity of the first Net Server process will be logged to net.log, the second to net001.log, the third to net002.log etc. up to the maximum value of DB_LOGVER.
DB_LOGVER is set in the conf/recital.conf file:
DB_LOGVER=10; export DB_LOGVER
Enabling Log Files: Recital Server for UNIX
To enable the system log file for the Recital Server for UNIX, include the following command in the conf/config.db file:
set syslogging on
To set up Client/Server logging, the Recital Server can be started with the 'logging' parameter, in which case, all relevant logging will take place.
# service startup logging
Alternatively, one or more of the following environment variables can be added to the <em>dbserver.conf</em> file or set at the Operating System prompt. The Recital Server must be restarted before environment variable changes will be recognized. Each environment variable should be set to the name of a log file.
Environment Variable
Logs Activity of...
UASLOG_PORT
Port Server (db_rsiserver)
UASLOG_NET
(Net) Server (db_netserver)
UASLOG_ORA
Oracle Server (db_oraserver)
UASLOG_INF
Informix Server (db_infserver)
UASLOG_ING
Ingres Server (db_ingserver)
UASLOG_JDB
JDBC Server (db_jdbserver)
UASLOG_REC
Recital Server (db_recserver)
Extract from recital.conf:
UASLOG_PORT="port.log" ; export UASLOG_PORT UASLOG_NET="net.log" ; export UASLOG_NET UASLOG_ORA="ora.log" ; export UASLOG_ORA UASLOG_INF="inf.log" ; export UASLOG_INF UASLOG_ING="ing.log" ; export UASLOG_ING UASLOG_JDB="jdb.log" ; export UASLOG_JDB UASLOG_REC="rec.log" ; export UASLOG_REC
DB_LOGDIR
If the environment variable DB_LOGDIR is set to an existing directory, all log files will be written to this directory. If not, the log files will be created in the bin directory.
DB_LOGDIR is set in the conf/recital.conf file. By default it is set to the log directory:
DB_LOGDIR=${DB_ROOT}log/ ; export DB_LOGDIR
DB_LOGVER
If the environment variable DB_LOGVER is greater than 0, version numbers are added to the file names. For example, the activity of the first Net Server process will be logged to net.log, the second to net001.log, the third to net002.log etc. up to the maximum value of DB_LOGVER.
DB_LOGVER is set in the conf/recital.conf file:
DB_LOGVER=10; export DB_LOGVER
Enabling Log Files: Recital Universal Application Server for OpenVMS
To enable the system log file for the Recital Universal Application Server for OpenVMS, include the following command in the db_uas:config.db file:
set syslogging on
To set up Client/Server logging, one or more of the following symbols can be added to the <em>db_uas:login.com</em> file. The Recital Server must be restarted before symbol changes will be recognized. Each symbol should be set to the name of a log file.
Symbol
Logs Activity of…
UASLOG_PORT
Port Server (db_rsiserver)
UASLOG_NET
(Net) Server (db_netserver)
UASLOG_ORA
Oracle Server (db_oraserver)
UASLOG_INF
Informix Server (db_infserver)
UASLOG_ING
Ingres Server (db_ingserver)
UASLOG_JDB
JDBC Server (db_jdbserver)
UASLOG_REC
Recital Server (db_recserver)
Extract from db_uas:login.com
$ uaslog_port :== port.log $ uaslog_net :== net.log $ uaslog_ora :== ora.log $ uaslog_inf :== inf.log $ uaslog_ing :== ing.log $ uaslog_jdb :== jdb.log $ uaslog_rec :== rec.log
DB_LOGDIR
If the symbol DB_LOGDIR is set to an existing directory, all log files will be written to this directory. If not, the log files will be created in the UAS directory.
DB_LOGDIR is set in the db_uas:login.com file. By default it is set to the UAS.log] directory:
$db_logdir :== 'db_root'.log] ! system logging directory
DB_LOGVER
If the symbol DB_LOGVER is enabled, version numbers are added to the file names. For example, the activity of the first Net Server process will be logged to net.log, the second to net001.log, the third to net002.log etc.
DB_LOGVER is set in the db_uas:login.com file:
$db_logver :== true ! enable multiple log files
Enabling Log Files: Recital for Linux
To enable the system log file for Recital for Linux, include the following command in the conf/config.db file:
set syslogging on
Enabling Log Files: Recital for UNIX
To enable the system log file for Recital for UNIX, include the following command in the conf/config.db file:
set syslogging on
Enabling Log Files: Recital for OpenVMS
To enable the system log file for Recital for OpenVMS, include the following command in the db_ovd:config.db file:
set syslogging on
In Brief
- Log files provide important information to aid problem resolution, but they are also an overhead, so logging should only be enabled when required, not in normal production operation.
- The System log provides a system-wide view of logins, exits and error codes.
- The System log can be viewed in table format via the SYSLOGGING System Table.
- The System log is enabled using the SET SYSLOGGING ON Recital/4GL command in the conf/config.db file.
- Client/Server logs provide detailed information on client/server requests and responses.
- Client/Server logs are enabled using environment variables, symbols or Registry entries or by specifying the 'logging' parameter when starting the Recital Server.
- The location of log files is determined by the DB_LOGDIR setting.
- Versioning of log files is determined by the DB_LOGVER setting.
- edit the .vmx file and add the following line
uuid.action = "keep"
- set the virtual machine to power off when vmware is stopped. Do not set this to "suspend" or it will not restart on the backup machine.
VMware products, such as ESX, Workstation, Server, and Fusion, come with a built-in VNC server to access guests.
This allows you to connect to the guest without having a VNC server installed in the guest - useful if a server doesn't exist for the guest or if you need access some time when a server would not work (say during the boot process). It's also good in conjunction with Headless Mode.
The VNC server is set up on a per-VM basis, and is disabled by default. To enable it, add the following lines to the .vmx:
RemoteDisplay.vnc.enabled = "TRUE" RemoteDisplay.vnc.port = "5901"
You can set a password with RemoteDisplay.vnc.key; details for how to calculate the obfuscated value given a plaintext password are in Compute hashed password for use with RemoteDisplay.vnc.key.
If you want more than one VM set up in this manner, make sure they have unique port numbers. To connect, use a VNC client pointing at host-ip-address:port. If you connect from a different computer, you may have to open a hole in the OS X firewall. If you use Leopard's Screen Sharing.app on the same computer as Fusion, don't use port 5900 since Screen Sharing refuses to connect to that.
When stress testing our loadbalancer, i was unable to get more than 20 reliable ssh connections. The following error would be reported.
ssh_exchange_identification: Connection closed by remote host
The resolution for this is quite simple.
Edit the /etc/ssh/sshd_config file and increase the MaxStartups. On my server i set this to 200.
Edit /etc/sysctrl.conf and add the following line:
net.core.netdev_max_backlog = 3000
Then apply this change:
# sysctl -p
This article looks at After Image Journaling and audit trails in Recital using SET JOURNAL and associated commands.
Overview
After Image Journaling, used in conjunction with a structured backup policy is an effective disaster recovery solution. Any transaction that takes place on a table that is being journaled is logged in the journal file. In the event of a disk crash or similar event in which the table is lost, the journaled transactions can be reapplied to the latest backup copy of the table. Alternatively or additionally, the journal can be used to provide an audit trail to all modifications made to the table data.
NOTE: Recital also provides Before Image Journaling via BEGIN TRANSACTION / END TRANSACTION blocks, allowing unsuccessful transactions to be rolled back to a set saved state.
SET JOURNAL and RECOVER
Regular backups are an essential routine for any system, but in high-transaction environments restoration of the latest backup can still mean a major loss of data. After image journaling can successfully be used as part of your disaster recovery strategy to minimize data loss and down time. Recital after image journaling functionality is based on the use of the SET JOURNAL and RECOVER commands.
SET JOURNAL
SET JOURNAL TO [<.dbj filename> | ()] SET JOURNAL ON | OFF | ( )
The SET JOURNAL command is used to enable the After Image Journaling and audit trail for the active table. The TO <.dbj filename> clause associates the specified transaction journal file with the active table. If the journal file does not exist, it will be created. The filename can be substituted with a <expC>, enclosed in round brackets, which returns a valid filename. If no file extension is specified, ‘.dbj’ is used. When specifying a journal file, it is recommended that the journal file is stored on a different disk than that which the table is stored on, so that if a fatal disk error occurs, then the journal file will not be lost along with the table.
//Enable journaling for the southwind!orders table open database southwind use orders set journal to /journals/ord_journ
The <.dbj filename> is a standard table. It contains seven
fields that are specific to a journal file, followed by the first
249 fields of the associated table.
The first seven fields in the journal are:
|
Field |
Type |
Display |
Storage |
Description |
|
AUD_DATE |
Date |
8 | 10 * |
4 |
The date on which the transaction was performed. |
|
AUD_TIME |
Character |
8 |
8 |
The time at which the transaction was performed, in the format HH:MM:SS. |
|
AUD_TERM |
Character |
12 |
12 |
The name of the terminal from which the transaction was performed |
|
AUD_UID |
Short |
5 |
2 |
The ID of the user who performed the transaction. |
|
AUD_GID |
Short |
5 |
2 |
The group ID of the user who performed the transaction. |
|
AUD_CMD |
Short |
4 |
2 |
The command number of the transaction performed from the command table below |
|
AUD_RECNO |
Integer |
7 |
4 |
The record number in the associated table which the transaction was performed on. |
* Dependent on SET CENTURY setting.
The AUD_CMD Command Reference Numbers are as follows:
|
Command |
Number |
|
DELETE |
14 |
|
RECALL |
36 |
|
REPLACE |
41 |
|
BROWSE |
6 |
|
CHANGE |
8 |
|
EDIT |
17 |
|
INSERT |
26 |
|
APPEND |
5 |
|
READ |
35 |
Since journal files are standard Recital tables, you can use standard Recital commands such as the REPORT command to print audit trails, transaction logs, etc.
//Enable journaling for the southwind!orders table open database southwind use orders set journal to /journals/ord_journ //.. transactions close data //View journaled records use /journals/ord_journ.dbj
Click image to display full size
Fig 1: Journal Record Example.
The SET JOURNAL TO command without a <.dbj filename> specified closes the active journal file and no further journaling will take place on the active table until the SET JOURNAL TO <.dbj filename> is reissued.
The journaling features are mainly used with shared tables. It should be noted that there is an overhead in enabling transaction journaling, as records updated in a table are also written to the journal file. When records are appended into a journal file, locking is automatically performed so that multiple users can update the journal concurrently. The associated table must be opened shareable for this to occur. Each table can have a journal file associated with it.
The SET JOURNAL ON | OFF command enables or disables transaction journaling. This command is primarily used in applications where journaling can be disabled for a certain class of operations. By default, SET JOURNAL is ON, but no journal files are set.
NOTE: Only the first 249 fields of a table can be journaled: subsequent fields are ignored. The maximum number of fields in a Recital table is 256.
RECOVER
RECOVER FROM <.dbj filename> | ()
The RECOVER command uses the journal file to reapply lost transactions to a previous backup of the data after a fatal error such as a disk head crash. The FROM clause specifies the journal file to use. The file name can be substituted with an <expC>, enclosed in round brackets, which returns a valid filename. If no file extension is specified, then ‘.dbj’ is assumed.
Regular backups are essential to the successful use of After Image Journaling. It is also very important to reinitialize the journal file after each backup: either open the journal file as you would a normal table and use the ZAP command, or delete the file completely. If a fatal error occurs, such as a disk head crash, the table and index files must be restored from a backup, then the RECOVER command executed. RECOVER will reapply' all of the transactions in the journal file to the table, and update the indexes. After the RECOVER command has completed, you can continue with normal processing.
//Create a backup of the southwind!orders table //...backup table and associated files //Reinitialize the journal file erase /journals/ord_journ.dbj //Enable journaling for the southwind!orders table open database southwind use orders set journal to /journals/ord_journ //.. transactions //Restore the backup of the southwind!orders table //...restore //Open the restored backup open database southwind use orders //Reapply the transactions using the journal recover from /journals/ord_journ.dbj //Now, enable the journal file again or //restart with a new backup
Journaling Memo Fields
By default, memo fields - variable length text fields - are not journaled due to the possible storage overhead of multiple copies of potentially large blocks of text. But, if memo journaling is required, the SET MEMOJOURNAL ON command can be used to enable this.
SET MEMOJOURNAL
SET MEMOJOURNAL ON | OFF | ()
The SET MEMOJOURNAL command causes memo fields to be journaled when journaling is set on a table. This command allows the optional logical expression <expL> to be evaluated. If a value of .T. is returned, MEMOJOURNAL is set ON. If a value of .F. is returned, MEMOJOURNAL is set OFF. By default SET MEMOJOURNAL is OFF.
Like a normal Recital table, the journal holds only a pointer to a data block in an associated memo file, not the actual memo data itself. The journal's memo file has a file extension of .dbm rather than the standard Recital .dbt. Therefore, if the journal is being opened as a table, in order to view the journal's memo data, the SET MEMOEXT command should be used.
//Enable journaling for the southwind!suppliers table open database southwind use suppliers set journal to /journals/sup_journ //.. transactions close data //Set filename extension for memo file set memoext to '.dbm' //View journaled records use /journals/sup_journ.dbj
Summary
The After Image Journaling enabled by the SET JOURNAL and RECOVER commands can be used in conjunction with a strict backup regime to minimize data loss in cases where tables become damaged or irretrievable. Journal files can be accessed like standard Recital tables and provide detailed information about the transactions applied to a table, so can be used for auditing purposes.
iptables -I INPUT -j ACCEPT -p tcp --destination-port 8001 -i lo
iptables -A INPUT -j DROP -p tcp --destination-port 8001 -i eth0
This article discusses Recital database security: from operating system file permissions through file and field protection to DES3 encryption.
Overview
A company's data is extremely valuable and must be protected, both in operation and in physical file format. Recital products provide a range of ways to protect your data.
Operating System File Permissions
The most basic level of database security is provided by the operating system. Recital database tables and indexes are individual files with their own respective operating system file permissions. Read permission is required to open a table and write permission to update a table. If a user does not have read permission they are denied access. Without write permission, a table will be opened read-only.
Here the owner, root, and members of the recital group have write permission, so can update the example table unless additional protection applies. Other users can only open the example table read-only.
# ls -l example* -rwxrwxr-x 1 root recital 147 Nov 29 14:27 example.dbd -rwxrwxr-x 1 root recital 41580 Nov 29 14:27 example.dbf -rwxrwxr-x 1 root recital 13312 Nov 29 14:28 example.dbt -rwxrwxr-x 1 root recital 19456 Nov 29 14:28 example.dbx
Note: As in the example above, a table's associated files should have the same permissions as the table itself:
|
File Extension |
File Type |
|
.dbd |
Dictionary |
|
.dbf |
Table |
|
.dbt |
Memo |
|
.dbx |
Index |
Database Dictionary
Each Recital table may have a Database Dictionary. The Dictionary can be used both to protect the integrity of the data and to protect access to the data. This section covers Column Constraints, Triggers, Security and Protection.
Column Constraints: Data Integrity
The Dictionary attributes or constraints either prevent the entry of incorrect data, e.g. must_enter and validation or aid the entry of correct data, e.g. default, picture and choicelist. The Dictionary can be modified in the character mode CREATE/MODIFY STRUCTURE worksurface, via SQL statements, or in the Recital Enterprise Studio Database Administrator.
Click image to display full size
Fig 1: MODIFY STRUCTURE Worksurface: Dictionary.
The SQL Column Constraints are as follows:
|
Constraint |
Description |
|
AUTO_INCREMENT | AUTOINC |
Used to auto increment the value of a column. |
|
CALCULATED |
Used to calculate the value of a column. |
|
CHECK | SET CHECK |
Used to validate a change to the value of a column. |
|
DEFAULT |
Used to set a default value for the specified column. |
|
DESCRIPTION |
Used set the column description for the specified column. |
|
ERROR |
Used to define an error message to be displayed when a validation check fails. |
|
FOREIGN KEY |
Used to define a column as a Foreign Key for a parent table. |
|
NOCPTRANS |
Used to prevent code page translation for character and memo fields. |
|
NOT NULL | NULL |
Used to disallow/allow NULL values. |
|
PRIMARY KEY |
Used to define a tables Primary Key. |
|
RANGE |
Used to specify minimum and maximum values for a date or numerical column. |
|
RECALCULATE |
Used to force recalculation of calculated columns when a columns value changes. |
|
REFERENCES |
Used to create a relationship to an index key of another table. |
|
UNIQUE |
Used to define the column as a candidate index for the table |
These can be specified in CREATE TABLE or ALTER TABLE statements:
exec sql OPEN DATABASE southwind; exec sql ALTER TABLE customers ADD COLUMN timeref char(8) CHECK validtime(timeref) ERROR "Not a valid time string";
Click image to display full size
Fig 2: Database Administrator: Column Constraints and Attributes.
TRIGGERS
Table Level Triggers are event-driven procedures called before an I/O operation. These can be used to introduce another layer of checks before a particular operation is permitted to take place or to simply set up logging of those operations.
The CREATE/MODIFY STRUCTURE worksurface <TRIGGERS> menu bar option allows you to specify table level triggers. You may edit a trigger procedure from within the <TRIGGERS> menu by placing the cursor next to the procedure name and pressing the [HELP] key. A text window pops up for editing. If the table triggers are stored in separate <.prg> files, rather than in a procedure library, procedures need not be predefined (SET PROCEDURE) before using the table.
Click image to display full size
Fig 3: MODIFY STRUCTURE Worksurface: Triggers.
The following triggers can be selected and associated with a specified procedure name in the <TRIGGERS> menu.
|
Trigger |
Description |
|
UPDATE |
The specified procedure is called prior to an update operation on the table. If the procedure returns .F., then the UPDATE is canceled. |
|
DELETE |
The specified procedure is called prior to a delete operation on the table. If the procedure returns .F., then the DELETE is canceled. |
|
APPEND |
The specified procedure is called prior to an append operation on the table. If the procedure returns .F., then the APPEND is canceled. |
|
OPEN |
The specified procedure is called after an open operation on the table. |
|
CLOSE |
The specified procedure is called prior to a close operation on the table. |
|
ROLLBACK |
The specified procedure is called when a user presses the [ABANDON] key in a forms based operation. |
The Recital Enterprise Studio Database Administrator also allows you to associate existing programs as Table Trigger Procedures.
Click image to display full size
Fig 4: Database Administrator: Triggers.
Programmatically, Trigger Procedures can also be associated with a table using SQL. The following table constraints may be applied in the SQL CREATE TABLE and ALTER TABLE statements:
|
Trigger |
Description |
|
ONUPDATE |
The specified procedure is called prior to an update operation
on the table. If the procedure returns .F., then the UPDATE
is canceled. |
|
ONDELETE |
The specified procedure is called prior to a delete operation
on the table. If the procedure returns .F., then the DELETE
is canceled. |
|
ONINSERT |
The specified procedure is called prior to an insert operation
on the table. If the procedure returns .F., then the INSERT
is canceled. |
|
ONOPEN |
The specified procedure is called after an open operation
on the table. |
|
ONCLOSE |
The specified procedure is called prior to a close operation
on the table. |
|
ONROLLBACK |
The specified procedure is called when a user presses the
[ABANDON] key in a forms based operation. |
SECURITY
As mentioned above, all Recital files are subject to Operating System read and write permissions. These permissions can be further refined, while still using the Operating System user and group IDs, in the Security and Protection sections of the Dictionary. The Security section handles table based operations and the Protection section focuses on individual fields.
Security and Protection rules can be defined in the CREATE/MODIFY STRUCTURE worksurface of Recital Terminal Developer, via the SQL GRANT and REVOKE statements or in the Recital Enterprise Studio Database Administrator.
Click image to display full size
Fig 5: MODIFY STRUCTURE Worksurface: Security.
The Security section has table operations for which Access Control Strings can be specified. An Access Control String (ACS) is a range of valid user identification codes, and is used to restrict table operations to certain individuals or groups. Each user on the system is allocated a group number and a user number. The user identification code is the combination of group and user numbers. When constructing an Access Control String of linked user identification codes, wild card characters may be used.
|
Example ACS |
Description |
|
[1,2] |
In group 1, user 2 |
|
[100,*] |
In group 100, all users |
|
[2-7,*] |
In groups 2-7, all users |
|
[*,100-200] |
In all groups, users 100-200 |
|
[1,*]&[2-7,1-7] |
In group 1, all users, in groups 2-7, users 1-7 |
Please note that the maximum ACS length is 254 characters. OpenVMS group and user numbers are stored and specified in octal. On other Operating Systems, group and user numbers are stored and specified in decimal.
Access Control Strings may be associated with the following operations:
|
Operation |
Description |
|
READONLY |
Users specified in the ACS have read-only access to the table. All other users have update access. |
|
UPDATE |
Users specified in the ACS have update access to the table. All other users are restricted to read-only access. |
|
APPEND |
Users specified in the ACS can append records into the table. No other users can append. |
|
DELETE |
Users specified in the ACS can delete records from the table. No other users can delete. |
|
COPY |
Users specified in the ACS can copy records from the table. No other users can copy. |
|
ADMIN |
Users specified in the ACS can use the following commands: |
The corresponding SQL privileges are:
|
Operation |
Description |
|
SELECT |
Users specified in the ACS may name any column in a SELECT statement. All other users have update access. |
|
UPDATE |
Users specified in the ACS may name any column in an UPDATE statement. All other users are restricted to read-only access. |
|
INSERT |
Users specified in the ACS can INSERT rows into the table. No other users can INSERT. |
|
DELETE |
Users specified in the ACS can DELETE rows from the table. No other users can DELETE. |
|
ALTER |
Users specified in the ACS can use the ALTER TABLE statement on this table. |
|
READONLY |
Users specified in the ACS may read any column in a SELECT statement. All other users have update access. |
// Grant insert privilege for the customer table exec sql OPEN DATABASE southwind; exec sql GRANT UPDATE (lastname, firstname) INSERT ON customers TO '[20,100]'; // Grant all privileges to all users exec sql OPEN DATABASE southwind; exec sql GRANT ALL ON shippers TO PUBLIC;
PROTECTION
Security and Protection rules can be defined in the CREATE/MODIFY STRUCTURE worksurface of Recital Terminal Developer, via the SQL GRANT and REVOKE statements or in the Recital Enterprise Studio Database Administrator.
Click image to display full size
Fig 6: Database Administrator: Protection.
The format of the ACS is the same as in <SECURITY> above.
The following protection can be defined:
|
Operation |
Description |
|
READONLY |
Users specified in the ACS have read-only access to the field. All other users have update access. |
|
UPDATE |
Users specified in the ACS have update access to the field. All other users are restricted to read-only access. |
Recital Terminal Developer also has 'HIDDEN' Protection:
|
Operation |
Description |
|
HIDDEN |
Users specified in the ACS see the 'hiddenfield'character rather than the data in the field. All other users see the data. |
Hidden fields can be accessed and viewed on a work surface, but the field contains the hiddenfield character, ?. If the field is referenced in an expression, it will contain the following: blanks for character fields, F for logical fields, 00/00/0000 for date fields and blank for memo fields.
The corresponding SQL privileges are:
|
Operation |
Description |
|
SELECT |
Users specified in the ACS may name the column in a SELECT statement. All other users have update access. |
|
UPDATE |
Users specified in the ACS may name the column in an UPDATE statement. All other users are restricted to read-only access. |
|
READONLY |
Users specified in the ACS may read the column in a SELECT statement. All other users have update access. |
// Grant update privilege for columns lastname and firstname from the customer table exec sql OPEN DATABASE southwind; exec sql GRANT UPDATE (lastname, firstname) customers TO '[20,100]';
Encryption
From Recital 8.5 onwards, Recital installations that have the additional DES3 license option have the ability to encrypt the data held in Recital database tables. Once a database table has been encrypted, the data cannot be accessed unless the correct three-part encryption key is specified, providing additional security for sensitive data.
ENCRYPT
The ENCRYPT Recital 4GL command is used to encrypt the data in the specified table or tables matching a skeleton. If the skeleton syntax is used, then all matching tables will be given the same encryption key. The encryption key is a three part comma-separated key and may optionally be enclosed in angled brackets. Each part of the key can be a maximum of 8 characters. The key is DES3 encrypted and stored in a .dkf file with the same basename as the table. After encryption, the three parts of the key must be specified correctly before the table can be accessed.
// Encrypt individual tables encrypt customers key "key_1,key_2,key_3" encrypt employees key "<key_1,key_2,key_3>" // Encrypt all .dbf files in the directory encrypt *.dbf key "key_1,key_2,key_3"
SET ENCRYPTION
If a database table is encrypted, the correct three-part encryption key must be specified before the table's data or structure can be accessed. The SET ENCRYPTION TO set command can be used to specify a default encryption key to be used whenever an encrypted table is accessed without the key being specified. The encryption key is a three part comma-separated key.
If the command to access the table includes the key, either by appending it to the table filename specification or using an explicit clause, this will take precedence over the key defined by SET ENCRYPTION TO.
Issuing SET ENCRYPTION TO without a key causes any previous setting to be cleared. The key must then be specified for each individual encrypted table.
The default key defined by SET ENCRYPTION is only active when SET ENCRYPTION is ON. SET ENCRYPTION OFF can be used to temporarily disable the default key. The SET ENCRYPTION ON | OFF setting does not change the default key itself. SET ENCRYPTION is ON by default.
// Encrypt individual tables encrypt customers key "key_1,key_2,key_3" encrypt shippers key "key_2,key_3,key_4" // Specify a default encryption key set encryption to "key_1,key_2,key_3" // Open customers table using the default encryption key use customers // Specify shippers table's encryption key use shippers<key_2,key_3,key_4> // Disable the default encryption key set encryption to // Specify the individual encryption keys use customers encryption "key_1,key_2,key_3" use shippers<key_2,key_3,key_4>
DECRYPT
The DECRYPT command is used to decrypt the data in the specified table or tables matching a skeleton. The specified key must contain the three part comma-separated key used to previously encrypt the table and may optionally be enclosed in angled brackets. The skeleton syntax can only be used if all tables matching the skeletonhave the same key.
The DECRYPT command decrypts the data and removes the table’s .dkf file. After decryption, the key need no longer be specified to gain access to the table.
// Decrypt individual tables decrypt customers key "key_1,key_2,key_3" decrypt employees key "<key_1,key_2,key_3>" // Decrypt all .dbf files in the directory decrypt *.dbf key "key_1,key_2,key_3"
All of the following commands are affected when a table is encrypted:
- APPEND FROM
- COPY FILE
- COPY STRUCTURE
- COPY TO
- DIR
- USE
- SQL INSERT
- SQL SELECT
- SQL UPDATE
APPEND FROM
Used to append records to the active table from another table.// The key must be specified for an encrypted source table
use mycustomers append from customers encryption "key_1,key_2,key_3"; for country = "UK"
COPY FILE
Used to copy a file.// The key file must also be copied for an encrypted source table // as the target table will be encrypted
encrypt customers key "key_1,key_2,key_3" copy file customers.dbf to newcustomers.dbf copy file customers.dkf to newcustomers.dkf use newcustomers encryption "key_1,key_2,key_3"
COPY STRUCTURE
Used to copy a table's structure to a new table.// The key file is automatically copied for an encrypted source table // and the target table encrypted
encrypt customers key "key_1,key_2,key_3"
use customers encryption "key_1,key_2,key_3" copy structure to blankcust use blankcust encryption "key_1,key_2,key_3"
COPY TO
Used to copy a table.// By default, the key file is automatically copied for an encrypted // source table and the target table encrypted with the same key encrypt customers key "key_1,key_2,key_3" use customers encryption "key_1,key_2,key_3" copy to newcustomers use newcustomers encryption "key_1,key_2,key_3" // You can also create a copy with a different key encrypt customers key "key_1,key_2,key_3" use customers encryption "key_1,key_2,key_3" copy to newcustomers encrypt "newkey_1,newkey_2,newkey_3" use newcustomers encryption "newkey_1,newkey_2,newkey_3" // Or create a decrypted copy encrypt customers key "key_1,key_2,key_3" use customers encryption "key_1,key_2,key_3" copy to newcustomers decrypt use newcustomers // You can also create an encrypted copy of a non-encrypted source table use orders copy to encorders encrypt "newkey_1,newkey_2,newkey_3" use encorders encryption "newkey_1,newkey_2,newkey_3"
DIR
Used to display a directory listing of tables.// Encrypted tables are flagged as such with (DES3) > open database southwind > dir
Current database: southwind Tables # Records Last Update Size Dictionary Triggers Security categories.dbf 8 01/10/06 24576 None None None cisamdemo.dbf ---> CISAM/Bridge [cisamdemo] customers.dbf (DES3) 91 05/12/04 49600 None None None employees.dbf 9 05/12/04 25520 None None None example.dbf (DES3) 100 12/24/05 38080 Yes Yes None order_details.dbf 2155 05/12/04 296320 None None None orders.dbf 829 05/12/04 232704 None None None products.dbf 77 05/12/04 37112 None None None productsbyname.dbf 77 05/12/04 29104 None None None shippers.dbf (DES3) 3 05/12/04 20864 None None None suppliers.dbf 29 12/08/05 29992 Yes None None 0.765 MB in 11 files. 1.093 GB remaining on drive.
USE
Used to open a table.// The three part key must be specified to open an // encrypted table. All of the following are valid. // 1. Specifying a default encryption key before opening the table set encryption to "key_1,key_2,key_3" use customers // 2. Appending the key to the filename use customers<key_1,key_2,key_3> // 3. Using the ENCRYPTION clause, optionally specifying angled brackets use customers encryption "key_1,key_2,key_3" use customers encryption "<key_1,key_2,key_3>"
SQL INSERT
Used to add a row to a table via SQL.// The three part key can be specified using a
// default encryption key before opening the table
exec sql
OPEN DATABASE southwind;
exec sql
SET ENCRYPTION TO "key_1,key_2,key_3";
exec sql
INSERT INTO customers
(customerid, companyname)
VALUES
('RECIT','Recital Corporation');
// Or by appending the key to the filename
exec sql
OPEN DATABASE southwind;
exec sql
INSERT INTO customers<key_1,key_2,key_3>
(customerid, companyname)
VALUES
('RECIT','Recital Corporation');
SQL SELECT
Used to return data from a table via SQL.// The three part key can be specified using a // default encryption key before opening the table exec sql OPEN DATABASE southwind; exec sql SET ENCRYPTION TO "key_1,key_2,key_3"; exec sql SELECT * FROM customers; // Or by appending the key to the filename exec sql OPEN DATABASE southwind; exec sql SELECT * FROM customers<key_1,key_2,key_3>;
SQL UPDATE
Used to update data in a table via SQL.// The three part key can be specified using a // default encryption key before opening the table exec sql OPEN DATABASE southwind; exec sql SET ENCRYPTION TO "key_1,key_2,key_3"; exec sql UPDATE customers SET companyname='Recital Corporation Inc.' WHERE customerid='RECIT'; // Or by appending the key to the filename exec sql OPEN DATABASE southwind; exec sql UPDATE customers<key_1,key_2,key_3> SET companyname='Recital Corporation Inc.' WHERE customerid='RECIT';
Summary
Recital offers a range of ways to keep your data secure. These start with the Operating System read/write permissions, which can be further refined to the level of table I/O operations and then field access in the Dictionary based Security and Protection rules. The Dictionary also provides the means to protect the integrity of the data via data validation and to assist in correct data entry through the use of choicelists, help messages and picture clauses etc. A further role of the Dictionary is in the provision of Table Triggers, which can be used to enable a programmatic response to table operations to add in additional checks or audit trails. For the most sensitive data, DES3 encryption is the ultimate protection: encrypting the physical data on the disk and only permitting table access on the production of the three part encryption key.
Key features of the Recital database include:
- SQL-92 and a broad subset of ANSI SQL 99, as well as extensions
- Cross-platform support
- Stored procedures
- Triggers
- Cursors
- Updatable Views
- System Tables
- Query caching
- High-performance
- Single-User and Multi-User
- Multi-Process
- ACID Transactions
- Referential Integrity
- Cascading Updates and Deletes
- Multi-table Joins
- Row-level Locking
- BLOBs (Binary Large Objects)
- UDFs (User Defined Functions)
- OLTP (On-Line Transaction Processing)
- Drivers for ODBC, JDBC, and .NET
- Sub-SELECTs (i.e. nested SELECTs)
- Embedded database library
- Database timelines providing data undo functionality
- Fault tolerant clustering support
- Hot backup