In this article Barry Mavin explains step by step how to setup a Linux HA (High Availability) cluster for the running of Recital applications on Redhat/Centos 5.3 although the general configuration should work for other linux versions with a few minor changes.

This article discusses Recital database security: from operating system file permissions through file and field protection to DES3 encryption.

Overview

A company's data is extremely valuable and must be protected, both in operation and in physical file format. Recital products provide a range of ways to protect your data.

Operating System File Permissions

The most basic level of database security is provided by the operating system. Recital database tables and indexes are individual files with their own respective operating system file permissions. Read permission is required to open a table and write permission to update a table. If a user does not have read permission they are denied access. Without write permission, a table will be opened read-only.

Here the owner, root, and members of the recital group have write permission, so can update the example table unless additional protection applies. Other users can only open the example table read-only.

# ls -l example*
-rwxrwxr-x    1 root     recital       147 Nov 29 14:27 example.dbd
-rwxrwxr-x    1 root     recital     41580 Nov 29 14:27 example.dbf
-rwxrwxr-x    1 root     recital     13312 Nov 29 14:28 example.dbt
-rwxrwxr-x    1 root     recital     19456 Nov 29 14:28 example.dbx

Note: As in the example above, a table's associated files should have the same permissions as the table itself:

File Extension	File Type
.dbd	Dictionary
.dbf	Table
.dbt	Memo
.dbx	Index

Database Dictionary

Each Recital table may have a Database Dictionary. The Dictionary can be used both to protect the integrity of the data and to protect access to the data. This section covers Column Constraints, Triggers, Security and Protection.

Column Constraints: Data Integrity

The Dictionary attributes or constraints either prevent the entry of incorrect data, e.g. must_enter and validation or aid the entry of correct data, e.g. default, picture and choicelist. The Dictionary can be modified in the character mode CREATE/MODIFY STRUCTURE worksurface, via SQL statements, or in the Recital Enterprise Studio Database Administrator.

Click image to display full size

Fig 1: MODIFY STRUCTURE Worksurface: Dictionary.

The SQL Column Constraints are as follows:

Constraint	Description
AUTO_INCREMENT | AUTOINC	Used to auto increment the value of a column.
CALCULATED	Used to calculate the value of a column.
CHECK | SET CHECK	Used to validate a change to the value of a column.
DEFAULT	Used to set a default value for the specified column.
DESCRIPTION	Used set the column description for the specified column.
ERROR	Used to define an error message to be displayed when a validation check fails.
FOREIGN KEY	Used to define a column as a Foreign Key for a parent table.
NOCPTRANS	Used to prevent code page translation for character and memo fields.
NOT NULL | NULL	Used to disallow/allow NULL values.
PRIMARY KEY	Used to define a table’s Primary Key.
RANGE	Used to specify minimum and maximum values for a date or numerical column.
RECALCULATE	Used to force recalculation of calculated columns when a column’s value changes.
REFERENCES	Used to create a relationship to an index key of another table.
UNIQUE	Used to define the column as a candidate index for the table

These can be specified in CREATE TABLE or ALTER TABLE statements:

exec sql
  OPEN DATABASE southwind;
exec sql
  ALTER TABLE customers ADD COLUMN timeref char(8) CHECK validtime(timeref)
  ERROR "Not a valid time string";

Click image to display full size

Fig 2: Database Administrator: Column Constraints and Attributes.

TRIGGERS

Table Level Triggers are event-driven procedures called before an I/O operation. These can be used to introduce another layer of checks before a particular operation is permitted to take place or to simply set up logging of those operations.

The CREATE/MODIFY STRUCTURE worksurface <TRIGGERS> menu bar option allows you to specify table level triggers. You may edit a trigger procedure from within the <TRIGGERS> menu by placing the cursor next to the procedure name and pressing the [HELP] key. A text window pops up for editing. If the table triggers are stored in separate <.prg> files, rather than in a procedure library, procedures need not be predefined (SET PROCEDURE) before using the table.

Click image to display full size

Fig 3: MODIFY STRUCTURE Worksurface: Triggers.

 

The following triggers can be selected and associated with a specified procedure name in the <TRIGGERS> menu.

Trigger	Description
UPDATE	The specified procedure is called prior to an update operation on the table. If the procedure returns .F., then the UPDATE is canceled.
DELETE	The specified procedure is called prior to a delete operation on the table. If the procedure returns .F., then the DELETE is canceled.
APPEND	The specified procedure is called prior to an append operation on the table. If the procedure returns .F., then the APPEND is canceled.
OPEN	The specified procedure is called after an open operation on the table.
CLOSE	The specified procedure is called prior to a close operation on the table.
ROLLBACK	The specified procedure is called when a user presses the [ABANDON] key in a forms based operation.

The Recital Enterprise Studio Database Administrator also allows you to associate existing programs as Table Trigger Procedures.

Click image to display full size

Fig 4: Database Administrator: Triggers.

 

Programmatically, Trigger Procedures can also be associated with a table using SQL. The following table constraints may be applied in the SQL CREATE TABLE and ALTER TABLE statements:

Trigger	Description
ONUPDATE	The specified procedure is called prior to an update operation on the table. If the procedure returns .F., then the UPDATE is canceled. e.g. SQL> ALTER TABLE customer modify ONUPDATE "p_update";
ONDELETE	The specified procedure is called prior to a delete operation on the table. If the procedure returns .F., then the DELETE is canceled. e.g. SQL> ALTER TABLE customer modify ONDELETE "p_delete";
ONINSERT	The specified procedure is called prior to an insert operation on the table. If the procedure returns .F., then the INSERT is canceled. e.g. SQL> ALTER TABLE customer modify ONINSERT "p_insert";
ONOPEN	The specified procedure is called after an open operation on the table. e.g. SQL> ALTER TABLE customer modify ONOPEN "p_open";
ONCLOSE	The specified procedure is called prior to a close operation on the table. e.g. SQL> ALTER TABLE customer modify ONCLOSE "p_close";
ONROLLBACK	The specified procedure is called when a user presses the [ABANDON] key in a forms based operation. e.g. SQL> ALTER TABLE customer modify ONROLLBACK "p_rollback";

SECURITY

As mentioned above, all Recital files are subject to Operating System read and write permissions. These permissions can be further refined, while still using the Operating System user and group IDs, in the Security and Protection sections of the Dictionary. The Security section handles table based operations and the Protection section focuses on individual fields.

Security and Protection rules can be defined in the CREATE/MODIFY STRUCTURE worksurface of Recital Terminal Developer, via the SQL GRANT and REVOKE statements or in the Recital Enterprise Studio Database Administrator.

Click image to display full size

Fig 5: MODIFY STRUCTURE Worksurface: Security.

 

The Security section has table operations for which Access Control Strings can be specified. An Access Control String (ACS) is a range of valid user identification codes, and is used to restrict table operations to certain individuals or groups. Each user on the system is allocated a group number and a user number. The user identification code is the combination of group and user numbers. When constructing an Access Control String of linked user identification codes, wild card characters may be used.

Example ACS	Description
[1,2]	In group 1, user 2
[100,*]	In group 100, all users
[2-7,*]	In groups 2-7, all users
[*,100-200]	In all groups, users 100-200
[1,*]&[2-7,1-7]	In group 1, all users, in groups 2-7, users 1-7

Please note that the maximum ACS length is 254 characters. OpenVMS group and user numbers are stored and specified in octal. On other Operating Systems, group and user numbers are stored and specified in decimal.

Access Control Strings may be associated with the following operations:

Operation	Description
READONLY	Users specified in the ACS have read-only access to the table. All other users have update access.
UPDATE	Users specified in the ACS have update access to the table. All other users are restricted to read-only access.
APPEND	Users specified in the ACS can append records into the table. No other users can append.
DELETE	Users specified in the ACS can delete records from the table. No other users can delete.
COPY	Users specified in the ACS can copy records from the table. No other users can copy.
ADMIN	Users specified in the ACS can use the following commands: SET DICTIONARY TO MODIFY STRUCTURE PACK ZAP REINDEX All other users cannot, except the creator of the table, who is always granted ADMIN access.

The corresponding SQL privileges are:

Operation	Description
SELECT	Users specified in the ACS may name any column in a SELECT statement. All other users have update access.
UPDATE	Users specified in the ACS may name any column in an UPDATE statement. All other users are restricted to read-only access.
INSERT	Users specified in the ACS can INSERT rows into the table. No other users can INSERT.
DELETE	Users specified in the ACS can DELETE rows from the table. No other users can DELETE.
ALTER	Users specified in the ACS can use the ALTER TABLE statement on this table.
READONLY	Users specified in the ACS may read any column in a SELECT statement. All other users have update access.

// Grant insert privilege for the customer table
exec sql
  OPEN DATABASE southwind;
exec sql
  GRANT UPDATE (lastname, firstname)
  INSERT ON customers
  TO '[20,100]'; 
	
// Grant all privileges to all users
exec sql
  OPEN DATABASE southwind;
exec sql
  GRANT ALL 
  ON shippers TO PUBLIC;

PROTECTION

Security and Protection rules can be defined in the CREATE/MODIFY STRUCTURE worksurface of Recital Terminal Developer, via the SQL GRANT and REVOKE statements or in the Recital Enterprise Studio Database Administrator.

Click image to display full size

Fig 6: Database Administrator: Protection.

 

The format of the ACS is the same as in <SECURITY> above. The following protection can be defined:

Operation	Description
READONLY	Users specified in the ACS have read-only access to the field. All other users have update access.
UPDATE	Users specified in the ACS have update access to the field. All other users are restricted to read-only access.

Recital Terminal Developer also has 'HIDDEN' Protection:

Operation	Description
HIDDEN	Users specified in the ACS see the 'hiddenfield'character rather than the data in the field. All other users see the data.

Hidden fields can be accessed and viewed on a work surface, but the field contains the hiddenfield character, ‘?’. If the field is referenced in an expression, it will contain the following: blanks for character fields, ‘F’ for logical fields, 00/00/0000 for date fields and blank for memo fields.

The corresponding SQL privileges are:

Operation	Description
SELECT	Users specified in the ACS may name the column in a SELECT statement. All other users have update access.
UPDATE	Users specified in the ACS may name the column in an UPDATE statement. All other users are restricted to read-only access.
READONLY	Users specified in the ACS may read the column in a SELECT statement. All other users have update access.

// Grant update privilege for columns lastname and firstname from the customer table
exec sql
  OPEN DATABASE southwind;
exec sql
  GRANT UPDATE (lastname, firstname)
  customers TO '[20,100]';

Encryption

From Recital 8.5 onwards, Recital installations that have the additional DES3 license option have the ability to encrypt the data held in Recital database tables. Once a database table has been encrypted, the data cannot be accessed unless the correct three-part encryption key is specified, providing additional security for sensitive data.

ENCRYPT

The ENCRYPT Recital 4GL command is used to encrypt the data in the specified table or tables matching a skeleton. If the skeleton syntax is used, then all matching tables will be given the same encryption key. The encryption key is a three part comma-separated key and may optionally be enclosed in angled brackets. Each part of the key can be a maximum of 8 characters. The key is DES3 encrypted and stored in a .dkf file with the same basename as the table. After encryption, the three parts of the key must be specified correctly before the table can be accessed.

// Encrypt individual tables
encrypt customers key "key_1,key_2,key_3"
encrypt employees key "<key_1,key_2,key_3>"

// Encrypt all .dbf files in the directory
encrypt *.dbf key "key_1,key_2,key_3"

SET ENCRYPTION

If a database table is encrypted, the correct three-part encryption key must be specified before the table's data or structure can be accessed. The SET ENCRYPTION TO set command can be used to specify a default encryption key to be used whenever an encrypted table is accessed without the key being specified. The encryption key is a three part comma-separated key.

If the command to access the table includes the key, either by appending it to the table filename specification or using an explicit clause, this will take precedence over the key defined by SET ENCRYPTION TO.

Issuing SET ENCRYPTION TO without a key causes any previous setting to be cleared. The key must then be specified for each individual encrypted table.

The default key defined by SET ENCRYPTION is only active when SET ENCRYPTION is ON. SET ENCRYPTION OFF can be used to temporarily disable the default key. The SET ENCRYPTION ON | OFF setting does not change the default key itself. SET ENCRYPTION is ON by default.

// Encrypt individual tables
encrypt customers key "key_1,key_2,key_3"
encrypt shippers key "key_2,key_3,key_4"
// Specify a default encryption key
set encryption to "key_1,key_2,key_3"
// Open customers table using the default encryption key
use customers
// Specify shippers table's encryption key
use shippers<key_2,key_3,key_4>
// Disable the default encryption key
set encryption to
// Specify the individual encryption keys
use customers encryption "key_1,key_2,key_3"
use shippers<key_2,key_3,key_4>

DECRYPT

The DECRYPT command is used to decrypt the data in the specified table or tables matching a skeleton. The specified key must contain the three part comma-separated key used to previously encrypt the table and may optionally be enclosed in angled brackets. The skeleton syntax can only be used if all tables matching the skeletonhave the same key.

The DECRYPT command decrypts the data and removes the table’s .dkf file. After decryption, the key need no longer be specified to gain access to the table.

// Decrypt individual tables
decrypt customers key "key_1,key_2,key_3"
decrypt employees key "<key_1,key_2,key_3>"

// Decrypt all .dbf files in the directory
decrypt *.dbf key "key_1,key_2,key_3"

All of the following commands are affected when a table is encrypted:

• APPEND FROM
• COPY FILE
• COPY STRUCTURE
• COPY TO
• DIR
• USE
• SQL INSERT
• SQL SELECT
• SQL UPDATE

APPEND FROM

Used to append records to the active table from another table.

// The key must be specified for an encrypted source table
use mycustomers
append from customers encryption "key_1,key_2,key_3";
for country = "UK"

COPY FILE

Used to copy a file.

// The key file must also be copied for an encrypted source table
// as the target table will be encrypted
encrypt customers key "key_1,key_2,key_3"
copy file customers.dbf to newcustomers.dbf
copy file customers.dkf to newcustomers.dkf
use newcustomers encryption "key_1,key_2,key_3"

COPY STRUCTURE

Used to copy a table's structure to a new table.

// The key file is automatically copied for an encrypted source table
// and the target table encrypted
encrypt customers key "key_1,key_2,key_3"
use customers encryption "key_1,key_2,key_3"
copy structure to blankcust
use blankcust encryption "key_1,key_2,key_3"

COPY TO

Used to copy a table.

// By default, the key file is automatically copied for an encrypted
// source table and the target table encrypted with the same key
encrypt customers key "key_1,key_2,key_3"
use customers encryption "key_1,key_2,key_3"
copy to newcustomers
use newcustomers encryption "key_1,key_2,key_3"

// You can also create a copy with a different key
encrypt customers key "key_1,key_2,key_3"
use customers encryption "key_1,key_2,key_3"
copy to newcustomers encrypt "newkey_1,newkey_2,newkey_3"
use newcustomers encryption "newkey_1,newkey_2,newkey_3"

// Or create a decrypted copy
encrypt customers key "key_1,key_2,key_3"
use customers encryption "key_1,key_2,key_3"
copy to newcustomers decrypt
use newcustomers

// You can also create an encrypted copy of a non-encrypted source table
use orders
copy to encorders encrypt "newkey_1,newkey_2,newkey_3"
use encorders encryption "newkey_1,newkey_2,newkey_3"

DIR

Used to display a directory listing of tables.

// Encrypted tables are flagged as such with (DES3)
> open database southwind
> dir

Current database: southwind
Tables				# Records		Last Update	Size		Dictionary	Triggers	Security
categories.dbf			8			01/10/06		24576	None		None		None
cisamdemo.dbf       ---> CISAM/Bridge        [cisamdemo]
customers.dbf (DES3)		91			05/12/04		49600	None		None		None
employees.dbf			9			05/12/04		25520	None		None		None
example.dbf   (DES3)		100			12/24/05		38080	Yes		Yes		None
order_details.dbf			2155			05/12/04		296320	None		None		None
orders.dbf				829			05/12/04		232704	None		None		None
products.dbf			77			05/12/04		37112	None		None		None
productsbyname.dbf		77			05/12/04		29104	None		None		None
shippers.dbf  (DES3)		3			05/12/04		20864	None		None		None
suppliers.dbf			29			12/08/05		29992	Yes		None		None

   0.765 MB in 11 files.
   1.093 GB remaining on drive.

USE

Used to open a table.

// The three part key must be specified to open an
// encrypted table.  All of the following are valid.
// 1. Specifying a default encryption key before opening the table
set encryption to "key_1,key_2,key_3"
use customers
// 2. Appending the key to the filename
use customers<key_1,key_2,key_3>
// 3. Using the ENCRYPTION clause, optionally specifying angled brackets
use customers encryption "key_1,key_2,key_3"
use customers encryption "<key_1,key_2,key_3>"

SQL INSERT

Used to add a row to a table via SQL.

// The three part key can be specified using a
// default encryption key before opening the table
exec sql
  OPEN DATABASE southwind;
exec sql
  SET ENCRYPTION TO "key_1,key_2,key_3";
exec sql
  INSERT INTO customers
  (customerid, companyname)
  VALUES
  ('RECIT','Recital Corporation');
// Or by appending the key to the filename
exec sql
  OPEN DATABASE southwind;
exec sql
  INSERT INTO customers<key_1,key_2,key_3>
  (customerid, companyname)
  VALUES
  ('RECIT','Recital Corporation');

SQL SELECT

Used to return data from a table via SQL.

// The three part key can be specified using a
// default encryption key before opening the table
exec sql
  OPEN DATABASE southwind;
exec sql
  SET ENCRYPTION TO "key_1,key_2,key_3";
exec sql
  SELECT * FROM customers;
// Or by appending the key to the filename
exec sql
  OPEN DATABASE southwind;
exec sql
  SELECT * FROM customers<key_1,key_2,key_3>;

SQL UPDATE

Used to update data in a table via SQL.

// The three part key can be specified using a
// default encryption key before opening the table
exec sql
  OPEN DATABASE southwind;
exec sql
  SET ENCRYPTION TO "key_1,key_2,key_3";
exec sql
  UPDATE customers
  SET companyname='Recital Corporation Inc.'
  WHERE customerid='RECIT';
// Or by appending the key to the filename
exec sql
  OPEN DATABASE southwind;
exec sql
  UPDATE customers<key_1,key_2,key_3>
  SET companyname='Recital Corporation Inc.'
  WHERE customerid='RECIT';

Summary

Recital offers a range of ways to keep your data secure. These start with the Operating System read/write permissions, which can be further refined to the level of table I/O operations and then field access in the Dictionary based Security and Protection rules. The Dictionary also provides the means to protect the integrity of the data via data validation and to assist in correct data entry through the use of choicelists, help messages and picture clauses etc. A further role of the Dictionary is in the provision of Table Triggers, which can be used to enable a programmatic response to table operations to add in additional checks or audit trails. For the most sensitive data, DES3 encryption is the ultimate protection: encrypting the physical data on the disk and only permitting table access on the production of the three part encryption key.

Subclipse is an Eclipse Team Provider plug-in providing support for Subversion within the Eclipse IDE. This plugin is required in order to use the recital eclipse workspace.

This article discusses the features in Recital that allow data to be imported and exported between platforms in Microsoft® ADO XML Format.

Overview

Extensible Markup Language, XML, is widely regarded as a lingua franca for the interchange of data. XML's text-based, platform-independent format and its integration of data and the schema to define and describe that data, make it the ideal import/export medium. Recital software provides the functionality to output the data from Recital - and other supported table formats such as FoxPro and FoxBASE - into XML file format and to import XML data into those tables' formats. Such import/export operations provide the means to exchange data with third-party applications and can also facilitate the transfer of data between Recital installations on binary-incompatible platforms.

The features examined in this article are available in Recital Terminal Developer and in the Recital Mirage and Recital Database Servers on all Recital supported platforms. Both the Recital/4GL and Recital/SQL provide XML import and export capabilities. The XML files discussed are in Microsoft® ADO XML format.

Microsoft® ActiveX® Data Objects XML Format

The ADO XML format is primarily designed for ADO Recordset persistence and ADO XML files created by Recital can be used in this way and loaded directly into ADO Recordsets. The format can, though also be used for more generic data transfer. An ADO XML file is self-contained, consisting of two sections: a schema section followed by a data section. The schema conforms to the W3C XML-Data specification and defines the data structure.
For additional information on the Microsoft® ActiveX® Data Objects XML Format, please see Appendix 1.

NOTE: The Recital XMLFORMAT setting should always be in its default setting of ADO for ADO XML Format operations.

set xmlformat to ADO

SQL

Recital/SQL offers the ability to export data into XML files using the SELECT and FETCH statements and import from XML using the CREATE TABLE and INSERT statements.

SQL: Exporting

The SELECT...SAVE AS XML statement allows the complete result set from a SELECT statement to be saved as an XML file. This could be a complete table:

open database southwind
  SELECT * from orders SAVE AS XML orders.xml

or a more complex multi-table query:

open database southwind
SELECT orders.orderid, orders.customerid,;
    employees.employeeid, employees.lastname, employees.firstname,;
    orders.orderdate, orders.freight, orders.requireddate,;
    orders.shippeddate, orders.shipvia, orders.shipname,;
    orders.shipaddress, orders.shipcity,;
    orders.shipregion, orders.shippostalcode, orders.shipcountry,;
    customers.companyname, customers.address, customers.city,;
    customers.region, customers.postalcode, customers.country; 
    FROM orders INNER JOIN customers;
    ON customers.customerid = orders.customerid,;
    orders INNER JOIN employees;
    ON orders.employeeid = employees.employeeid;
    SAVE AS XML orderinfo

The resulting XML file can then be further processed within the same or a different Recital environment or transferred to a third party product.

<x-ml xmlns:z="#RowsetSchema" xmlns:rs="urn:schemas-microsoft-com:rowset"
 xmlns:dt="uuid:C2F41010-65B3-11d1-A29F-00AA00C14882" xmlns:s="uuid:BDC6E3F0-6DA3-11d1-A2A3-
00AA00C14882">
 <s:schema id="RowsetSchema">
  <s:elementtype rs:updatable="true" content="eltOnly" name="row">
   <s:attributetype rs:basecolumn="orderid" rs:basetable="orders.dbf" rs:write="true"
 rs:nullable="true" rs:number="1" name="orderid">
   <s:datatype rs:fixedlength="true" rs:precision="14" rs:scale="0" dt:maxlength="10"
 rs:dbtype="numeric" dt:type="number">
   </s:datatype></s:attributetype>
   <s:attributetype rs:basecolumn="customerid" rs:basetable="orders.dbf" rs:write="true"
 rs:nullable="true" rs:number="2" name="customerid">
   <s:datatype rs:fixedlength="true" dt:maxlength="5" rs:dbtype="str" dt:type="string">
   </s:datatype></s:attributetype>
   <s:attributetype rs:basecolumn="employeeid" rs:basetable="orders.dbf" rs:write="true"
 rs:nullable="false" rs:number="3" name="employeeid">
   <s:datatype rs:fixedlength="true" rs:precision="20" rs:scale="0" dt:maxlength="10"
 rs:dbtype="numeric" dt:type="number">
   </s:datatype></s:attributetype>
   <s:attributetype rs:basecolumn="lastname" rs:basetable="orders.dbf" rs:write="true"
 rs:nullable="false" rs:number="4" name="lastname">
   <s:datatype rs:fixedlength="true" dt:maxlength="20" rs:dbtype="str" dt:type="string">
   </s:datatype></s:attributetype>
   <s:attributetype rs:basecolumn="firstname" rs:basetable="orders.dbf" rs:write="true"
 rs:nullable="false" rs:number="5" name="firstname">
   <s:datatype rs:fixedlength="true" dt:maxlength="10" rs:dbtype="str" dt:type="string">
   </s:datatype></s:attributetype>
   <s:attributetype rs:basecolumn="orderdate" rs:basetable="orders.dbf" rs:write="true"
 rs:nullable="true" rs:number="6" name="orderdate">
   <s:datatype rs:fixedlength="true" dt:maxlength="10" rs:dbtype="Date" dt:type="Date">
   </s:datatype></s:attributetype>
   <s:attributetype name="freight" ...

Click image to display full size

Fig 1: Microsoft® Office Excel 2003: orderinfo.xml.

For data accessed through a Recital Database Gateway, such as Oracle, MySQL or PostgreSQL, the FETCH command can be used to save a cursor results set into an XML file:

// Connect to MySQL Database 'mydata' via Recital Database Gateway
nStatHand=SQLSTRINGCONNECT("mys@mysql1:user1/pass1-mydata",.T.)
if nStatHand < 1
  dialog box [Could not connect]
else
  DECLARE cursor1 CURSOR FOR;
      SELECT account_no, last_name, first_name FROM example
  OPEN cursor1
  FETCH cursor1 INTO XML exa1.xml
  SQLDISCONNECT(nStatHand)
endif

SQL: Importing

The CREATE TABLE statement allows a new table to be created based on the structure defined in an XML file. The data from the XML file can optionally be loaded into this new table if the LOAD keyword is included. For example, a new 'orderinfo' table can be created and populated with data from the orderinfo.xml file created by the SELECT...SAVE AS XML statement shown earlier:

open database southwind
SELECT orders.orderid, orders.customerid,;
    employees.employeeid, employees.lastname, employees.firstname,;
    orders.orderdate, orders.freight, orders.requireddate,;
    orders.shippeddate, orders.shipvia, orders.shipname,;
    orders.shipaddress, orders.shipcity,;
    orders.shipregion, orders.shippostalcode, orders.shipcountry,;
    customers.companyname, customers.address, customers.city,;
    customers.region, customers.postalcode, customers.country; 
    FROM orders INNER JOIN customers;
    ON customers.customerid = orders.customerid,;
    orders INNER JOIN employees;
    ON orders.employeeid = employees.employeeid;
    SAVE AS XML orderinfo

CREATE TABLE orderinfo FROM XML orderinfo LOAD

The INSERT statement can be used to load data when the table structure already exists. Taking our earlier orderinfo.xml file again, the data can be loaded using INSERT:

open database southwind;
SELECT orders.orderid, orders.customerid,;
    employees.employeeid, employees.lastname, employees.firstname,;
    orders.orderdate, orders.freight, orders.requireddate,;
    orders.shippeddate, orders.shipvia, orders.shipname,;
    orders.shipaddress, orders.shipcity,;;
    orders.shipregion, orders.shippostalcode, orders.shipcountry,;
    customers.companyname, customers.address, customers.city,;
    customers.region, customers.postalcode, customers.country; 
    FROM orders INNER JOIN customers;
    ON customers.customerid = orders.customerid,;
    orders INNER JOIN employees;
    ON orders.employeeid = employees.employeeid;
    SAVE AS XML orderinfo
CREATE TABLE orderinfo FROM XML orderinfo

INSERT INTO orderinfo FROM XML orderinfo

The examples above show the export and import in a single piece of code. To transfer data between binary-incompatible platforms, the export phase using SELECT...SAVE AS XML would be carried out on the source platform, the resulting XML file would be transferred to the target platform, then the import phase using CREATE TABLE...LOAD or CREATE TABLE + INSERT would be run on the target platform.

Recital/4GL

The Recital/4GL offers the ability to export data into XML files using the COPY TO ... TYPE XML command and import from XML using the XMLFIRST() and XMLNEXT() functions.

Recital/4GL: Exporting

The COPY TO command can be used to export data from Recital and other natively supported tables out to a wide range of formats. This includes exporting to an XML file. The '.xml' file extension is added automatically. The COPY TO command can be used to export an entire table:

open database southwind
use orders
copy to orders type xml

or, using the FIELDS clause and the FOR or WHILE clauses, restrict the field list and export only those records which match a particular condition:

open database southwind
use orders
copy to orders type xml fields orderid for year(orderdate) = 1996

Only the orderid field from those records which match the condition is exported:

<x-ml xmlns:z="#RowsetSchema" xmlns:rs="urn:schemas-microsoft-com:rowset"
 xmlns:dt="uuid:C2F41010-65B3-11d1-A29F-00AA00C14882" xmlns:s="uuid:BDC6E3F0-6DA3-11d1-A2A3-
00AA00C14882">
 <s:schema id="RowsetSchema">
  <s:elementtype rs:updatable="true" content="eltOnly" name="row">
   <s:attributetype rs:basecolumn="ORDERID" rs:basetable="ORDERS" rs:write="true" rs:nullable="true"
 rs:number="1" name="ORDERID">
   <s:datatype rs:fixedlength="true" rs:precision="10" rs:scale="0" dt:maxlength="10"
 rs:dbtype="numeric" dt:type="number">
   </s:datatype></s:attributetype>
  </s:elementtype>
 </s:schema>
 <rs:data>
 <z:row orderid="10248">
 <z:row orderid="10249">
 <z:row orderid="10250">
 <z:row ...

Recital/4GL: Importing

Data from an XML file can be extracted one record at a time using the XMLFIRST() and XMLNEXT() functions. XMLFIRST() reads the first record from an XML file and loads information from the file into a series of memory variables and arrays. The record data is loaded into a one-dimensional array which is created automatically. Each element in the array contains the data for its corresponding field in string format. The field names are loaded into another automatically-created array. The XMLNEXT() function works in a similar way to deal with all the subsequent records in the XML file. The XMLCOUNT() function can be used, as in the example below, to determine how many data records the XML file has.

The Recital/4GL includes a vast range of functions for manipulation and conversion of arrays and their individual elements. In the example program below, the XMLFIRST() and XMLNEXT() functions are used to sequentially extract each record from an XML file, whose name is passed to the program as a parameter. Once loaded into an array, the data is converted to the correct Recital data type then appended into a table. The table name is also passed as a parameter.

procedure replaceit
  append blank
  for i = 1 to numfields
    if type(field(i)) = "N"
      replace &(field(i)) with val(data[&i])
    elseif type(field(i)) = "D"
      replace &(field(i)) with stod(data[&i])
    elseif type(field(i)) = "T"
      replace &(field(i)) with ctot(data[&i])
    elseif type(field(i)) = "L"
      replace &(field(i)) with iif(data[&i]="T",.T.,.F.)
    elseif type(field(i)) = "Y"
      replace &(field(i)) with val(data[&i])
    else
      replace &(field(i)) with data[&i]
    endif
  next
return
 
procedure starthere                    
  parameters cTable, cFile
  numfields=xmlfirst(cFile,targ,trans,where,fldnames,data)
  if numfields < 1
    dialog box [No records in XML file]
  else
    use &cTable
    replaceit()
  endif
  numrecs = xmlcount(cFile)
  if numrecs > 1
    numleft = numrecs -1
    for i = 1 to numleft
      xmlnext(trans,where,fldnames,data)
      replaceit()
    next
  endif
return

Alternative Import/Export Methods

Other features exist in Recital to facilitate the import and export of data:

RDDs

The RDDs, Replaceable Database Drivers, are available on Windows, Linux and all supported 32-bit UNIX platforms. They allow for the use and creation of database tables and indexes in FoxPro, dBase and Clipper formats. The file format is the same across all the platforms that support the RDDs, allowing the tables and indexes to be transferred as required. The formats are also supported by a wide range of third-party products as well as their originating database systems. For more information on the RDDs, please see the online documentation on Xbase migration and the SET FILETYPE command.

BUILD/INSTALL

These are Recital/4GL commands for the export (BUILD) and import (INSTALL) of Recital tables and their associated memo, dictionary and multiple index files in ASCII format to allow them to be transferred across binary incompatible platforms. For more information, please see the online documentation on Recital/4GL commands.

COPY Commands

The COPY TO, COPY STRUCTURE, COPY STRUCTURE EXTENDED and CREATE FROM commands can all be used to enable data to be transferred between different formats and different platforms. For more information, please see the online documentation on Recital/4GL commands.

Appendix 1: Microsoft® ActiveX® Data Objects XML Format

For detailed information on the Microsoft® ActiveX® Data Objects XML Format, please consult the following Microsoft documentation:

Link

XML Persistence Format

Namespaces

Schema Section

Data Section