This article discusses the features in Recital that allow data to be imported and exported between platforms in Microsoft® ADO XML Format.

Overview

Extensible Markup Language, XML, is widely regarded as a lingua franca for the interchange of data. XML's text-based, platform-independent format and its integration of data and the schema to define and describe that data, make it the ideal import/export medium. Recital software provides the functionality to output the data from Recital - and other supported table formats such as FoxPro and FoxBASE - into XML file format and to import XML data into those tables' formats. Such import/export operations provide the means to exchange data with third-party applications and can also facilitate the transfer of data between Recital installations on binary-incompatible platforms.

The features examined in this article are available in Recital Terminal Developer and in the Recital Mirage and Recital Database Servers on all Recital supported platforms. Both the Recital/4GL and Recital/SQL provide XML import and export capabilities. The XML files discussed are in Microsoft® ADO XML format.

Microsoft® ActiveX® Data Objects XML Format

The ADO XML format is primarily designed for ADO Recordset persistence and ADO XML files created by Recital can be used in this way and loaded directly into ADO Recordsets. The format can, though also be used for more generic data transfer. An ADO XML file is self-contained, consisting of two sections: a schema section followed by a data section. The schema conforms to the W3C XML-Data specification and defines the data structure.
For additional information on the Microsoft® ActiveX® Data Objects XML Format, please see Appendix 1.

NOTE: The Recital XMLFORMAT setting should always be in its default setting of ADO for ADO XML Format operations.

set xmlformat to ADO

SQL

Recital/SQL offers the ability to export data into XML files using the SELECT and FETCH statements and import from XML using the CREATE TABLE and INSERT statements.

SQL: Exporting

The SELECT...SAVE AS XML statement allows the complete result set from a SELECT statement to be saved as an XML file. This could be a complete table:

open database southwind
  SELECT * from orders SAVE AS XML orders.xml

or a more complex multi-table query:

open database southwind
SELECT orders.orderid, orders.customerid,;
    employees.employeeid, employees.lastname, employees.firstname,;
    orders.orderdate, orders.freight, orders.requireddate,;
    orders.shippeddate, orders.shipvia, orders.shipname,;
    orders.shipaddress, orders.shipcity,;
    orders.shipregion, orders.shippostalcode, orders.shipcountry,;
    customers.companyname, customers.address, customers.city,;
    customers.region, customers.postalcode, customers.country; 
    FROM orders INNER JOIN customers;
    ON customers.customerid = orders.customerid,;
    orders INNER JOIN employees;
    ON orders.employeeid = employees.employeeid;
    SAVE AS XML orderinfo

The resulting XML file can then be further processed within the same or a different Recital environment or transferred to a third party product.

<x-ml xmlns:z="#RowsetSchema" xmlns:rs="urn:schemas-microsoft-com:rowset"
 xmlns:dt="uuid:C2F41010-65B3-11d1-A29F-00AA00C14882" xmlns:s="uuid:BDC6E3F0-6DA3-11d1-A2A3-
00AA00C14882">
 <s:schema id="RowsetSchema">
  <s:elementtype rs:updatable="true" content="eltOnly" name="row">
   <s:attributetype rs:basecolumn="orderid" rs:basetable="orders.dbf" rs:write="true"
 rs:nullable="true" rs:number="1" name="orderid">
   <s:datatype rs:fixedlength="true" rs:precision="14" rs:scale="0" dt:maxlength="10"
 rs:dbtype="numeric" dt:type="number">
   </s:datatype></s:attributetype>
   <s:attributetype rs:basecolumn="customerid" rs:basetable="orders.dbf" rs:write="true"
 rs:nullable="true" rs:number="2" name="customerid">
   <s:datatype rs:fixedlength="true" dt:maxlength="5" rs:dbtype="str" dt:type="string">
   </s:datatype></s:attributetype>
   <s:attributetype rs:basecolumn="employeeid" rs:basetable="orders.dbf" rs:write="true"
 rs:nullable="false" rs:number="3" name="employeeid">
   <s:datatype rs:fixedlength="true" rs:precision="20" rs:scale="0" dt:maxlength="10"
 rs:dbtype="numeric" dt:type="number">
   </s:datatype></s:attributetype>
   <s:attributetype rs:basecolumn="lastname" rs:basetable="orders.dbf" rs:write="true"
 rs:nullable="false" rs:number="4" name="lastname">
   <s:datatype rs:fixedlength="true" dt:maxlength="20" rs:dbtype="str" dt:type="string">
   </s:datatype></s:attributetype>
   <s:attributetype rs:basecolumn="firstname" rs:basetable="orders.dbf" rs:write="true"
 rs:nullable="false" rs:number="5" name="firstname">
   <s:datatype rs:fixedlength="true" dt:maxlength="10" rs:dbtype="str" dt:type="string">
   </s:datatype></s:attributetype>
   <s:attributetype rs:basecolumn="orderdate" rs:basetable="orders.dbf" rs:write="true"
 rs:nullable="true" rs:number="6" name="orderdate">
   <s:datatype rs:fixedlength="true" dt:maxlength="10" rs:dbtype="Date" dt:type="Date">
   </s:datatype></s:attributetype>
   <s:attributetype name="freight" ...

Click image to display full size

Fig 1: Microsoft® Office Excel 2003: orderinfo.xml.

For data accessed through a Recital Database Gateway, such as Oracle, MySQL or PostgreSQL, the FETCH command can be used to save a cursor results set into an XML file:

// Connect to MySQL Database 'mydata' via Recital Database Gateway
nStatHand=SQLSTRINGCONNECT("mys@mysql1:user1/pass1-mydata",.T.)
if nStatHand < 1
  dialog box [Could not connect]
else
  DECLARE cursor1 CURSOR FOR;
      SELECT account_no, last_name, first_name FROM example
  OPEN cursor1
  FETCH cursor1 INTO XML exa1.xml
  SQLDISCONNECT(nStatHand)
endif

SQL: Importing

The CREATE TABLE statement allows a new table to be created based on the structure defined in an XML file. The data from the XML file can optionally be loaded into this new table if the LOAD keyword is included. For example, a new 'orderinfo' table can be created and populated with data from the orderinfo.xml file created by the SELECT...SAVE AS XML statement shown earlier:

open database southwind
SELECT orders.orderid, orders.customerid,;
    employees.employeeid, employees.lastname, employees.firstname,;
    orders.orderdate, orders.freight, orders.requireddate,;
    orders.shippeddate, orders.shipvia, orders.shipname,;
    orders.shipaddress, orders.shipcity,;
    orders.shipregion, orders.shippostalcode, orders.shipcountry,;
    customers.companyname, customers.address, customers.city,;
    customers.region, customers.postalcode, customers.country; 
    FROM orders INNER JOIN customers;
    ON customers.customerid = orders.customerid,;
    orders INNER JOIN employees;
    ON orders.employeeid = employees.employeeid;
    SAVE AS XML orderinfo

CREATE TABLE orderinfo FROM XML orderinfo LOAD

The INSERT statement can be used to load data when the table structure already exists. Taking our earlier orderinfo.xml file again, the data can be loaded using INSERT:

open database southwind;
SELECT orders.orderid, orders.customerid,;
    employees.employeeid, employees.lastname, employees.firstname,;
    orders.orderdate, orders.freight, orders.requireddate,;
    orders.shippeddate, orders.shipvia, orders.shipname,;
    orders.shipaddress, orders.shipcity,;;
    orders.shipregion, orders.shippostalcode, orders.shipcountry,;
    customers.companyname, customers.address, customers.city,;
    customers.region, customers.postalcode, customers.country; 
    FROM orders INNER JOIN customers;
    ON customers.customerid = orders.customerid,;
    orders INNER JOIN employees;
    ON orders.employeeid = employees.employeeid;
    SAVE AS XML orderinfo
CREATE TABLE orderinfo FROM XML orderinfo

INSERT INTO orderinfo FROM XML orderinfo

The examples above show the export and import in a single piece of code. To transfer data between binary-incompatible platforms, the export phase using SELECT...SAVE AS XML would be carried out on the source platform, the resulting XML file would be transferred to the target platform, then the import phase using CREATE TABLE...LOAD or CREATE TABLE + INSERT would be run on the target platform.

Recital/4GL

The Recital/4GL offers the ability to export data into XML files using the COPY TO ... TYPE XML command and import from XML using the XMLFIRST() and XMLNEXT() functions.

Recital/4GL: Exporting

The COPY TO command can be used to export data from Recital and other natively supported tables out to a wide range of formats. This includes exporting to an XML file. The '.xml' file extension is added automatically. The COPY TO command can be used to export an entire table:

open database southwind
use orders
copy to orders type xml

or, using the FIELDS clause and the FOR or WHILE clauses, restrict the field list and export only those records which match a particular condition:

open database southwind
use orders
copy to orders type xml fields orderid for year(orderdate) = 1996

Only the orderid field from those records which match the condition is exported:

<x-ml xmlns:z="#RowsetSchema" xmlns:rs="urn:schemas-microsoft-com:rowset"
 xmlns:dt="uuid:C2F41010-65B3-11d1-A29F-00AA00C14882" xmlns:s="uuid:BDC6E3F0-6DA3-11d1-A2A3-
00AA00C14882">
 <s:schema id="RowsetSchema">
  <s:elementtype rs:updatable="true" content="eltOnly" name="row">
   <s:attributetype rs:basecolumn="ORDERID" rs:basetable="ORDERS" rs:write="true" rs:nullable="true"
 rs:number="1" name="ORDERID">
   <s:datatype rs:fixedlength="true" rs:precision="10" rs:scale="0" dt:maxlength="10"
 rs:dbtype="numeric" dt:type="number">
   </s:datatype></s:attributetype>
  </s:elementtype>
 </s:schema>
 <rs:data>
 <z:row orderid="10248">
 <z:row orderid="10249">
 <z:row orderid="10250">
 <z:row ...

Recital/4GL: Importing

Data from an XML file can be extracted one record at a time using the XMLFIRST() and XMLNEXT() functions. XMLFIRST() reads the first record from an XML file and loads information from the file into a series of memory variables and arrays. The record data is loaded into a one-dimensional array which is created automatically. Each element in the array contains the data for its corresponding field in string format. The field names are loaded into another automatically-created array. The XMLNEXT() function works in a similar way to deal with all the subsequent records in the XML file. The XMLCOUNT() function can be used, as in the example below, to determine how many data records the XML file has.

The Recital/4GL includes a vast range of functions for manipulation and conversion of arrays and their individual elements. In the example program below, the XMLFIRST() and XMLNEXT() functions are used to sequentially extract each record from an XML file, whose name is passed to the program as a parameter. Once loaded into an array, the data is converted to the correct Recital data type then appended into a table. The table name is also passed as a parameter.

procedure replaceit
  append blank
  for i = 1 to numfields
    if type(field(i)) = "N"
      replace &(field(i)) with val(data[&i])
    elseif type(field(i)) = "D"
      replace &(field(i)) with stod(data[&i])
    elseif type(field(i)) = "T"
      replace &(field(i)) with ctot(data[&i])
    elseif type(field(i)) = "L"
      replace &(field(i)) with iif(data[&i]="T",.T.,.F.)
    elseif type(field(i)) = "Y"
      replace &(field(i)) with val(data[&i])
    else
      replace &(field(i)) with data[&i]
    endif
  next
return
 
procedure starthere                    
  parameters cTable, cFile
  numfields=xmlfirst(cFile,targ,trans,where,fldnames,data)
  if numfields < 1
    dialog box [No records in XML file]
  else
    use &cTable
    replaceit()
  endif
  numrecs = xmlcount(cFile)
  if numrecs > 1
    numleft = numrecs -1
    for i = 1 to numleft
      xmlnext(trans,where,fldnames,data)
      replaceit()
    next
  endif
return

Alternative Import/Export Methods

Other features exist in Recital to facilitate the import and export of data:

RDDs

The RDDs, Replaceable Database Drivers, are available on Windows, Linux and all supported 32-bit UNIX platforms. They allow for the use and creation of database tables and indexes in FoxPro, dBase and Clipper formats. The file format is the same across all the platforms that support the RDDs, allowing the tables and indexes to be transferred as required. The formats are also supported by a wide range of third-party products as well as their originating database systems. For more information on the RDDs, please see the online documentation on Xbase migration and the SET FILETYPE command.

BUILD/INSTALL

These are Recital/4GL commands for the export (BUILD) and import (INSTALL) of Recital tables and their associated memo, dictionary and multiple index files in ASCII format to allow them to be transferred across binary incompatible platforms. For more information, please see the online documentation on Recital/4GL commands.

COPY Commands

The COPY TO, COPY STRUCTURE, COPY STRUCTURE EXTENDED and CREATE FROM commands can all be used to enable data to be transferred between different formats and different platforms. For more information, please see the online documentation on Recital/4GL commands.

Appendix 1: Microsoft® ActiveX® Data Objects XML Format

For detailed information on the Microsoft® ActiveX® Data Objects XML Format, please consult the following Microsoft documentation:

If when your attempt to create device meta-data fails this is drbd preventing you from corrupting a file system present on the target partition.

$ drbdadm create-md drbd0

v08 Magic number not found
md_offset 30005817344
al_offset 30005784576
bm_offset 30004867072

Found ext2 filesystem which uses 190804004 kB
current configuration leaves usable 29301628 kB

Device size would be truncated, which
would corrupt data and result in
'access beyond end of device' errors.
You need to either
 * use external meta data (recommended)
 * shrink that filesystem first
 * zero out the device (destroy the filesystem)
Operation refused.

Command 'drbdmeta /dev/drbd0 v08 /dev/sda4 internal create-md' terminated with exit code 40
drbdadm aborting

Once you have confirmed that the file system present on the target partition is no longer required at the prompt type the following:

Replace /dev/sdaX with the block device you are targeting.

dd if=/dev/zero of=/dev/sdaX bs=1M count=128

Once this has completed the drbdadm create-md drbd0 command will complete with a "success."

$ drbdadm create-md drbd0
v08 Magic number not found
v07 Magic number not found
v07 Magic number not found
v08 Magic number not found
Writing meta data...
initialising activity log
NOT initialized bitmap
New drbd meta data block successfully created.
success
$

This is a good primer for getting familiar with using Infiniband with Redhat/Centos Linux.
http://people.redhat.com/dledford/infiniband_get_started.html

Getting Started with InfiniBand

The first step to using a new infiniband based network is to get the right packages installed. These are the infiniband related packages we ship and what they are there for (Note, the Fedora packages have not all been built or pushed to the repos yet, so their mention here is as a "Coming soon" variety, not an already done variety):

If you have software packages which you wish to share with others or simply between your own personal machines, a neat and easy solution is to create your own YUM repository and provide your .repo file for download.

YUM is by far the easiest method of installing software on Red hat, Centos and Fedora. Not only does it mean you don't need to trawl the web looking for somewhere to download the packages, YUM does a great job of satisfying any package dependencies. As long as the required packages are available in the enabled repositories on your system, YUM will go out and get everything you need.

To create your own YUM repository, you will need to install the yum-utils and createrepo packages:

yum install yum-utils createrepo

yum-utils contains the tools you will need to manage your soon to be created repository, and createrepo is used to create the xml based rpm metadata you will require for your repository.

Once you have installed these required tools, create a directory in your chosen web server's document root e.g:

mkdir -p /var/www/html/repo/recital/updates

Copy the rpm's you wish to host into this newly created directory.

The next step is to create the xml based rpm metadata. To create this use the createrepo program we installed earlier.

At the shell type the following command:

createrepo -v -s md5 /var/www/html/repo/recital/updates

This will create the required metadata in the repodata directory of your /var/www/html/repo/recital/updates directory.

root@test repodata]# ls -l
rwotal 44
-rw-r--r-- 1 root root 28996 Jan 13 21:42 filelists.xml.gz
-rw-r--r-- 1 root root   284 Jan 13 21:42 other.xml.gz
-rw-r--r-- 1 root root  1082 Jan 13 21:42 primary.xml.gz
-rw-r--r-- 1 root root   951 Jan 13 21:42 repomd.xml

To do a final consistency check on your repository run the following command:

verifytree /var/www/html/repo/recital/updates

We now have a fully functioning YUM repository for our hosted rpm packages.
The next process is to create a .repo file in the client systems /etc/yum.repos.d directory.

Navigate to the /etc/yum.repos.d directory on your system as root.

Using your preferred text editor to create the .repo file. In this example I will call it recital.repo.
Now paste in the following lines:

[Recital]
name=Recital Update Server
baseurl=http://ftp.recitalsoftware.com/repo/recital/updates
enabled=1
gpgcheck=1

Once that is saved, at the shell prompt on the same machine (YUM client system).

$ yum repolist
Loaded plugins: presto, refresh-packagekit
repo id                  repo name                                 status
Recital                  Recital Update Server                     enabled:      1
adobe-linux-i386         Adobe Systems Incorporated                enabled:     17
fedora                   Fedora 12 - i386                          enabled: 15,366

As you can see the Recital repo is now being picked up and we have access to all the packages it is hosting.

See how easy that was!