This article discusses the features in Recital that allow data to be imported and exported between platforms in Microsoft® ADO XML Format.

Overview

Extensible Markup Language, XML, is widely regarded as a lingua franca for the interchange of data. XML's text-based, platform-independent format and its integration of data and the schema to define and describe that data, make it the ideal import/export medium. Recital software provides the functionality to output the data from Recital - and other supported table formats such as FoxPro and FoxBASE - into XML file format and to import XML data into those tables' formats. Such import/export operations provide the means to exchange data with third-party applications and can also facilitate the transfer of data between Recital installations on binary-incompatible platforms.

The features examined in this article are available in Recital Terminal Developer and in the Recital Mirage and Recital Database Servers on all Recital supported platforms. Both the Recital/4GL and Recital/SQL provide XML import and export capabilities. The XML files discussed are in Microsoft® ADO XML format.

Microsoft® ActiveX® Data Objects XML Format

The ADO XML format is primarily designed for ADO Recordset persistence and ADO XML files created by Recital can be used in this way and loaded directly into ADO Recordsets. The format can, though also be used for more generic data transfer. An ADO XML file is self-contained, consisting of two sections: a schema section followed by a data section. The schema conforms to the W3C XML-Data specification and defines the data structure.
For additional information on the Microsoft® ActiveX® Data Objects XML Format, please see Appendix 1.

NOTE: The Recital XMLFORMAT setting should always be in its default setting of ADO for ADO XML Format operations.

set xmlformat to ADO

SQL

Recital/SQL offers the ability to export data into XML files using the SELECT and FETCH statements and import from XML using the CREATE TABLE and INSERT statements.

SQL: Exporting

The SELECT...SAVE AS XML statement allows the complete result set from a SELECT statement to be saved as an XML file. This could be a complete table:

open database southwind
  SELECT * from orders SAVE AS XML orders.xml

or a more complex multi-table query:

open database southwind
SELECT orders.orderid, orders.customerid,;
    employees.employeeid, employees.lastname, employees.firstname,;
    orders.orderdate, orders.freight, orders.requireddate,;
    orders.shippeddate, orders.shipvia, orders.shipname,;
    orders.shipaddress, orders.shipcity,;
    orders.shipregion, orders.shippostalcode, orders.shipcountry,;
    customers.companyname, customers.address, customers.city,;
    customers.region, customers.postalcode, customers.country; 
    FROM orders INNER JOIN customers;
    ON customers.customerid = orders.customerid,;
    orders INNER JOIN employees;
    ON orders.employeeid = employees.employeeid;
    SAVE AS XML orderinfo

The resulting XML file can then be further processed within the same or a different Recital environment or transferred to a third party product.

<x-ml xmlns:z="#RowsetSchema" xmlns:rs="urn:schemas-microsoft-com:rowset"
 xmlns:dt="uuid:C2F41010-65B3-11d1-A29F-00AA00C14882" xmlns:s="uuid:BDC6E3F0-6DA3-11d1-A2A3-
00AA00C14882">
 <s:schema id="RowsetSchema">
  <s:elementtype rs:updatable="true" content="eltOnly" name="row">
   <s:attributetype rs:basecolumn="orderid" rs:basetable="orders.dbf" rs:write="true"
 rs:nullable="true" rs:number="1" name="orderid">
   <s:datatype rs:fixedlength="true" rs:precision="14" rs:scale="0" dt:maxlength="10"
 rs:dbtype="numeric" dt:type="number">
   </s:datatype></s:attributetype>
   <s:attributetype rs:basecolumn="customerid" rs:basetable="orders.dbf" rs:write="true"
 rs:nullable="true" rs:number="2" name="customerid">
   <s:datatype rs:fixedlength="true" dt:maxlength="5" rs:dbtype="str" dt:type="string">
   </s:datatype></s:attributetype>
   <s:attributetype rs:basecolumn="employeeid" rs:basetable="orders.dbf" rs:write="true"
 rs:nullable="false" rs:number="3" name="employeeid">
   <s:datatype rs:fixedlength="true" rs:precision="20" rs:scale="0" dt:maxlength="10"
 rs:dbtype="numeric" dt:type="number">
   </s:datatype></s:attributetype>
   <s:attributetype rs:basecolumn="lastname" rs:basetable="orders.dbf" rs:write="true"
 rs:nullable="false" rs:number="4" name="lastname">
   <s:datatype rs:fixedlength="true" dt:maxlength="20" rs:dbtype="str" dt:type="string">
   </s:datatype></s:attributetype>
   <s:attributetype rs:basecolumn="firstname" rs:basetable="orders.dbf" rs:write="true"
 rs:nullable="false" rs:number="5" name="firstname">
   <s:datatype rs:fixedlength="true" dt:maxlength="10" rs:dbtype="str" dt:type="string">
   </s:datatype></s:attributetype>
   <s:attributetype rs:basecolumn="orderdate" rs:basetable="orders.dbf" rs:write="true"
 rs:nullable="true" rs:number="6" name="orderdate">
   <s:datatype rs:fixedlength="true" dt:maxlength="10" rs:dbtype="Date" dt:type="Date">
   </s:datatype></s:attributetype>
   <s:attributetype name="freight" ...

Click image to display full size

Fig 1: Microsoft® Office Excel 2003: orderinfo.xml.

For data accessed through a Recital Database Gateway, such as Oracle, MySQL or PostgreSQL, the FETCH command can be used to save a cursor results set into an XML file:

// Connect to MySQL Database 'mydata' via Recital Database Gateway
nStatHand=SQLSTRINGCONNECT("mys@mysql1:user1/pass1-mydata",.T.)
if nStatHand < 1
  dialog box [Could not connect]
else
  DECLARE cursor1 CURSOR FOR;
      SELECT account_no, last_name, first_name FROM example
  OPEN cursor1
  FETCH cursor1 INTO XML exa1.xml
  SQLDISCONNECT(nStatHand)
endif

SQL: Importing

The CREATE TABLE statement allows a new table to be created based on the structure defined in an XML file. The data from the XML file can optionally be loaded into this new table if the LOAD keyword is included. For example, a new 'orderinfo' table can be created and populated with data from the orderinfo.xml file created by the SELECT...SAVE AS XML statement shown earlier:

open database southwind
SELECT orders.orderid, orders.customerid,;
    employees.employeeid, employees.lastname, employees.firstname,;
    orders.orderdate, orders.freight, orders.requireddate,;
    orders.shippeddate, orders.shipvia, orders.shipname,;
    orders.shipaddress, orders.shipcity,;
    orders.shipregion, orders.shippostalcode, orders.shipcountry,;
    customers.companyname, customers.address, customers.city,;
    customers.region, customers.postalcode, customers.country; 
    FROM orders INNER JOIN customers;
    ON customers.customerid = orders.customerid,;
    orders INNER JOIN employees;
    ON orders.employeeid = employees.employeeid;
    SAVE AS XML orderinfo

CREATE TABLE orderinfo FROM XML orderinfo LOAD

The INSERT statement can be used to load data when the table structure already exists. Taking our earlier orderinfo.xml file again, the data can be loaded using INSERT:

open database southwind;
SELECT orders.orderid, orders.customerid,;
    employees.employeeid, employees.lastname, employees.firstname,;
    orders.orderdate, orders.freight, orders.requireddate,;
    orders.shippeddate, orders.shipvia, orders.shipname,;
    orders.shipaddress, orders.shipcity,;;
    orders.shipregion, orders.shippostalcode, orders.shipcountry,;
    customers.companyname, customers.address, customers.city,;
    customers.region, customers.postalcode, customers.country; 
    FROM orders INNER JOIN customers;
    ON customers.customerid = orders.customerid,;
    orders INNER JOIN employees;
    ON orders.employeeid = employees.employeeid;
    SAVE AS XML orderinfo
CREATE TABLE orderinfo FROM XML orderinfo

INSERT INTO orderinfo FROM XML orderinfo

The examples above show the export and import in a single piece of code. To transfer data between binary-incompatible platforms, the export phase using SELECT...SAVE AS XML would be carried out on the source platform, the resulting XML file would be transferred to the target platform, then the import phase using CREATE TABLE...LOAD or CREATE TABLE + INSERT would be run on the target platform.

Recital/4GL

The Recital/4GL offers the ability to export data into XML files using the COPY TO ... TYPE XML command and import from XML using the XMLFIRST() and XMLNEXT() functions.

Recital/4GL: Exporting

The COPY TO command can be used to export data from Recital and other natively supported tables out to a wide range of formats. This includes exporting to an XML file. The '.xml' file extension is added automatically. The COPY TO command can be used to export an entire table:

open database southwind
use orders
copy to orders type xml

or, using the FIELDS clause and the FOR or WHILE clauses, restrict the field list and export only those records which match a particular condition:

open database southwind
use orders
copy to orders type xml fields orderid for year(orderdate) = 1996

Only the orderid field from those records which match the condition is exported:

<x-ml xmlns:z="#RowsetSchema" xmlns:rs="urn:schemas-microsoft-com:rowset"
 xmlns:dt="uuid:C2F41010-65B3-11d1-A29F-00AA00C14882" xmlns:s="uuid:BDC6E3F0-6DA3-11d1-A2A3-
00AA00C14882">
 <s:schema id="RowsetSchema">
  <s:elementtype rs:updatable="true" content="eltOnly" name="row">
   <s:attributetype rs:basecolumn="ORDERID" rs:basetable="ORDERS" rs:write="true" rs:nullable="true"
 rs:number="1" name="ORDERID">
   <s:datatype rs:fixedlength="true" rs:precision="10" rs:scale="0" dt:maxlength="10"
 rs:dbtype="numeric" dt:type="number">
   </s:datatype></s:attributetype>
  </s:elementtype>
 </s:schema>
 <rs:data>
 <z:row orderid="10248">
 <z:row orderid="10249">
 <z:row orderid="10250">
 <z:row ...

Recital/4GL: Importing

Data from an XML file can be extracted one record at a time using the XMLFIRST() and XMLNEXT() functions. XMLFIRST() reads the first record from an XML file and loads information from the file into a series of memory variables and arrays. The record data is loaded into a one-dimensional array which is created automatically. Each element in the array contains the data for its corresponding field in string format. The field names are loaded into another automatically-created array. The XMLNEXT() function works in a similar way to deal with all the subsequent records in the XML file. The XMLCOUNT() function can be used, as in the example below, to determine how many data records the XML file has.

The Recital/4GL includes a vast range of functions for manipulation and conversion of arrays and their individual elements. In the example program below, the XMLFIRST() and XMLNEXT() functions are used to sequentially extract each record from an XML file, whose name is passed to the program as a parameter. Once loaded into an array, the data is converted to the correct Recital data type then appended into a table. The table name is also passed as a parameter.

procedure replaceit
  append blank
  for i = 1 to numfields
    if type(field(i)) = "N"
      replace &(field(i)) with val(data[&i])
    elseif type(field(i)) = "D"
      replace &(field(i)) with stod(data[&i])
    elseif type(field(i)) = "T"
      replace &(field(i)) with ctot(data[&i])
    elseif type(field(i)) = "L"
      replace &(field(i)) with iif(data[&i]="T",.T.,.F.)
    elseif type(field(i)) = "Y"
      replace &(field(i)) with val(data[&i])
    else
      replace &(field(i)) with data[&i]
    endif
  next
return
 
procedure starthere                    
  parameters cTable, cFile
  numfields=xmlfirst(cFile,targ,trans,where,fldnames,data)
  if numfields < 1
    dialog box [No records in XML file]
  else
    use &cTable
    replaceit()
  endif
  numrecs = xmlcount(cFile)
  if numrecs > 1
    numleft = numrecs -1
    for i = 1 to numleft
      xmlnext(trans,where,fldnames,data)
      replaceit()
    next
  endif
return

Alternative Import/Export Methods

Other features exist in Recital to facilitate the import and export of data:

RDDs

The RDDs, Replaceable Database Drivers, are available on Windows, Linux and all supported 32-bit UNIX platforms. They allow for the use and creation of database tables and indexes in FoxPro, dBase and Clipper formats. The file format is the same across all the platforms that support the RDDs, allowing the tables and indexes to be transferred as required. The formats are also supported by a wide range of third-party products as well as their originating database systems. For more information on the RDDs, please see the online documentation on Xbase migration and the SET FILETYPE command.

BUILD/INSTALL

These are Recital/4GL commands for the export (BUILD) and import (INSTALL) of Recital tables and their associated memo, dictionary and multiple index files in ASCII format to allow them to be transferred across binary incompatible platforms. For more information, please see the online documentation on Recital/4GL commands.

COPY Commands

The COPY TO, COPY STRUCTURE, COPY STRUCTURE EXTENDED and CREATE FROM commands can all be used to enable data to be transferred between different formats and different platforms. For more information, please see the online documentation on Recital/4GL commands.

Appendix 1: Microsoft® ActiveX® Data Objects XML Format

For detailed information on the Microsoft® ActiveX® Data Objects XML Format, please consult the following Microsoft documentation:

Privacy Policy

SET DATADIR TO [ <directory> ] 

In this article Chris Mavin, explains and details how to use the Recital Database Server with the Open Source Servlet Container Apache Tomcat.

Overview

PHP has exploded on the Internet, but its not the only way to create web applications and dynamic websites. Using Java Servlets, JavaServer Pages and Apache Tomcat you can develop web applications in a more powerful full featured Object Oriented Language, that is easier to debug, maintain, and improve.

Tomcat Installation

There are a number of popular Java application servers such as IBM Web Sphere and BEA WebLogic but today we will be talking about the use of Apache Tomcat 5, the Open Source implementation of the Java Servlet and JavaServer Pages technologies developed at the Apache Software Foundation. The Tomcat Servlet engine is the official reference implementation for both the Servlet and JSP specifications, which are developed by Sun under the Java Community Process. What this means is that the Tomcat Server implements the Servlet and JSP specifications as well or better than most commercial application servers.

Apache Tomcat is available for free but offers many of the same features that commercially available Web application containers boast.

Tomcat 5 supports the latest Servlet and JSP specifications, Servlet 2.4, and JSP 2.0, along with features such as:

• Tomcat can run as a standalone webserver or a Servlet/JSP engine for other Web Servers.

• Multiple connectors - for enabling multiple protocol handlers to access the same Servlet engine.

• JNDI - The Java Naming and Domain Interface is supported.

• Realms - Databases of usernames and passwords that identify valid users of a web application.

• Virtual hosts - a single server can host applications for multiple domain names. You need to edit server.xml to configure virtual hosts.

• Valve chains.

• JDBC - Tomcat can be configured to use any JDBC driver.

• DBCP - Tomcat can use the Apache commons DBCP for connection pooling.

• Servlet reloading (Tomcat monitors any changes to the classes deployed within that web server.)

• HTTP functionality - Tomcat functions as a fully featured Web Server.

• JMX, JSP and Struts-based administration.

Tomcat Installation

In this next two sections we will walk through the install and setup of Tomcat for use with the Recital database server.

To download Tomcat visit the Apache Tomcat web site is at http://jakarta.apache.org/tomcat.
Follow the download links to the binary for the hardware and operating system you require.

For Tomcat to function fully you need a full Java Development Kit (JDK). If you intend to simply run pre compiled JavaServer pages you can do so using just the Java Runtime Environment(JRE).

The JDK 1.5 is the preferred Java install to work with Tomcat 5, although it is possible to run Tomcat 5 with JDK 1.4 but you will have to download and install the compat archive available from the Tomcat website.

For the purpose of this article we will be downloading and using Tomcat 5 for Linux and JDK 5.0, 
you can download the JDK at http://java.sun.com/javase/downloads/index.jsp.

Now we have the JDK, if the JAVA_HOME environment variable isn't set we need to set it to refer to the base JDK install directory.

Linux/Unix:

$ JAVA_HOME= /usr/lib/j2se/1.4/
$ EXPORT $JAVA_HOME

Windows NT/2000/XP:

Follow the following steps:

1. Open Control Panel.
2. Click the System icon.
3. Go to the Advanced tab.
4. Click the Environment Variables button.
5. Add the JAVA_HOME variable into the system environment variables.

The directory structure of a Tomcat installation comprises of the following:

/bin 			- Contains startup, shutdown and other scripts. 
	/common  	- Common classes that the container and web applications can use.
	/conf 		- Contains Tomcat XML configuration files XML files.
	/logs 		- Serlvet container and application logs.
	/server 		- Classes used only by the Container.
	/shared 		- Classes shared by all web application.
	/webapps 	- Directory containing the web applications.
	/work 		- Temporary directory for files and directories.

The important files that you should know about are the following:

• server.xml

The Tomcat Server main configuration file is the [tomcat install path]\conf\server.xml file. This file is mostly setup correctly for general use. It is within this file where you specify the port you wish to be running the server on. Later in this article I show you how to change the default port used from 8080 to port 80.

• web.xml

The web.xml file provides the configuration for your web applications. There are two locations where the web.xml file is used, 
web-inf\web.xml provides individual web application configurations and [tomcat install path]conf\web.xml contains the server wide configuration.

Setting up Tomcat for use

We'll start by changing the port that Tomcat will be listening on to 80.

To do this we need to edit [tomcat install path]/conf/server.xml and change the port attribute of the connector element from 8080 to 80.

After you have made the alteration the entry should read as:

<!-- Define a non-SSL HTTP/1.1 Connector on port 8080 -->
<Connector port="80" maxHttpHeaderSize="8192"

Next we want to turn on Servlet reloading, this will cause the web application to be recompiled each time it is accessed, allowing us to make changes to the files without having to worry about if the page is being recompiled or not.

To enable this you need to edit [tomcat install path]/conf/context.xml and change <Context> element to <Context reloadable="true">.

After you have made the alteration the entry should read as:

<Context reloadable="true">
<WatchedResource>WEB-INF/web.xml</WatchedResource>
</Context>

Next we want to enable the invoker Servlet.

The "invoker" Servlet executes anonymous Servlet classes that have not been defined in a web.xml file.  Traditionally, this Servlet is mapped to the URL pattern "/servlet/*", but you can map it to other patterns as well.  The extra path info portion of such a request must be the fully qualified class name of a Java class that implements Servlet, or the Servlet name of an existing Servlet definition.

To enable the invoker Servlet you need to edit the to [tomcat install path]/conf/web.xml and uncomment the Servlet and Servlet-mapping elements that map the invoker /servlet/*.

After you have made the alteration the entry should read as:

<servlet>
<servlet-name>invoker</servlet-name>
<servlet-class>org.apache.catalina.servlets.InvokerServlet</servlet-class>
<init-param>
<param-name>debug</param-name>
<param-value>0</param-value>
</init-param>
<load-on-startup>2</load-on-startup>
</servlet>

<servlet-mapping>
<servlet-name>invoker</servlet-name>
<url-pattern>/servlet/*</url-pattern>
</servlet-mapping>

If you are you not interested in setting up your own install of Tomcat there are prebuilt versions Tomcat that has all of the above changes already made, and has the test HTML, JSP, and Servlet files already bundled. Just unzip the file, set your JAVA_HOME

Next we will give Tomcat and your web applications access to the Recital JDBC driver.

For the purposes of this article we are going to install the Recital JDBC driver in the /[tomcat install path]/common/lib/ this gives Tomcat and your web applications access to the Recital JDBC driver. The driver can be installed in a number of places in the Tomcat tree, giving access to the driver to specific application or just to the web application and not the container. For more information refer to the Tomcat documentation.

Copy the recitalJDBC.jar which is located at /[recital install path]/drivers/recitalJDBC.jar to the /[tomcat install path]/common/lib/ directory.

Linux:

$cp /[recital install path]/drivers/recitalJDBC.jar /[tomcat install path]/common/lib/

Once you have completed all the steps detailed above, fire up the server using the script used by your platform's Tomcat installation.

Linux/Unix:

[tomcat install path]/bin/startup.sh

Windows:

[tomcat install path]/bin/startup

If you are having problems configuring your Tomcat Installation or would like more detail visit the online documentation a the Apache Tomcat site.

Example and Links

Now we have setup our Tomcat installation, lets get down to it with a JSP example which uses the Recital JDBC driver to access the demonstration database (southwind) shipped with the Recital Database Server.

The example provided below is a basic JDBC web application, where the user simply selects a supplier from the listbox and requests the products supplied by that supplier.

To run the example download and extract the tar archive or simple save each of the two jsp pages individually into /[tomcat install path]/webapps/ROOT/ on your server.

By enabling the invoker Servlet earlier we have removed the need to set the example up as a web application in the Tomcat configuration files.

You can now access the example web application at http://[Server Name]/supplier.jsp if the page doesn't display, check you have followed all the Tomcat installation steps detailed earlier in this article and then make sure both Tomcat and a licensed Recital UAS are running.

Downloads:
Archive: jspExample.tar

Right click and save as individual files and rename as .jsp files:
supplier.txt details.txt

Further Reading on JSP and JDBC can be found at http://www-128.ibm.com/developerworks/java/library/j-webdata/

Final Thoughts

Recital and Apache tomcat are a powerful combination, using Java Servlet technology you can separate application logic and the presentation extremely well. Tomcat, JSP, Java Servlets and the Recital database server form a robust platform independent, easily maintained and administered solution with which to unlock the power of your Recital, Foxpro, Foxbase, Clipper, RMS and C-SAM data.