In this article Chris Mavin, explains and details how to use the Recital Database Server with the Open Source Servlet Container Apache Tomcat.
Overview
PHP has exploded on the Internet, but its not the only way to create web applications and dynamic websites. Using Java Servlets, JavaServer Pages and Apache Tomcat you can develop web applications in a more powerful full featured Object Oriented Language, that is easier to debug, maintain, and improve.
Tomcat Installation
There are a number of popular Java application servers such as IBM Web Sphere and BEA WebLogic but today we will be talking about the use of Apache Tomcat 5, the Open Source implementation of the Java Servlet and JavaServer Pages technologies developed at the Apache Software Foundation. The Tomcat Servlet engine is the official reference implementation for both the Servlet and JSP specifications, which are developed by Sun under the Java Community Process. What this means is that the Tomcat Server implements the Servlet and JSP specifications as well or better than most commercial application servers.
Apache Tomcat is available for free but offers many of the same features that commercially available Web application containers boast.
Tomcat 5 supports the latest Servlet and JSP specifications, Servlet 2.4, and JSP 2.0, along with features such as:
-
Tomcat can run as a standalone webserver or a Servlet/JSP engine for other Web Servers.
-
Multiple connectors - for enabling multiple protocol handlers to access the same Servlet engine.
-
JNDI - The Java Naming and Domain Interface is supported.
-
Realms - Databases of usernames and passwords that identify valid users of a web application.
-
Virtual hosts - a single server can host applications for multiple domain names. You need to edit server.xml to configure virtual hosts.
-
Valve chains.
-
JDBC - Tomcat can be configured to use any JDBC driver.
-
DBCP - Tomcat can use the Apache commons DBCP for connection pooling.
-
Servlet reloading (Tomcat monitors any changes to the classes deployed within that web server.)
-
HTTP functionality - Tomcat functions as a fully featured Web Server.
-
JMX, JSP and Struts-based administration.
Tomcat Installation
In this next two sections we will walk through the install and setup of Tomcat for use with the Recital database server.
To download Tomcat visit the Apache Tomcat web site is at http://jakarta.apache.org/tomcat.
Follow the download links to the binary for the hardware and operating system you require.
For Tomcat to function fully you need a full Java Development Kit (JDK). If you intend to simply run pre compiled JavaServer pages you can do so using just the Java Runtime Environment(JRE).
The JDK 1.5 is the preferred Java install to work with Tomcat 5, although it is possible to run Tomcat 5 with JDK 1.4 but you will have to download and install the compat archive available from the Tomcat website.
For the purpose of this article we will be downloading and using Tomcat 5 for Linux and JDK 5.0,
you can download the JDK at http://java.sun.com/javase/downloads/index.jsp.
Now we have the JDK, if the JAVA_HOME environment variable isn't set we need to set it to refer to the base JDK install directory.
Linux/Unix:
$ JAVA_HOME= /usr/lib/j2se/1.4/ $ EXPORT $JAVA_HOME
Windows NT/2000/XP:
Follow the following steps:
1. Open Control Panel.
2. Click the System icon.
3. Go to the Advanced tab.
4. Click the Environment Variables button.
5. Add the JAVA_HOME variable into the system environment variables.
The directory structure of a Tomcat installation comprises of the following:
/bin - Contains startup, shutdown and other scripts. /common - Common classes that the container and web applications can use. /conf - Contains Tomcat XML configuration files XML files. /logs - Serlvet container and application logs. /server - Classes used only by the Container. /shared - Classes shared by all web application. /webapps - Directory containing the web applications. /work - Temporary directory for files and directories.
The important files that you should know about are the following:
-
server.xml
The Tomcat Server main configuration file is the [tomcat install path]\conf\server.xml file. This file is mostly setup correctly for general use. It is within this file where you specify the port you wish to be running the server on. Later in this article I show you how to change the default port used from 8080 to port 80.
-
web.xml
The web.xml file provides the configuration for your web applications. There are two locations where the web.xml file is used,
web-inf\web.xml provides individual web application configurations and [tomcat install path]conf\web.xml contains the server wide configuration.
Setting up Tomcat for use
We'll start by changing the port that Tomcat will be listening on to 80.
To do this we need to edit [tomcat install path]/conf/server.xml and change the port attribute of the connector element from 8080 to 80.
After you have made the alteration the entry should read as:
<!-- Define a non-SSL HTTP/1.1 Connector on port 8080 --> <Connector port="80" maxHttpHeaderSize="8192"
Next we want to turn on Servlet reloading, this will cause the web application to be recompiled each time it is accessed, allowing us to make changes to the files without having to worry about if the page is being recompiled or not.
To enable this you need to edit [tomcat install path]/conf/context.xml and change <Context> element to <Context reloadable="true">.
After you have made the alteration the entry should read as:
<Context reloadable="true"> <WatchedResource>WEB-INF/web.xml</WatchedResource> </Context>
Next we want to enable the invoker Servlet.
The "invoker" Servlet executes anonymous Servlet classes that have not been defined in a web.xml file. Traditionally, this Servlet is mapped to the URL pattern "/servlet/*", but you can map it to other patterns as well. The extra path info portion of such a request must be the fully qualified class name of a Java class that implements Servlet, or the Servlet name of an existing Servlet definition.
To enable the invoker Servlet you need to edit the to [tomcat install path]/conf/web.xml and uncomment the Servlet and Servlet-mapping elements that map the invoker /servlet/*.
After you have made the alteration the entry should read as:
<servlet> <servlet-name>invoker</servlet-name> <servlet-class>org.apache.catalina.servlets.InvokerServlet</servlet-class> <init-param> <param-name>debug</param-name> <param-value>0</param-value> </init-param> <load-on-startup>2</load-on-startup> </servlet> <servlet-mapping> <servlet-name>invoker</servlet-name> <url-pattern>/servlet/*</url-pattern> </servlet-mapping>
If you are you not interested in setting up your own install of Tomcat there are prebuilt versions Tomcat that has all of the above changes already made, and has the test HTML, JSP, and Servlet files already bundled. Just unzip the file, set your JAVA_HOME
Next we will give Tomcat and your web applications access to the Recital JDBC driver.
For the purposes of this article we are going to install the Recital JDBC driver in the /[tomcat install path]/common/lib/ this gives Tomcat and your web applications access to the Recital JDBC driver. The driver can be installed in a number of places in the Tomcat tree, giving access to the driver to specific application or just to the web application and not the container. For more information refer to the Tomcat documentation.
Copy the recitalJDBC.jar which is located at /[recital install path]/drivers/recitalJDBC.jar to the /[tomcat install path]/common/lib/ directory.
Linux:
$cp /[recital install path]/drivers/recitalJDBC.jar /[tomcat install path]/common/lib/
Once you have completed all the steps detailed above, fire up the server using the script used by your platform's Tomcat installation.
Linux/Unix:
[tomcat install path]/bin/startup.sh
Windows:
[tomcat install path]/bin/startup
If you are having problems configuring your Tomcat Installation or would like more detail visit the online documentation a the Apache Tomcat site.
Example and Links
Now we have setup our Tomcat installation, lets get down to it with a JSP example which uses the Recital JDBC driver to access the demonstration database (southwind) shipped with the Recital Database Server.
The example provided below is a basic JDBC web application, where the user simply selects a supplier from the listbox and requests the products supplied by that supplier.
To run the example download and extract the tar archive or simple save each of the two jsp pages individually into /[tomcat install path]/webapps/ROOT/ on your server.
By enabling the invoker Servlet earlier we have removed the need to set the example up as a web application in the Tomcat configuration files.
You can now access the example web application at http://[Server Name]/supplier.jsp if the page doesn't display, check you have followed all the Tomcat installation steps detailed earlier in this article and then make sure both Tomcat and a licensed Recital UAS are running.
Downloads:
Archive: jspExample.tar
Right click and save as individual files and rename as .jsp files:
supplier.txt details.txt
Further Reading on JSP and JDBC can be found at http://www-128.ibm.com/developerworks/java/library/j-webdata/
Final Thoughts
Recital and Apache tomcat are a powerful combination, using Java Servlet technology you can separate application logic and the presentation extremely well. Tomcat, JSP, Java Servlets and the Recital database server form a robust platform independent, easily maintained and administered solution with which to unlock the power of your Recital, Foxpro, Foxbase, Clipper, RMS and C-SAM data.
- For building shared libraries on the MAC the following need to be set
-
- The shared library file extension should be .dylib
- The compile flag is -dynamic
- For accessing the shared libraries at runtime
-
- DYLD_LIBRARY_PATH needs to be set to the location of the shared libraries
- Useful utilities for shared library support
-
- The following command will display the table of contents of the dynamically linked library
otool -TV sharedlibraryfile.dylib
The getUIComponentBitmapData method can create bitmapdata for a given IUIComponent. Pass any UIcomponent to get its respective bitmapdata.
public static function getUIComponentBitmapData(target:IUIComponent):BitmapData { var resultBitmapData:BitmapData = new BitmapData(target.width, target.height); var m:Matrix = new Matrix(); resultBitmapData.draw(target, m); return resultBitmapData; }
Now convert the bitmapdata to a jpeg bytearray.
private static function encodeToJPEG(data:BitmapData, quality:Number = 75):ByteArray { var encoder:JPGEncoder = new JPGEncoder(quality); return encoder.encode(data); }
Now encode the ByteArray into Base64.
public static function base64Encode(data:ByteArray):String { var encoder:Base64Encoder = new Base64Encoder(); encoder.encodeBytes(data); return encoder.flush(); }
Upload the base64 encoded ByteArray to the server.
public static uploadData():void { var url:String = "saveFile.php"; var urlRequest:URLRequest = new URLRequest(url); urlRequest.method = URLRequestMethod.POST; var urlLoader:URLLoader = new URLLoader(); var urlVariables:URLVariables = new URLVariables(); urlVariables.file = jpgEncodedFile; // as returned from base64Encode() urlLoader.data = urlVariables; urlLoader.load(urlRequest); }
The saveFile.php file on the server.
$input = $_POST['file']; $fp = fopen('filename.jpg', 'w'); fwrite($fp, base64_decode($input)); fclose($fp); ?>
$ lsof | grep db.exe | grep accounts db.exe 16897 john 6uw REG 253,0 20012 3413872 /usr/recital100/qa/accounts.dbf db.exe 16897 john 7u REG 253,0 4176 3413885 /usr/recital100/qa/accounts.dbxIf you want to check for locks you can use lslk, for example;
$ lslk | grep db.exe | grep accounts db.exe 16897 253,0 3413872 20012 w 0 0 0 12319 0 /usr/recital100/qa/accounts.dbfIf you don't have lslk installed you can install it with one of the updaters, for example on redhat linux:
$ yum update lslk
Opening SSH to the outside world is a security risk. Here is how to restrict SSH access to certain IP addresses on a machine.
- Edit the /etc/hosts.allow file to include these lines, assuming your machine is on the 192.168.2.x nonrouting IP block, and you want to enable an external address of 217.40.111.121 IP block: Remember to add the period on the end of each incomplete IP number. If you have another complete IP address or range, add a space and that range on the end.
sshd,sshdfwd-X11: 192.168.2. 217.40.111.121
- Edit your /etc/hosts.deny file to include this line:
sshd,sshdfwd-X11:ALL
- These lines refuse SSH connections from anyone not in the IP address blocks listed.
Additionally you can restrict SSH access by username.
- Edit the /etc/ssh/sshd_config file and add the following lines
PermitRootLogin no
AllowUsers user1 user2 user3 etc
PasswordAuthentication yes
Now restart the ssh daemon for these changes to take effect
service sshd restart
If you are using the Oracle Gateway in Recital, make sure the Oracle environment (ORACLE_HOME, ORACLE_SID etc.) is set up before starting the Recital Server. If not, you will see the error ORA-01019. A call to the Oracle environment setup script can be added to the /etc/init.d/recital script if your Recital Server is set to run on startup.
Overview
file which contains information, including full path details, about all the files that belong to that particular database. Opening a database gives an application access to all that database's files - operating system and data dictionary protection and security permitting - in whichever directory on the system they reside and database commands can target the files as a single unit.NOTE: The word 'database' has often been used in Xbase products to refer to an individual '.dbf' file. In this article these are referred to as 'tables' and a database may contain many tables.
Advantages
Recital has always offered developers the maximum in flexibility in the design and deployment of their applications. The SET DEFAULT and SET PATH commands along with the ability to access files via their full Operating System path name has allowed data and code to be created and stored in any required location. Such flexibility does however put the onus on the developer to manage all elements of the application and ensure that maintenance tasks cover all files. The introduction of the database commands retains the established developer-driven design, but provides functionality to facilitate the access and maintenance of the components of that design. The functionality offered is in three main areas:- Facilitate data access
- Facilitate the storage and extraction of information about the files in an application
- Facilitate the maintenance of the files in an application
These three areas are looked at in more detail in the sections below.
Data Access
The immediate effect of using a database is that all the database's tables and associated files (indexes, dictionaries, memo files) are accessible with the issue of a single command, the OPEN DATABASE command.// Open the database
> open database southwind
// List the database tables
> list tables
Tables in Database southwind:
Name Source
categories /usr/recital/data/southwind/categories.dbf
customers /usr/recital/data/southwind/customers.dbf
employees /usr/recital/data/southwind/employees.dbf
example /usr/recital/data/southwind/example.dbf
order_details /usr/recital/data/southwind/order_details.dbf
orders /usr/recital/data/southwind/orders.dbf
productsbyname /usr/recital/data/southwind/productsbyname.dbf
products /usr/recital/data/southwind/products.dbf
shippers /usr/recital/data/southwind/shippers.dbf
suppliers /usr/recital/data/southwind/suppliers.dbf
cisamdemo /usr/recital/data/southwind/cisamdemo.dbf
/ Open a tableIn the case of the sample southwind database that ships with Recital products on all platforms and is shown above, this effect can be achieved in other ways (SET PATH or SET DEFAULT), since its files all reside in the southwind directory. The database advantage comes when the database catalog contains files from multiple directories, e.g.
> use example
// Open the database
> open database myapp
// List the database tables
> list tables
Tables in Database myapp:
Name Source
zipcodes /usr/myapp/data/lookups/zipcodes.dbf
customers /usr/myapp/data/current/customers.dbf
archive03 /usr/myapp/data/archive/archive03.dbf
archive04 /usr/myapp/data/archive/archive04.dbf
users /usr/allapps/data/users.dbf
menus /usr/myapp/data/system/menus.dbf
// Open a tableThe OPEN DATABASE command requires only the database name to be specified, not the complete path of a directory as with SET PATH or SET DEFAULT. Databases are searched for in the sub-directories of the directory defined in the DB_DATADIR environment variable or symbol. DB_DATADIR is set in the system wide configuration file.
> use users
#---------------------------------------------------NOTE: DB_DATADIR is read from these files at startup to determine the home directory for databases. Updates to DB_DATADIR once a Recital process is running do not change this setting. The OPEN DATABASE command and certain other database commands allow a '?' to be specified in place of a database name. In this case the 'Select a file' dialog is displayed, listing the available files in the DB_DATADIR directory.
# location of directories and important files
#---------------------------------------------------
DB_DATADIR="${ROI_ROOT}data/" ;export DB_DATADIR
Click image to display full size
Fig 1: Recital Terminal Developer OPEN DATABASE ? command.
This functionality is also supported by the PACK DATABASE, REBUILD DATABASE and REINDEX DATABASE commands.
Databases can also simplify data access for Recital Client Drivers using the Recital Database Server: instead of specifying a starting directory, only a database name is required. The database can handle files from multiple directories and associate tables with their single index files.
Recital Universal .NET Data Provider
/////////////////////////////////////////
// include the references below
using System.Data;
using Recital.Data;
/////////////////////////////////////////
// sample code to call a Stored Procedure
public int CallStoredProcedure()
{
string source = "Data Source=localhost;" +
"Database=southwind;" +
"uid=?;"+
"pwd=?";
RecitalConnection conn = new RecitalConnection(source);
...
Recital Universal JDBC Driver
import java.sql.*;
import java.io.*;
import java.net.URL;
import java.math.BigDecimal;
import Recital.sql.*;
public class jdbc_test {
public static void main(String argv[]) {
int i;
ResultSet rs;
ResultSetMetaData rsmd;
System.out.println("Recital JDBC driver verification.");
for (int ii=0; ii<1; ++ii) {
try {
//----------------------------------------------------------
//-- Load the Client Driver for the
//-- Recital Universal Application Server
//----------------------------------------------------------
System.out.println("Loading Recital JDBC driver.");
new RecitalDriver();
//-----------------------------------------------------------
//-- The standard format of the connection URL is as follows:
//-----------------------------------------------------------
String url = "jdbc:Recital:" +
"SERVERNAME=?;" +
"DATABASE=jdbc_test;" +
"USERNAME=?;" +
"PASSWORD=?;" +
"ENCRYPTION=false";
...
Windows Recital Universal ODBC Driver
Click image to display full size
Fig 2: Recital Universal ODBC Driver DSN setup using a database.
Database Procedure Libraries
A database can have an associated procedure library, which is automatically opened when the database is opened. This way, any procedures required by the database's data files are always available. The procedure library should reside in the database's directory and be named dbc_xxx_library.prg, where 'xxx' is the name of the database. When the OPEN DATABASE command is issued, a check is made for the database procedure library and a SET PROCEDURE TO dbc_xxx_library.prg ADDITIVE issued automatically. The procedure library is in turn closed when the CLOSE DATABASES command is issued. This is particularly convenient for client/server database access.Database Events
Issuing the OPEN DATABASE or CLOSE DATABASES command also fires a database event. Database events, like triggers for tables and forms, can have programs associated with them. The OPEN DATABASE command fires the DBC_OPENDATA event and will run a program called db_opendata.prg if one exists in the database's directory. Similarly, the CLOSE DATABASES command fires the DBC_CLOSEDATA event and runs the db_closedata.prg program. Both of these events also allow the current open or close operation to be abandoned if the associated program returns a logical false (.F.).Information
Databases, specifically database catalogs, are an excellent place to store information about the files required by a particular application. Instead of having to search through application code to determine which tables are being used and what index keys they have, this information can be viewed in the database catalog. Database catalogs are themselves Recital tables and can be viewed and/or updated in the same way as any other Recital table. There is also a series of commands to provide information about the currently open database. DISPLAY/LIST DATABASE The DISPLAY DATABASE and LIST DATABASE commands display the database name and path, followed by the equivalent of LIST STRUCTURE INDEX and LIST DICTIONARY for each table in the database, e.g.> open database southwind
> display database
Database Name: southwindDISPLAY DATABASE shows the data one screen at a time, whereas LIST DATABASE is ideal for output to a file.
Database Path: /usr/recital-9.0/data/southwind/
Structure for database : categories.dbf
Number of data records : 8
Date of creation : 05/12/2004
Date of last update : 05/12/2004
Database encrypted : False
Field Field Name Type Width Dec Description 1 CATEGORYID Numeric 10 Category ID 2 CATEGORYNAME Character 15 Category Name 3 DESCRIPTION Memo 8 Description 4 PICTURE Memo 8 Picture ** Total ** 42
Production DBX file: categories.dbx
Master Index TAG: CATEGORYID
Key: categoryid
Type: Binary
Len: 8
...
> open database southwind
> list database to file info.txt
DISPLAY/LIST TABLES
LIST TABLES, as we have seen above, lists all the tables from the database, giving each table's name and path. DISPLAY TABLES shows the information one screen at a time.Maintenance
Using a database can simplify the maintenance of an application's programs, tables and indexes. Multiple files, possibly in different directories, can be targeted by a single database command.COMPILE DATABASE
The COMPILE DATABASE command can be used to issue a COMPILE command for all program files listed in the database catalog.// Open the databaseNOTE: Program files are added to a database using the SQL CREATE PROCEDURE command.
open database myapp
// Compile all the database's programs
compile database
PACK DATABASE
The PACK DATABASE command issues a PACK command for every table in the database catalog. The PACK command is used to permanently remove any records that have been marked for deletion using the DELETE command.// Open the database
open database southwind
// Pack all the database's tables
pack database
REINDEX DATABASE
The REINDEX DATABASE command rebuilds all the tag and single indexes in the catalog. This is the equivalent of an INDEX ON <key> TO | TAG <index> for each catalogued index key.// Open the databaseNOTE: Using a database helps protect against a table being opened without its associated single index files also being opened. Any single index files that are included in the database catalog will automatically be opened when their table is opened via the database. If a single index appears in the database catalog, but the physical file no longer exists, it will be removed from the catalog when its associated table is next opened. All indexes, tagged indexes or single indexes, created while the database is open, are added automatically to the database catalog.
open database southwind
// Rebuild all the database's indexes
reindex database
REBUILD DATABASE
The REBUILD DATABASE command is used to PACK, rebuild the index tags and refresh the contents of the database catalog file.// Rebuild the database
rebuild database southwind
Creating and populating a database
Databases are created using the CREATE DATABASE command.// Create new databaseThe CREATE DATABASE creates a sub-directory in the DB_DATADIR and initializes the new database's catalog file. The catalog file is given the same basename as the database and is a Recital table with a '.cat' file extension. It has a production index file with a '.cax' file extension and a memo file with a '.cam' file extension.
create database newdb
// Open the databaseWith the database open, the catalog file table is opened in the highest available workarea and is given an alias name of its basename preceded by an underscore, e.g. '_newdb'. Any new tables or indexes that are created will be automatically added into the catalog and form part of the database. 'Free' tables can also be manually added into a database using the SQL ADD TABLE command.
open database newdb
// config.dbAs the application runs, 'myapp' in the example above, each table that is opened is added to the database catalog. Indexes that are opened are added in the same way. Using SET AUTOCATALOG OFF, inclusion in the catalog can be restricted. Once all required areas of the application have been catalogued, the application can be updated to make use of the database commands.
set sql on
set sql to vfp
// end of config.db
// Create a 'free' table, with no database open
create table free1 (field1 char(10), field2 date)
// Open the database
open database newdb
add table free1
Converting an existing application
The AUTOCATALOG commands can be used to automatically create a database catalog based on an existing application. The SET AUTOCATALOG TO >database< command will create the database if it does not already exist.// Database must be closed during autocatalog process
close databases
// Specify the database
set autocatalog to myappdb
// Start the autocatalog process
set autocatalog on
do myapp
// The autocatalog process can be toggled off to exclude
// certain parts of the application if required
// set autocatalog off
Exporting a database
The BACKUP DATABASE and RESTORE DATABASE commands have been added to simplify the process of moving applications between binary incompatible platforms. Binary files such as tables, memo files and data dictionaries must be exported to ASCII format to allow them to be transferred from one platform to another where the platforms have different binary ordering. For example, this is the case when moving files between a SUN Sparc Solaris machine and an Intel Linux machine. The BACKUP DATABASE and RESTORE DATABASE commands are extensions of the BUILD and INSTALL commands: where BUILD and INSTALL operate on a specified list of tables, BACKUP DATABASE and RESTORE DATABASE automatically handle an entire database.// Open the databaseThe BACKUP DATABASE command goes through the database catalog, exporting each file into an ASCII format that can be handled by the RESTORE DATABASE command. The files are created in a directory with the same name as the database. This directory is a sub-directory of the directory specified in the environment variable DB_BACKUPDIR and is created automatically. By default, DB_BACKUPDIR is the 'backup' sub-directory of the Recital home directory.
open database southwind
// Export the database to ASCII format
backup database
// Query the DB_BACKUPDIR environment variable settingLike DB_DATADIR, DB_BACKUPDIR is set in the system-wide configuration file and is read at startup to determine the home directory for database backups. Updates to DB_BACKUPDIR once a Recital process is running do not change this setting.
> ? getenv([DB_BACKUPDIR])
/usr/recital/backup
# profile.db/uas extractOnce the BACKUP DATABASE command has completed, the files can be transferred to another platform, for example from Intel SCO OpenServer to IBM AIX and the RESTORE DATABASE command used to recreate the database.
#---------------------------------------------------
# location of directories and important files
#---------------------------------------------------
DB_BACKUPDIR="${ROI_ROOT}backup/" ;export DB_BACKUPDIR
// Export the database to ASCII format
// Note: the BACKUP DATABASE command operates
// on the active or specified database
$ db
> backup database southwind
> quit
// 'tar' up the files for transfer
$ cd /usr/recital/backup
$ tar cvf southwind.tar ./southwind
// Transfer the tar archive to DB_BAKUPDIR on the
// target machine, then extract the files
$ pwd
/usr/recital/backup
$ tar xvf southwind.tar
// Rebuild the database on the target platform
// The database is automatically created
// as a sub-directory of DB_DATADIR
$ db
> restore database southwind
This article discusses Recital database security: from operating system file permissions through file and field protection to DES3 encryption.
Overview
A company's data is extremely valuable and must be protected, both in operation and in physical file format. Recital products provide a range of ways to protect your data.
Operating System File Permissions
The most basic level of database security is provided by the operating system. Recital database tables and indexes are individual files with their own respective operating system file permissions. Read permission is required to open a table and write permission to update a table. If a user does not have read permission they are denied access. Without write permission, a table will be opened read-only.
Here the owner, root, and members of the recital group have write permission, so can update the example table unless additional protection applies. Other users can only open the example table read-only.
# ls -l example* -rwxrwxr-x 1 root recital 147 Nov 29 14:27 example.dbd -rwxrwxr-x 1 root recital 41580 Nov 29 14:27 example.dbf -rwxrwxr-x 1 root recital 13312 Nov 29 14:28 example.dbt -rwxrwxr-x 1 root recital 19456 Nov 29 14:28 example.dbx
Note: As in the example above, a table's associated files should have the same permissions as the table itself:
File Extension |
File Type |
.dbd |
Dictionary |
.dbf |
Table |
.dbt |
Memo |
.dbx |
Index |
Database Dictionary
Each Recital table may have a Database Dictionary. The Dictionary can be used both to protect the integrity of the data and to protect access to the data. This section covers Column Constraints, Triggers, Security and Protection.
Column Constraints: Data Integrity
The Dictionary attributes or constraints either prevent the entry of incorrect data, e.g. must_enter and validation or aid the entry of correct data, e.g. default, picture and choicelist. The Dictionary can be modified in the character mode CREATE/MODIFY STRUCTURE worksurface, via SQL statements, or in the Recital Enterprise Studio Database Administrator.
Click image to display full size
Fig 1: MODIFY STRUCTURE Worksurface: Dictionary.
The SQL Column Constraints are as follows:
Constraint |
Description |
AUTO_INCREMENT | AUTOINC |
Used to auto increment the value of a column. |
CALCULATED |
Used to calculate the value of a column. |
CHECK | SET CHECK |
Used to validate a change to the value of a column. |
DEFAULT |
Used to set a default value for the specified column. |
DESCRIPTION |
Used set the column description for the specified column. |
ERROR |
Used to define an error message to be displayed when a validation check fails. |
FOREIGN KEY |
Used to define a column as a Foreign Key for a parent table. |
NOCPTRANS |
Used to prevent code page translation for character and memo fields. |
NOT NULL | NULL |
Used to disallow/allow NULL values. |
PRIMARY KEY |
Used to define a tables Primary Key. |
RANGE |
Used to specify minimum and maximum values for a date or numerical column. |
RECALCULATE |
Used to force recalculation of calculated columns when a columns value changes. |
REFERENCES |
Used to create a relationship to an index key of another table. |
UNIQUE |
Used to define the column as a candidate index for the table |
These can be specified in CREATE TABLE or ALTER TABLE statements:
exec sql OPEN DATABASE southwind; exec sql ALTER TABLE customers ADD COLUMN timeref char(8) CHECK validtime(timeref) ERROR "Not a valid time string";
Click image to display full size
Fig 2: Database Administrator: Column Constraints and Attributes.
TRIGGERS
Table Level Triggers are event-driven procedures called before an I/O operation. These can be used to introduce another layer of checks before a particular operation is permitted to take place or to simply set up logging of those operations.
The CREATE/MODIFY STRUCTURE worksurface <TRIGGERS> menu bar option allows you to specify table level triggers. You may edit a trigger procedure from within the <TRIGGERS> menu by placing the cursor next to the procedure name and pressing the [HELP] key. A text window pops up for editing. If the table triggers are stored in separate <.prg> files, rather than in a procedure library, procedures need not be predefined (SET PROCEDURE) before using the table.
Click image to display full size
Fig 3: MODIFY STRUCTURE Worksurface: Triggers.
The following triggers can be selected and associated with a specified procedure name in the <TRIGGERS> menu.
Trigger |
Description |
UPDATE |
The specified procedure is called prior to an update operation on the table. If the procedure returns .F., then the UPDATE is canceled. |
DELETE |
The specified procedure is called prior to a delete operation on the table. If the procedure returns .F., then the DELETE is canceled. |
APPEND |
The specified procedure is called prior to an append operation on the table. If the procedure returns .F., then the APPEND is canceled. |
OPEN |
The specified procedure is called after an open operation on the table. |
CLOSE |
The specified procedure is called prior to a close operation on the table. |
ROLLBACK |
The specified procedure is called when a user presses the [ABANDON] key in a forms based operation. |
The Recital Enterprise Studio Database Administrator also allows you to associate existing programs as Table Trigger Procedures.
Click image to display full size
Fig 4: Database Administrator: Triggers.
Programmatically, Trigger Procedures can also be associated with a table using SQL. The following table constraints may be applied in the SQL CREATE TABLE and ALTER TABLE statements:
Trigger |
Description |
ONUPDATE |
The specified procedure is called prior to an update operation
on the table. If the procedure returns .F., then the UPDATE
is canceled. |
ONDELETE |
The specified procedure is called prior to a delete operation
on the table. If the procedure returns .F., then the DELETE
is canceled. |
ONINSERT |
The specified procedure is called prior to an insert operation
on the table. If the procedure returns .F., then the INSERT
is canceled. |
ONOPEN |
The specified procedure is called after an open operation
on the table. |
ONCLOSE |
The specified procedure is called prior to a close operation
on the table. |
ONROLLBACK |
The specified procedure is called when a user presses the
[ABANDON] key in a forms based operation. |
SECURITY
As mentioned above, all Recital files are subject to Operating System read and write permissions. These permissions can be further refined, while still using the Operating System user and group IDs, in the Security and Protection sections of the Dictionary. The Security section handles table based operations and the Protection section focuses on individual fields.
Security and Protection rules can be defined in the CREATE/MODIFY STRUCTURE worksurface of Recital Terminal Developer, via the SQL GRANT and REVOKE statements or in the Recital Enterprise Studio Database Administrator.
Click image to display full size
Fig 5: MODIFY STRUCTURE Worksurface: Security.
The Security section has table operations for which Access Control Strings can be specified. An Access Control String (ACS) is a range of valid user identification codes, and is used to restrict table operations to certain individuals or groups. Each user on the system is allocated a group number and a user number. The user identification code is the combination of group and user numbers. When constructing an Access Control String of linked user identification codes, wild card characters may be used.
Example ACS |
Description |
[1,2] |
In group 1, user 2 |
[100,*] |
In group 100, all users |
[2-7,*] |
In groups 2-7, all users |
[*,100-200] |
In all groups, users 100-200 |
[1,*]&[2-7,1-7] |
In group 1, all users, in groups 2-7, users 1-7 |
Please note that the maximum ACS length is 254 characters. OpenVMS group and user numbers are stored and specified in octal. On other Operating Systems, group and user numbers are stored and specified in decimal.
Access Control Strings may be associated with the following operations:
Operation |
Description |
READONLY |
Users specified in the ACS have read-only access to the table. All other users have update access. |
UPDATE |
Users specified in the ACS have update access to the table. All other users are restricted to read-only access. |
APPEND |
Users specified in the ACS can append records into the table. No other users can append. |
DELETE |
Users specified in the ACS can delete records from the table. No other users can delete. |
COPY |
Users specified in the ACS can copy records from the table. No other users can copy. |
ADMIN |
Users specified in the ACS can use the following commands: |
The corresponding SQL privileges are:
Operation |
Description |
SELECT |
Users specified in the ACS may name any column in a SELECT statement. All other users have update access. |
UPDATE |
Users specified in the ACS may name any column in an UPDATE statement. All other users are restricted to read-only access. |
INSERT |
Users specified in the ACS can INSERT rows into the table. No other users can INSERT. |
DELETE |
Users specified in the ACS can DELETE rows from the table. No other users can DELETE. |
ALTER |
Users specified in the ACS can use the ALTER TABLE statement on this table. |
READONLY |
Users specified in the ACS may read any column in a SELECT statement. All other users have update access. |
// Grant insert privilege for the customer table exec sql OPEN DATABASE southwind; exec sql GRANT UPDATE (lastname, firstname) INSERT ON customers TO '[20,100]'; // Grant all privileges to all users exec sql OPEN DATABASE southwind; exec sql GRANT ALL ON shippers TO PUBLIC;
PROTECTION
Security and Protection rules can be defined in the CREATE/MODIFY STRUCTURE worksurface of Recital Terminal Developer, via the SQL GRANT and REVOKE statements or in the Recital Enterprise Studio Database Administrator.
Click image to display full size
Fig 6: Database Administrator: Protection.
The format of the ACS is the same as in <SECURITY> above.
The following protection can be defined:
Operation |
Description |
READONLY |
Users specified in the ACS have read-only access to the field. All other users have update access. |
UPDATE |
Users specified in the ACS have update access to the field. All other users are restricted to read-only access. |
Recital Terminal Developer also has 'HIDDEN' Protection:
Operation |
Description |
HIDDEN |
Users specified in the ACS see the 'hiddenfield'character rather than the data in the field. All other users see the data. |
Hidden fields can be accessed and viewed on a work surface, but the field contains the hiddenfield character, ?. If the field is referenced in an expression, it will contain the following: blanks for character fields, F for logical fields, 00/00/0000 for date fields and blank for memo fields.
The corresponding SQL privileges are:
Operation |
Description |
SELECT |
Users specified in the ACS may name the column in a SELECT statement. All other users have update access. |
UPDATE |
Users specified in the ACS may name the column in an UPDATE statement. All other users are restricted to read-only access. |
READONLY |
Users specified in the ACS may read the column in a SELECT statement. All other users have update access. |
// Grant update privilege for columns lastname and firstname from the customer table exec sql OPEN DATABASE southwind; exec sql GRANT UPDATE (lastname, firstname) customers TO '[20,100]';
Encryption
From Recital 8.5 onwards, Recital installations that have the additional DES3 license option have the ability to encrypt the data held in Recital database tables. Once a database table has been encrypted, the data cannot be accessed unless the correct three-part encryption key is specified, providing additional security for sensitive data.
ENCRYPT
The ENCRYPT Recital 4GL command is used to encrypt the data in the specified table or tables matching a skeleton. If the skeleton syntax is used, then all matching tables will be given the same encryption key. The encryption key is a three part comma-separated key and may optionally be enclosed in angled brackets. Each part of the key can be a maximum of 8 characters. The key is DES3 encrypted and stored in a .dkf file with the same basename as the table. After encryption, the three parts of the key must be specified correctly before the table can be accessed.
// Encrypt individual tables encrypt customers key "key_1,key_2,key_3" encrypt employees key "<key_1,key_2,key_3>" // Encrypt all .dbf files in the directory encrypt *.dbf key "key_1,key_2,key_3"
SET ENCRYPTION
If a database table is encrypted, the correct three-part encryption key must be specified before the table's data or structure can be accessed. The SET ENCRYPTION TO set command can be used to specify a default encryption key to be used whenever an encrypted table is accessed without the key being specified. The encryption key is a three part comma-separated key.
If the command to access the table includes the key, either by appending it to the table filename specification or using an explicit clause, this will take precedence over the key defined by SET ENCRYPTION TO.
Issuing SET ENCRYPTION TO without a key causes any previous setting to be cleared. The key must then be specified for each individual encrypted table.
The default key defined by SET ENCRYPTION is only active when SET ENCRYPTION is ON. SET ENCRYPTION OFF can be used to temporarily disable the default key. The SET ENCRYPTION ON | OFF setting does not change the default key itself. SET ENCRYPTION is ON by default.
// Encrypt individual tables encrypt customers key "key_1,key_2,key_3" encrypt shippers key "key_2,key_3,key_4" // Specify a default encryption key set encryption to "key_1,key_2,key_3" // Open customers table using the default encryption key use customers // Specify shippers table's encryption key use shippers<key_2,key_3,key_4> // Disable the default encryption key set encryption to // Specify the individual encryption keys use customers encryption "key_1,key_2,key_3" use shippers<key_2,key_3,key_4>
DECRYPT
The DECRYPT command is used to decrypt the data in the specified table or tables matching a skeleton. The specified key must contain the three part comma-separated key used to previously encrypt the table and may optionally be enclosed in angled brackets. The skeleton syntax can only be used if all tables matching the skeletonhave the same key.
The DECRYPT command decrypts the data and removes the table’s .dkf file. After decryption, the key need no longer be specified to gain access to the table.
// Decrypt individual tables decrypt customers key "key_1,key_2,key_3" decrypt employees key "<key_1,key_2,key_3>" // Decrypt all .dbf files in the directory decrypt *.dbf key "key_1,key_2,key_3"
All of the following commands are affected when a table is encrypted:
- APPEND FROM
- COPY FILE
- COPY STRUCTURE
- COPY TO
- DIR
- USE
- SQL INSERT
- SQL SELECT
- SQL UPDATE
APPEND FROM
Used to append records to the active table from another table.// The key must be specified for an encrypted source table
use mycustomers append from customers encryption "key_1,key_2,key_3"; for country = "UK"
COPY FILE
Used to copy a file.// The key file must also be copied for an encrypted source table // as the target table will be encrypted
encrypt customers key "key_1,key_2,key_3" copy file customers.dbf to newcustomers.dbf copy file customers.dkf to newcustomers.dkf use newcustomers encryption "key_1,key_2,key_3"
COPY STRUCTURE
Used to copy a table's structure to a new table.// The key file is automatically copied for an encrypted source table // and the target table encrypted
encrypt customers key "key_1,key_2,key_3"
use customers encryption "key_1,key_2,key_3" copy structure to blankcust use blankcust encryption "key_1,key_2,key_3"
COPY TO
Used to copy a table.// By default, the key file is automatically copied for an encrypted // source table and the target table encrypted with the same key encrypt customers key "key_1,key_2,key_3" use customers encryption "key_1,key_2,key_3" copy to newcustomers use newcustomers encryption "key_1,key_2,key_3" // You can also create a copy with a different key encrypt customers key "key_1,key_2,key_3" use customers encryption "key_1,key_2,key_3" copy to newcustomers encrypt "newkey_1,newkey_2,newkey_3" use newcustomers encryption "newkey_1,newkey_2,newkey_3" // Or create a decrypted copy encrypt customers key "key_1,key_2,key_3" use customers encryption "key_1,key_2,key_3" copy to newcustomers decrypt use newcustomers // You can also create an encrypted copy of a non-encrypted source table use orders copy to encorders encrypt "newkey_1,newkey_2,newkey_3" use encorders encryption "newkey_1,newkey_2,newkey_3"
DIR
Used to display a directory listing of tables.// Encrypted tables are flagged as such with (DES3) > open database southwind > dir
Current database: southwind Tables # Records Last Update Size Dictionary Triggers Security categories.dbf 8 01/10/06 24576 None None None cisamdemo.dbf ---> CISAM/Bridge [cisamdemo] customers.dbf (DES3) 91 05/12/04 49600 None None None employees.dbf 9 05/12/04 25520 None None None example.dbf (DES3) 100 12/24/05 38080 Yes Yes None order_details.dbf 2155 05/12/04 296320 None None None orders.dbf 829 05/12/04 232704 None None None products.dbf 77 05/12/04 37112 None None None productsbyname.dbf 77 05/12/04 29104 None None None shippers.dbf (DES3) 3 05/12/04 20864 None None None suppliers.dbf 29 12/08/05 29992 Yes None None 0.765 MB in 11 files. 1.093 GB remaining on drive.
USE
Used to open a table.// The three part key must be specified to open an // encrypted table. All of the following are valid. // 1. Specifying a default encryption key before opening the table set encryption to "key_1,key_2,key_3" use customers // 2. Appending the key to the filename use customers<key_1,key_2,key_3> // 3. Using the ENCRYPTION clause, optionally specifying angled brackets use customers encryption "key_1,key_2,key_3" use customers encryption "<key_1,key_2,key_3>"
SQL INSERT
Used to add a row to a table via SQL.// The three part key can be specified using a // default encryption key before opening the table exec sql OPEN DATABASE southwind; exec sql SET ENCRYPTION TO "key_1,key_2,key_3"; exec sql INSERT INTO customers (customerid, companyname) VALUES ('RECIT','Recital Corporation'); // Or by appending the key to the filename exec sql OPEN DATABASE southwind; exec sql INSERT INTO customers<key_1,key_2,key_3> (customerid, companyname) VALUES ('RECIT','Recital Corporation');
SQL SELECT
Used to return data from a table via SQL.// The three part key can be specified using a // default encryption key before opening the table exec sql OPEN DATABASE southwind; exec sql SET ENCRYPTION TO "key_1,key_2,key_3"; exec sql SELECT * FROM customers; // Or by appending the key to the filename exec sql OPEN DATABASE southwind; exec sql SELECT * FROM customers<key_1,key_2,key_3>;
SQL UPDATE
Used to update data in a table via SQL.// The three part key can be specified using a // default encryption key before opening the table exec sql OPEN DATABASE southwind; exec sql SET ENCRYPTION TO "key_1,key_2,key_3"; exec sql UPDATE customers SET companyname='Recital Corporation Inc.' WHERE customerid='RECIT'; // Or by appending the key to the filename exec sql OPEN DATABASE southwind; exec sql UPDATE customers<key_1,key_2,key_3> SET companyname='Recital Corporation Inc.' WHERE customerid='RECIT';
Summary
Recital offers a range of ways to keep your data secure. These start with the Operating System read/write permissions, which can be further refined to the level of table I/O operations and then field access in the Dictionary based Security and Protection rules. The Dictionary also provides the means to protect the integrity of the data via data validation and to assist in correct data entry through the use of choicelists, help messages and picture clauses etc. A further role of the Dictionary is in the provision of Table Triggers, which can be used to enable a programmatic response to table operations to add in additional checks or audit trails. For the most sensitive data, DES3 encryption is the ultimate protection: encrypting the physical data on the disk and only permitting table access on the production of the three part encryption key.
It would appear that gigabit LAN is not! In fact it often runs at the same speed as 100Mbps LAN. Let's look at why exactly.
After configuring your network you can use the ifconfig command to see what speeds the LAN is connected. Even though 1000Mbps is reported by the card, the reality is that the overall throughtput may well be ~100Mpbs. You can try copying a large file using scp to demonstrate this.
As it turns out, in order to use a gigabit LAN you need to use CAT6 cables. CAT5 and CAT5E are not good enough. End result, the ethernet cards throttle back the speed to reduce dropped packets and errors.
You can find a good article here titled Squeeze Your Gigabit NIC for Top Performance. After tuning up the TCP parameters i found that it made no dfifference. The principal reasons behind low gigabit ethernet performance can be summed up as follows.
- Need to use CAT6 cables
- Slow Disk speed
- Limitations of the PCI bus which the gigabit ethernet cards use
You can get an idea about the disk speed using the hdparm command:
Display the disk partitions and choose the main linux partition which has the / filesystem.
# fdisk -l
Then get disk cache and disk read statistics:
# hdparm -Tt /dev/sda0
On my desktop system the sata disk perfomance is a limiting factor. These were the results:
/dev/sda1:
Timing cached reads: 9984 MB in 2.00 seconds = 4996.41 MB/sec
Timing buffered disk reads: 84 MB in 3.13 seconds = 58.49 MB/sec
Well, that equates to a raw disk read speed of 58.49 * 8 = 467Mbps which is half the speed of a gigabit LAN.
So.. NAS storage with lots of memory looks to be the way to go... If you use the right cables!
If you have 4 GB or more RAM use the Linux kernel compiled for PAE capable machines. Your machine may not show up total 4GB ram. All you have to do is install PAE kernel package.
This package includes a version of the Linux kernel with support for up to 64GB of high memory. It requires a CPU with Physical Address Extensions (PAE).
The non-PAE kernel can only address up to 4GB of memory. Install the kernel-PAE package if your machine has more than 4GB of memory (>=4GB).
# yum install kernel-PAE
If you want to know how much memory centos is using type this in a terminal:
# cat /proc/meminfo