Recital provides a wide variety of connectivity solutions to external data sources. This article provides an overview.
When stress testing our loadbalancer, i was unable to get more than 20 reliable ssh connections. The following error would be reported.
ssh_exchange_identification: Connection closed by remote host
The resolution for this is quite simple.
Edit the /etc/ssh/sshd_config file and increase the MaxStartups. On my server i set this to 200.
Edit /etc/sysctrl.conf and add the following line:
net.core.netdev_max_backlog = 3000
Then apply this change:
# sysctl -p
APPEND FROM <table-name>Before when appending into a shared Recital table each new row was locked along with the table header, then unlocked after it was inserted. This operation has now been enhanced to lock the table once, complete inserting all the rows from the table and then unlock the table. The performance of this operation has been increased by using this method. All the database and table constraints are still enforced.
echo "Hello world\n"
This article looks at After Image Journaling and audit trails in Recital using SET JOURNAL and associated commands.
Overview
After Image Journaling, used in conjunction with a structured backup policy is an effective disaster recovery solution. Any transaction that takes place on a table that is being journaled is logged in the journal file. In the event of a disk crash or similar event in which the table is lost, the journaled transactions can be reapplied to the latest backup copy of the table. Alternatively or additionally, the journal can be used to provide an audit trail to all modifications made to the table data.
NOTE: Recital also provides Before Image Journaling via BEGIN TRANSACTION / END TRANSACTION blocks, allowing unsuccessful transactions to be rolled back to a set saved state.
SET JOURNAL and RECOVER
Regular backups are an essential routine for any system, but in high-transaction environments restoration of the latest backup can still mean a major loss of data. After image journaling can successfully be used as part of your disaster recovery strategy to minimize data loss and down time. Recital after image journaling functionality is based on the use of the SET JOURNAL and RECOVER commands.
SET JOURNAL
SET JOURNAL TO [<.dbj filename> | ()] SET JOURNAL ON | OFF | ( )
The SET JOURNAL command is used to enable the After Image Journaling and audit trail for the active table. The TO <.dbj filename> clause associates the specified transaction journal file with the active table. If the journal file does not exist, it will be created. The filename can be substituted with a <expC>, enclosed in round brackets, which returns a valid filename. If no file extension is specified, ‘.dbj’ is used. When specifying a journal file, it is recommended that the journal file is stored on a different disk than that which the table is stored on, so that if a fatal disk error occurs, then the journal file will not be lost along with the table.
//Enable journaling for the southwind!orders table open database southwind use orders set journal to /journals/ord_journ
The <.dbj filename> is a standard table. It contains seven
fields that are specific to a journal file, followed by the first
249 fields of the associated table.
The first seven fields in the journal are:
|
Field |
Type |
Display |
Storage |
Description |
|
AUD_DATE |
Date |
8 | 10 * |
4 |
The date on which the transaction was performed. |
|
AUD_TIME |
Character |
8 |
8 |
The time at which the transaction was performed, in the format HH:MM:SS. |
|
AUD_TERM |
Character |
12 |
12 |
The name of the terminal from which the transaction was performed |
|
AUD_UID |
Short |
5 |
2 |
The ID of the user who performed the transaction. |
|
AUD_GID |
Short |
5 |
2 |
The group ID of the user who performed the transaction. |
|
AUD_CMD |
Short |
4 |
2 |
The command number of the transaction performed from the command table below |
|
AUD_RECNO |
Integer |
7 |
4 |
The record number in the associated table which the transaction was performed on. |
* Dependent on SET CENTURY setting.
The AUD_CMD Command Reference Numbers are as follows:
|
Command |
Number |
|
DELETE |
14 |
|
RECALL |
36 |
|
REPLACE |
41 |
|
BROWSE |
6 |
|
CHANGE |
8 |
|
EDIT |
17 |
|
INSERT |
26 |
|
APPEND |
5 |
|
READ |
35 |
Since journal files are standard Recital tables, you can use standard Recital commands such as the REPORT command to print audit trails, transaction logs, etc.
//Enable journaling for the southwind!orders table open database southwind use orders set journal to /journals/ord_journ //.. transactions close data //View journaled records use /journals/ord_journ.dbj
Click image to display full size
Fig 1: Journal Record Example.
The SET JOURNAL TO command without a <.dbj filename> specified closes the active journal file and no further journaling will take place on the active table until the SET JOURNAL TO <.dbj filename> is reissued.
The journaling features are mainly used with shared tables. It should be noted that there is an overhead in enabling transaction journaling, as records updated in a table are also written to the journal file. When records are appended into a journal file, locking is automatically performed so that multiple users can update the journal concurrently. The associated table must be opened shareable for this to occur. Each table can have a journal file associated with it.
The SET JOURNAL ON | OFF command enables or disables transaction journaling. This command is primarily used in applications where journaling can be disabled for a certain class of operations. By default, SET JOURNAL is ON, but no journal files are set.
NOTE: Only the first 249 fields of a table can be journaled: subsequent fields are ignored. The maximum number of fields in a Recital table is 256.
RECOVER
RECOVER FROM <.dbj filename> | ()
The RECOVER command uses the journal file to reapply lost transactions to a previous backup of the data after a fatal error such as a disk head crash. The FROM clause specifies the journal file to use. The file name can be substituted with an <expC>, enclosed in round brackets, which returns a valid filename. If no file extension is specified, then ‘.dbj’ is assumed.
Regular backups are essential to the successful use of After Image Journaling. It is also very important to reinitialize the journal file after each backup: either open the journal file as you would a normal table and use the ZAP command, or delete the file completely. If a fatal error occurs, such as a disk head crash, the table and index files must be restored from a backup, then the RECOVER command executed. RECOVER will reapply' all of the transactions in the journal file to the table, and update the indexes. After the RECOVER command has completed, you can continue with normal processing.
//Create a backup of the southwind!orders table //...backup table and associated files //Reinitialize the journal file erase /journals/ord_journ.dbj //Enable journaling for the southwind!orders table open database southwind use orders set journal to /journals/ord_journ //.. transactions //Restore the backup of the southwind!orders table //...restore //Open the restored backup open database southwind use orders //Reapply the transactions using the journal recover from /journals/ord_journ.dbj //Now, enable the journal file again or //restart with a new backup
Journaling Memo Fields
By default, memo fields - variable length text fields - are not journaled due to the possible storage overhead of multiple copies of potentially large blocks of text. But, if memo journaling is required, the SET MEMOJOURNAL ON command can be used to enable this.
SET MEMOJOURNAL
SET MEMOJOURNAL ON | OFF | ()
The SET MEMOJOURNAL command causes memo fields to be journaled when journaling is set on a table. This command allows the optional logical expression <expL> to be evaluated. If a value of .T. is returned, MEMOJOURNAL is set ON. If a value of .F. is returned, MEMOJOURNAL is set OFF. By default SET MEMOJOURNAL is OFF.
Like a normal Recital table, the journal holds only a pointer to a data block in an associated memo file, not the actual memo data itself. The journal's memo file has a file extension of .dbm rather than the standard Recital .dbt. Therefore, if the journal is being opened as a table, in order to view the journal's memo data, the SET MEMOEXT command should be used.
//Enable journaling for the southwind!suppliers table open database southwind use suppliers set journal to /journals/sup_journ //.. transactions close data //Set filename extension for memo file set memoext to '.dbm' //View journaled records use /journals/sup_journ.dbj
Summary
The After Image Journaling enabled by the SET JOURNAL and RECOVER commands can be used in conjunction with a strict backup regime to minimize data loss in cases where tables become damaged or irretrievable. Journal files can be accessed like standard Recital tables and provide detailed information about the transactions applied to a table, so can be used for auditing purposes.
When you start the loadbalancer.org appliance you will see the following:
Default login:
Username: root
Password: loadbalancer
Access to webclient from an external client is:
http://192.168.1.129:9080
http://192.168.1.129:9443
You can access the web administrator using the IP and ports described onscreen.
For the sri lanka porject we are looking for performance and the network diagram indicates we are happy to have the cluster on the same subnet as the rest of the network.
Direct routing is the fasted performance possible, it has the advantage over NAT that the Loadbalancer does not become a bottleneck for incoming and outgoing packets. With DR the loadbalancer simply examines incoming packets and the servers to route the packets directly back to the requesting user.
The web interfaceis the only way to fully configure the loadbalancer vm. The console tool lbwizard will get it initiallised and any further configurations can then be done via the webinterface.
Using lbwizard for the Sri lanka configuration follow these steps.
On the first Loadbalancer:
//Start
Is this unit part for a HA Pair?
YES
Have you already setup the Slave?
NO
Is this a one-armed configuration?
YES
Enter the IP Address for the interface eth0?
Enter IP address you wish to be assigned to the SLAVE loadbalancer.
Enter the netmask for interface eth0?
Enter netmask for the subnet.
Enter the Floating IP adrress?
Enter the IP address that will be IP assosiacted the the HA-pair of loadbalancers.
//Finish
On the 2nd loadbalancer VM, run the lbwizard.
//Start
Is this unit part of an HA-Pair?
YES
Have you already set up the Slave?
YES
What is the slave units UP address?
Enter the IP which you entered when configuring the other loadbalancer VM.
Is this a one-armed configuration?
YES
Enter the IP Address for the interface eth0?
Enter the IP that will be assigned to the MASTER loadbalancer
Enter the netmask for interface eth0?
Enter the subnet netmask.
Enter the Floating IP address?
Enter the IP address that will be IP assosiacted the the HA-pair of loadbalancers.
Enter the address of the default gateway?
Enter the deafult gateway for the subnet.
Enter the IP of the nameserver?
Enter the dns server.
Enter the port for the first Virtual server?
Enter 22 for ssh
Enter the IP address of the first real server?
Enter the real IP of the first appserver
//Finish
Now this is complete we need to go to the web admin interface to configure the 2nd Real Server. As the lbwizard program will only allow you to configure 1 real server.
Now login to the web admin using the default password:
username: loadbalancer
password: loadbalancer
Note: Connect to the IP you have now set for your master loadbalancer
Goto the edit configuration tab
Now click add a real server:
Enter a label
IP address of the server plus the port of the service i.e. 192.168.1.125:22
Edit Configuration -> Virtual Servers
persistancte -> NO
Scheduler-> LC
LC - Least-Connection: assign more jobs to real servers with
fewer active jobs.
Service to check -> custom1
Check port -> 22
Forwarding Method -> DR
Feedback Method -> Agent
Arp Problem when using DR
Every real server must be configured to respond to the VIP address as well as the RIP
address.
You can use iptables (netfilter) on the real server to re-direct incoming packets destined for the virtual
server IP address.
This is a simple case of adding the following command to your start up script (rc.local):
//replace 10.0.0.21 with the Virtual Server IP
iptables -t nat -A PREROUTING -p tcp -d 10.0.0.21 -j REDIRECT
chkconfig iptables on
Latest Development News
The Lianja Application Platform is a cost-effective cloud database computing platform for SMEs (Small and Medium-sized Enterprises) that lets them focus on developing and deploying business Apps without the need to invest in lengthy application development times and an expensive IT infrastructure.The three pillars of Lianja are:
- The Lianja App Builder
- The Lianja Cloud Database
- Lianja.com Apps
The best way to learn Recital is to build some applications. The developers of Recital have written a book "Recital Essentials" which you can read here.
If you have 4 GB or more RAM use the Linux kernel compiled for PAE capable machines. Your machine may not show up total 4GB ram. All you have to do is install PAE kernel package.
This package includes a version of the Linux kernel with support for up to 64GB of high memory. It requires a CPU with Physical Address Extensions (PAE).
The non-PAE kernel can only address up to 4GB of memory. Install the kernel-PAE package if your machine has more than 4GB of memory (>=4GB).
# yum install kernel-PAE
If you want to know how much memory centos is using type this in a terminal:
# cat /proc/meminfo