AGNMERGE V1.4.0

Introduction

This program will merge a number of Psion 3a agenda files into a single agenda file.   During the merge process it will optionally delete memos or alarms, delete crossed out entries, or uncross crossed out entrues, change the display occurrence or repeating entries, omit entries that are outside a date range, prefix all or part of the agenda file name to each entries from that agenda,  change the style of entries, filter entries by year code, omit entries from the year view by year code, change the visability of to-do entries in the general views, or remove duplicate entries have specified what constitutes a duplciate entry.  All these options can be saved to a configuration file for repeated use.


Installation

AGNMERGE is an application consisting of two OPA file that should both be placed in the \APP directory. AGNMERGE saves configuration files in directory \APP\AGNMERGE.  This directory must be created before the program can be run.  The icon for AGNMERGE should be added to the system screen using the install option (Psion + I).

Only the OPA files are required for execution.  The other files are all available for development.


Background

The program was written to solve the following problem.  I have an agenda file for work only, an another for home only.  I have other agendas which contains details of the cycling club with which  I ride, and another of sunrise/sunset times.  To put all these together in a single agenda produces a file so big that I would be very slow to work with.  It would sometimes present too much information.  The largest of my agenda files is about 20k.  This is an acceptable size to work with.  However, when setting an appointment you cannot see potential clashes between the agendas.  I started manually merging them, but the file was too big to be manageable, and alarms would be duplicated or would still go off unless deleted in two places.  AGNMERGE will optionally strip out memos and alarms, and will merge within a date range.  Currently I merge for a whole year and the combined agenda is 32k.

Much of the work done by AGNMERGE is concerned with minimising the size of the output agenda.  All deleted record are removed.  The output agenda will not be further compressed by the agenda compress utility.

AGNMERGE could also be used to shared agenda files between people.  The name of the source agenda file can be placed at the front of every entry in the combined agenda.

The program will not work will Series 3 agenda files.

The program has been tested on the Sienna emulator and appears to work.  There is a minor difference between the 3a and 3c agenda formats.  This program is not designed for the 3c format.

AGNMERGE is a complicated process that can take several minutes to run on a Psion 3a.  It was designed to minimize the size of the output file.  This is achieved at the expense of performance.  Tests have show that it runs up to 16 times faster on the Psion 3a emulator for PC (on a 486 DX2 at 66MHz).  If you are sharing agendas between Psions then it might be easier to copy the files to a PC and use the Psion emulator to run the merge process.

If an agenda file is already opened by another process, then this will cause an error during the merge.  All agenda processes involved must be closed during merge.  Also AGNMERGE uses a large quantity of memory.  In my own case 140K is consumed by the program itself and extra system overhead.  My personal technique is to run AGNMERGE at the same time as I backup the Psion (several times a week).  I stop all the processes with JBSTOPGO, run AGNMERGE, perform the backup, and restart the processes with JBSTOPGO.  JBSTOPGO is available as freeware from all the usual places.


Menu Driven

AGNMERGE, like many Psion application, is menu driven.  On startup the parameter screen is shown.  Press the Psion key to see the menu.  Use the cursor keys to navigate around the menu.  Each menu option shows the key combination equivalent.


Control Files

This menu item holds options to load and save configuration files.  These files record the combination of agenda files and how they are to be combined.


New (Psion + N)

This option clears the current configuration and enables the user to set up a configuration from scratch.


Load (Psion + L)

This will load a previously saved configuration.  The configuration files are shown on the system window below the icon before the program is executed.  AGNMERGE can automatically load a configuration file from the main system window.


Save (Psion + S)

This will save a configuration to disk.  You can choose the name of the file.  By default it will be written to \APP\AGNMERGE.  If it has been previously saved the configuration will be written to the same file.


Save As (Psion + A)

This option will save the current configuration to a different file name.


Agenda Files

This section deals with specifying which agenda files will be combined, and which agenda file they will be written to.


Add Input (Psion + I)

This option adds an agenda file to the list of agenda files to be merged.  Up to seven agendas can be merged.  From version 1.2.0 the same file may be specified for input more than once.  This allows different parts to be filtered, and have different attributes applied from the same agenda file.

From Version 1.4 and duplicates that would appear in the output agenda can be removed.


Remove Input (Psion + R)

This option removes an agenda file to the list of agenda files to be merged.


Set Preferences (Psion + F)

This option specifies which of the input agendas will be used to supply the agenda preferences.


Set Output (Psion + O)

This option specifies the name of the agenda file that will be written.  This must not be the same as any of the input agendas otherwise a file error will result.

This dialog will also allow you to set the read only and modified attributes of the output agenda.

By default the output agenda file is made read only to prevent accidental updating of the wrong file.  If the previously generated output agenda file is read only it will be made writable at the beginning of this process.

The modified attribute is used by some backup utilities to determine which files to backup.  By default the modified attribute is set.  Turning it off can prevent unnecessary backup of a generated file.


Style

These options allow a style to be applied or removed for each entry.  It is possible to specify, for each input agenda, whether each of the three styles (bold, underline and italic) will be applied, removed, or left on each entry on all the entries for that agenda.

For instance you might want all the entries from a particular agenda to be italic but not bold.  By default existing styles are retained.

Bold (Psion + B)
Italic (Psion + I)
Underline (Psion + U)


Filtering

This menu item contains options to exclude or include only those entries that pass a fitlering condition.  All the filters may be used in combination.  Just like optical filters the effect is cumulative.  There is in effect a logical AND between each filter condition.

Each agenda file has its own set of filters.  The is in effect a logical OR between the agenda files.  In order to achieve more complex filtering, or multiple conditions for a single filter, merge the same input agenda more than once with different filters on each pass.


Date  (Psion + D)

This option will filter entries by date.  Undated to-do entries are assumed to be dated with today's date.  The options for the filter are the whole agenda, ie. not filtering, from today, until today,  this calendar year, this calendar month, one, two, three, six months or a year from today, or a fixed user defined period.  The date range for the agenda is 1st January 1980 to 31st December 2049.

Repeated entries receive special handling if date filtering is enabled.  If the end of the repeat period is before the beginning of the date filter period, or the start of the repeat period is after the end of the filter period, then the entry is completely omitted.  Otherwise, if the end of the repeat is after the end of the date filter period it is adjusted to be no later than the end of the date filter period.  If the start of repeat is before the date filter period, then first repeat on of after the date filter period is calculated.  If this is after the end of the repeat period or the date filter period, then the entry is omitted from the merge.


Year Code  (Psion+Y)

This feature is available from Version 1.1.  Entries that appear in the year view have a single character identifier.  It is possible, for each agenda being merged to decide how to process that entry acording to their code letter.  Entries that do not appear in the year view are deemed to have a space as their code letter.  If enabled a list of up to 16 code letters can be specified.  This processing can be either disabled, or enabled in one of three ways.

Either entries are included in the output agenda if their letter appears in the list, and all other entries are omitted,

or entries are excluded from the output agenda if their letter appears in the list, all other entries are included,

or entries whose letter appears in the list are omitted from the year view and their code letter is set to a space.


Text (Psion + Z)

This feature is available from Version 1.2.0.  It is possible to, for each input agenda, to specify a search string and then merge only those entries in an agenda which either do or do not contain that search string in the title text.

A search string can be up to 16 characters.  It is case sensitive.  This processing can be either disabled, or enabled in one or two ways.

Either entries are included in the output agenda if their title text matches the search string, and all other entries are omitted,

or entries are excluded from the output agenda if their title text matches the seatch string, all other entries are included,


Crossed Out (Psion + C)

This options specifies whether crossed out entries are copied over to the output agenda.  By default crossed out entries are deleted.

From V1.4 it is also possible to uncross crossed out entries and then include them in the output agenda.


Entry Type (Psion + K)

This feature is available from Version 1.2.  This option enables the user to specify which types of agenda entry from each agenda will be merged, or discarded.  The entry types are untimed, timed, anniversary and to-do.  By default all types are included.


To-dos in General Views (Psion+G)

This feature is available from Version 1.1.  To-do entries may not visible in the general views because their priority is below the value set for the to-do list.  This facility can change the priority value in the to-do list definition.

For each agenda file being merged the lowest display priority can be raised to minimum or lowered to a maximum value.  Either to-do items can be revealed down to a given priority, or hidden below a given priority.

Hiding to-do items below priority 0 effectively hides all to-do items.  Revealing all to-do items down to priority 0 has no affect, and so is the default.  Revealing all to-do items down to priority 9 reveals everything.

These settings apply to all the to-do lists within a given agenda.



Remove Duplicates (Psion+Q)

This feature is new in Version 1.4.  Duplicate agenda entries can optionally be removed from the final output agenda if the entry about to added already exists in the output agenda. 

Duplicate matching works by building an index on the generated agenda as it is built.  The index file is created in the same directory as the output agenda, it is given the same name but a extention of IDX.  If is deleted when the merge process completes successfully.  The overhead of building and scanning the index file will significantly slow the program.

If any of the matching criteria are enabled then duplicate matching will be performed.  Only if none of them are selected will duplicate matching be disabled.  There is a significant performance overhead incurred by enabled duplicate matching.

Tip: It is possible to use AGNMERGE to remove duplicates from a single agenda file.

The user can choose what attributes of an entry have to match in order that the entry is considered to be a duplicate.

Most options are set simply to yes or no, to enable or disable them as a part of the matching criteria.

On the first screen there are seven options

Entry Text: there are three options for 'Entry Text' matching

Disabled: the program does not search for duplicate entries.

Exact Text Match:  The text in the entry must be exactly the same in order for the record to be considered to be a duplicate.

Ignore Name Prepend:  The prepended agenda name, if any, is ignored during text matching.

Case Sensitive: if entry text matching is not selected it is set to 'n/a'.  Otherwise it can be set to yes or no.

Date/Display From:  This is the date of the entry, of the 'display from' date of a to do entry.

Time/Slot: This is the time of a timed entry, or the time of the slot in which it appears.

Duration/Base Year/Due Date: This is the duration of a timed entry, or the base year of an anniversary entry, or the due date of a to-do entry.  This entry has no meaning for an untimed entry.

Record Type: Distinguishes between timed or untimed day entries, to-dos, or anniversaries.

Attributes:  The are six further attributes that can be enabled which are on a second screen

The options on the second screen are

Repeated/Once Only: Distingusihes between repeated and single entries.

Pending/Crossed Out: Distingusihes between crossed out and not crossed out entries.

Display Code:  Distinguishes between entries that do or do not have year codes, and if they do have year code it distingusihes between different year codes.

Alarm Set/Not Set:  Distinguishes between entries that do or do not have an alarm, but not between the alarm settings.

Memo/No Memo:  Distinguishes between entries that do ro do not have a memo attached.

Entry Style:  Distinguishes between the style of entries: bold, underline, or italic



Configuration

This menu item contains options to control what parts of the agenda record is copied, and how it will be presented.  All options are specified for each input agenda file.


Memo (Psion + E)

This option specifies whether memos are copied over with the entry to which they are attached, or deleted from it.


Alarm (Psion + A)

This option specifies whether alarms are copied over with the entry to which they apply, or deleted from the entry.  By default alarms are deleted.

Tip: Delete the alarms from agenda which are usually kept open otherwise alarms will be doubled up.


Prefix agenda name (Psion + P)

In order to determine from which input agenda an entry in the output agenda has come it is possible to prefix the entry with the name of the input agenda.  This option will allow any number of letters, from none to all, of the input agenda name to be prefixed to the original entry.  The default is for all of the agenda name to be used.


Repeat occurrence show (Psion + W)

This option controls how repeated entries will be displayed in the output agenda.  The repeat occurrence can be left unchanged, which is the default, or all repeats can be set to either next only or all.


Special

This menu item contains three options


Merge (Psion + M)

This option runs the loaded merge process as defined by the other options.  While it runs there are two bar graphs displayed.  This give an idea of how far the process has got.  Each graph is labelled with the agenda file to which they relate.

The lower graph is for the output agenda file.

The upper graph is for the input agenda file.   It is redrawn for each input agenda.  Each input agenda is parsed twice.  During the first parse nothing is written to the output agenda, this will be reflected in the lower graph.  The lower graph will not move until the upper graph reaches the halfway  point.


Information (Psion + I)

This displays a simple version and copyright screen.


Exit (Psion + X)

This option terminates the application.  It does not check to that the configuration requires to be saved.


Change Control

V0.1.0	28/09/96	first stable version
V0.2.0 	28/12/96	date filtering added
V0.2.1	01/01/97	days by month repeat calculation correction
V1.0.0	20/01/97	first release version
V1.0.1	23/01/97	bug fix to to-do warning processing
V1.1.0	05/02/97	added year code filtering and to-do visibility control
V1.1.1	24/04/97	bug fix to record address offset processing
V1.2.0	03/06/97	text filtering added, menu structure reorganised
V1.2.1	12/07/97	bug fix to record type filtering parameter
V1.2.2	02/10/97	added 1 year to date filtering
V1.3.0	31/10/97	year code filter made case sensitive, split into two
				executables.  Source code shipped.
V1.4.0	18/11/97	duplicate filtering, uncrossing crossed out entries

Copyright

AGNMERGE is the intellectual property of David Kurtz (david@kurzbyrn.demon.co.uk, david_kurtz@peoplesoft.com).

I have decided to release it as free software because  I do not suppose that there are many people who would find so useful that they would part with money.  The effort adding nag messages and registration codes and in administering a registration scheme would not be worth the expected income.  Lastly, it might cause contractual problems with my employer.

If you do pass AGNMERGE on then please pass on all of the files!  if you make changes, please tell me about them.


Caveat

While I have used my best endevors to develop this software,  I will not be responsible for an damage or loss of data or anything else.  You use this software at your own risk.  However, so do I!


Feedback

If you find any problems, or wish to make an suggestions for enhancements then please mail me.  Please also let me know if you wish to be kept informed of any upgrades.  My e-mail address is:

david@kurzbyrn.demon.co.uk

This and other pieces of software are available from my website

http://www.kurzbyrn.demon.co.uk/psion.htm


