###############################################################
#                                                             #
#   PHP Groupware Synch Engine - Technical Design Document    #
#                                                             #
###############################################################

###############################################################
##
## Record layout
##
###############################################################

## Database tables
The SyncML application will (probably) need two databasetables to support 
synchronization with devices. The first table, phpgw_syncml_device, contains
the  device names, the date and time they were last synchronized, the date
and time of the current synchronization (next) and a cache of the device's
capabilities:
	phpgw_syncml_device
	device_id, 
	name (...),
	owner (int 11),
	server_last_sync (int 11),
	client_last_sync (...),
	capabilities 

The second table, phpgw_syncml_guidmap, maps the PHPGroupWare record to a
new GUID.  
	phpgw_syncml_guidmap
	  guid (...)
	  appname (calendar/address...),
	  record id (...)
	  status (soft/hard deleted)

The third table, phpgw_syncml_luidmap, maps the PHPGroupWare GUID (Global
Unique  Identifier) to the LUID (Local Unique Identifier) on the device. This table
also  contains the status for each GUID, which can be active, soft deleted or hard 
deleted:
	phpgw_syncml_luidmap
	  device (...),
	  guid (...),
	  luid (...)

## vCard Record Translation
PHP GroupWare already contains support for importing and exporting vCards in 
phpgroupware/addressbook/inc/class.vcard.inc.php. The SyncML protocol cannot
use the PHPGW vCard class, because the final vCard format that is sent to a
client depends on the capabilities exchange. The PHPGW server should only send
those fields to the client that the client supports when synchronizing.
iCal Record Translation 

## iCal Record Translation
The iCalender RFC2445 still needs to be compared to the PHP GroupWare data 
formats. Then, a class for importing and exporting iCal objects should be 
written, similar to the vCard class. Again, this class will need to be able
to export iCals based on the clients abilities.

###############################################################
##
## Synchronization
##
###############################################################

## Overview
A synchronization session starts by a request. This request will be directed 
towards the SyncML PHP Wrapper ( 
http://www.domain.com/phpgroupware/syncml/syncml.php). The PHP Wrapper will
call the SyncML Toolkit, create a new SyncML session and send the incoming 
synchronization message to the SyncML session for parsing. The SyncML
toolkit will call the PHP Wrapper when it finds certain SyncML commands (start
message, start sync, copy, update, end sync, end message). These commands will be 
forwarded to the PHPGW Sync Engine. The commands will then be parsed by the
Sync Engine. The Sync Engine will then compose a response message.
Composing a response message 

After receiving the EndMessage command, the PHPGW Sync Engine will create a 
response message to the synchronizing device using the SyncML Toolkit. This 
message will be sent back to the device by the PHP Wrapper.

## Receiving a message
  -> SyncML->SyncML (encoding (SML_XML/SML_WBXML), wssize, wsname, [array handlers])
  -> SyncML->write_in_buffer()
  -> SyncML->process_data()
        -> StartMessage()
		Before doing any parsing, we will expect a startmessage call.
		We do not need to perform any actions now, so we can ignore
		this call :-)
        -> StartSync()
        -> StartAtomic()
		If we receive a StartAtomic, we will set $syncml->atomic to 1.
		All commands from now on will only be added to one large
		series of SQL statements.
        -> EndAtomic()
		We unset $syncml->atomic.  
		For Postgres we can use Transaction support to ensure that
		we run all transactions at once.  At the end of the
		transaction we will use either COMMIT if all SQL statements
		were successful, or ROLLBACK if they were not.
		Since MySQL does not support transactions, we can never do a 
		rollback.  All we can do to ensure some level of atomicity
		is to lock the tables before a transaction.  During the
		lock, other applications will not be able to write to the
		table.  This is probably not a problem, since it is very 
		unlikely that a client will request atomic transactions.
        -> StartSequence()
		If we receive a StartSequence, we will set $syncml->sequence
		to 1. All commands from now on will only be added to one large
		series of SQL statements.
        -> EndSequence()
		Unset $syncml->sequence.
		A sequence only requires that individual commands are
		processed in the order they are received.  Dunno what we
		should do with alerts that are received in the meantime!!!
        -> Alert()
        -> AddCmd()
		Add a new record to the database.
        -> DeleteCmd()
		See what record they're talking about and delete it!
        -> GetCmd()
		Retrieve the requested item from the database and add it to
		the response message we're trying to create.
        -> PutCmd()
        -> MapCmd()
        -> ResultsCmd()
        -> StatusCmd()
        -> ReplaceCmd()
        -> CopyCmd()
        -> ExecCmd()
        -> SearchCmd()
        -> EndSync()
        -> EndMessage()
		Check $syncml->atomic and $syncml->sequence!!!!
		The client is done.  We can now start creating our response
		message, see below....

## Sending a response
  -> SyncML->start_message
        -> SyncML->sml_start_sync()
        -> SyncML->sml_end_sync()
        -> SyncML->sml_StartAtomic()
        -> SyncML->sml_EndAtomic()
        -> SyncML->sml_StartSequence()
        -> SyncML->sml_EndSequence()
        -> SyncML->sml_add_cmd()
        -> SyncML->sml_AlertCmd()
        -> SyncML->sml_CopyCmd()
        -> SyncML->sml_DeleteCmd()
        -> SyncML->sml_ExecCmd()
        -> SyncML->sml_GetCmd()
        -> SyncML->sml_PutCmd()
        -> SyncML->sml_MapCmd()
        -> SyncML->sml_ResultsCmd()
        -> SyncML->sml_SearchCmd()
        -> SyncML->sml_StatusCmd()
        -> SyncML->sml_ReplaceCmd()
  -> SyncML->end_message

###############################################################
##
## Changes to PHP Groupware
##
###############################################################
Several changes are necessary to PHP Groupware in order to adhere to the
SyncML  standards.

## Last Modification Date
For each record that may be synchronized to a remote device, the
synchronization  module needs to know the last modification date and time 
of the record. If this  data is not known, all data has to be synchronized
on every synchronization. Implementing this, would mean adding an additional 
field (last_mod) to the following tables:
	phpgw_addressbook
	phpgw_cal
	phpgw_categories
	phpgw_notes
	phpgw_todo

## Problem: GUID
The SyncML Specification requires each record to have it's own, global
unique  id, the GUID. This GUID will be mapped to the LUID (the local unique
id on the  device) using the phpgw_syncml_map table.  However, each
application in PHPGroupWare uses it's own unique ideas. The GUID  therefore
has to be assembled from the type of record and the unique id for that  type
of record. A calendar entry guid would then look like this: c253

The following letters will be used for assigning GUID's to records:
	a addressbook record
	c calendar record
	n note record
	t todo record
In the future, it would be wise to design a function that would provide
32-bit  GUID's for each new record, instead of leaving it up to the database
or the  specific application to define such an id.

## Problem: Categories
Currently, all categories in PHPGroupWare are global. Because of the growing 
number of applications, this may prove to be a problem for people with large 
amounts of data in the future. The Palm, for example, has different categories
for each application.  Synchronizing it to PHPGroupWare would, in the current
situation, generate a lot  of global categories, since each local category on
the Palm would be translated to a global category on PHP Groupware.

Two solutions exist to this problem:
1. Rewriting all applications to allow both local and global categories.
2. Going from global to local categories for PHP Groupware. Both solutions
   have their pro's and con's and deciding which way to go is up to 
   the PHPGroupWare Project Management.

## Problem: vCard Binary records
The vCard specifications allows for binary files to be sent, for example for
the  logo, the photo or a sound. The current phpgw_addressbook does not
contain a  record for photo's or logo's.  The sound record does exist, but it
is a varchar field. In the vCard  specification, the sound field can be either
a URI or a binary. The preferred  type, binary, cannot be stored in the
phpgw_addressbook table. For now, sound  records will be dismissed when
synchronizing. In the future, binary records (photo, sound, logo, etc.)
should be stored either in the database or in the filesystem. 

## Problem: vCard Record Translation
The SyncML protocol cannot use the PHPGW vCard class, because the final vCard 
format that is sent to a client depends on the capabilities exchange. The PHPGW 
server should only send those fields to the client that the client supports.

###############################################################
##
## Future possibilities
##
###############################################################
Currently, several ideas exist that could be implemented in future
applications for synching PHPGroupWare to remote devices, such as PDA's
and mobile phones. 

## Notes synchronization
The first version of the Sync Engine will only support iCalender (Calendar
and  Todo) and vCard (Addressbook) synchronization. Most devices are also
capable of  storing notes. However, since the format in which these notes
are synchronized is unknown, we will have to wait on this. 

## SyncML Toolkit in PHP
The SyncML Toolkit as delivered by the SycnML Consortium is currently used
for  the major part of the SyncML functionality. In the future, it is
probably wise  to rewrite the toolkit in PHP.
Rewriting the toolkit in PHP will ensure cross-platform functionality for
the  PHPGroupWare sync engine. The current toolkit implementation as
delivered by the  SyncML Consortium, only works on either Linux, Windows
or Palm platforms.

## Expense application
No PHPGroupWare expense application exists at the moment. On a lot of mobile 
platforms, an expense application does exist. Being able to sync such an 
application to PHPGroupWare would provide large benefits to the user
community.  Being web based, PHPGroupWare can add major functionalities to
an expense  application, such as daily, weekly, monthly and yearly graphs,
category-based graphs, weekly or monthly comparison, and so on.

## E-mail synchronization
Most mobile platforms such as mobile phones and PDA's have the ability to
send  or receive e-mail. On devices such as Palm's, this e-mail can be read
or written  online. Allowing synchronization between PHPGW and such devices
might prove to be a great benefit.
