AvantGo Synchronization with Pocket PC

Michael Jarrett - JudgeBeavis AT hotmail DOT com

April 12th, 2003

Updated client with proxy support, synce support, and some prettification. I'm busy for the next few weeks, so I've made it available as 0.2-pre for people to hack on.

April 1st, 2003

IT WORKS! Apparently the 'implied' result code from the device on 0x02 is important, and should basically be AGCLIENT_CONINUE in all cases except the goodbye command, where it should be AGCLIENT_IDLE. Without that, the malsync code got grumpy and reset the connection.

I've successfully updated my device with today's content, using nothing but the cradle and agsync. I've put my source code up for download here for those who are interested. You may have to synchronize a few times, or even force a complete resync of content from the Avantgo configuration panel on your device. Feel free to play, and let me know if you have success!

March 20th, 2003

Updated page. Added more details on commands acquired from reverse-engineering. The code I acquired seems to be so old that the stream protocol has had significant changes. But it does give me several clues as to what to look for.

At this point, I've figured out all the Avantgo commands except for 0x01 and 0x0D, and have tested their operation.

I've got the prototype running successfully, but I'm not having much success because the server isn't sending proper page updates. I'm still trying to figure out why, and will post my results once it works.


AvantGo.com is a company that specializes in synchronization between servers and handheld devices. They offer a free service to the outside world where one can have a handheld device download webpages from an AvantGo server for later offline viewing. The server does translations in some cases to format images for the device, and in some cases, have deals so that companies can provide specialized content for mobile devices.

Most common mobile devices can have Avantgo software installed. With a network connection, the device can sync directly with Avantgo, using the Mobile Application Link protocol. However, the more common mode of synchronization is for a plugin on a desktop computer to perform the synchronization with AvantGo, acting as a middleman between the device and the Internet.

A project called malsync was released under the Mozilla Public License, which provided the ability for non-Windows users to synchronize. However, it was designed only to support Palm Pilot users, and, as far as I can see, does not have code supporting the synchronizaiton process for Pocket PC devices. The AvantGo code is quite complex, and I may not have found it yet if it's there. Please email me if you make a discovery.

I wish to add the ability for Pocket PC users to synchronize with AvantGo through a desktop. My eventual goal is to contribute what I can to the synce project, without which we wouldn't even be experimenting with this.

Sample Code

agsync is a small app that demonstrates a working synchonization process, relying heavily on malsync, and what parts of the protocol I've reverse-engineered.

This is not intended for end-users! It's source-code only, and could well destroy your device for all I know! It synchronized, for me, in my particular environment, at one particular time. That's not a guarantee it will do the same for you. Experiment at your own risk.


Please note, I'm assuming those reading this are fairly familiar with the way Windows CE synchronizes, and the basic protocols used for it.

From a mile up, the synchronization process is as follows:

  1. Desktop meets a Pocket PC device. If you want details on this, please refer to the synce project.
  2. Desktop opens a NEW RAPI connection to the device. This is because it relies on CeRapiInvoke, which blocks the ability to make RAPI calls.
  3. The first thing the desktop does is use the CeRapiInvoke function in stream mode to malclmgr.dll function _RAPI_HandleStream2. The form of this command is detailed below.
  4. On success of this call, an unknown protocol is used to get a common view between the desktop and the device as to the state of the Avantgo world. This is the main point of analysis.
  5. Using information from the device, the desktop forms an HTTP connection to the specified Avantgo server. It makes an HTTP POST to the server, as specified by the Mobile Application Link Protocol (MAL). This protocol will not be outlined here, since malsync provides a working implementation already.
  6. The MAL server responds with data, in what it says is image/gif format. There's no GIF header. It turns out this is just a way of confusing people for no good reason. The response actually contains a response in the MAL protocol format.
  7. The desktop uses this information, and continues to talk on the link, updating the Avantgo databases as required.
  8. Desktop is done. Connection is closed and promptly RST (how rude!).


Unfornately, not yet implemented by synce. But hopefully soon. It's not critical: we always send exactly the same packet.
62 00 00 00 Small-endian 32-bit integer (int32-s). Specifies that 98 bytes are coming after this.
45 00 00 00 Function code for CeRapiInvoke
00 00 00 00 Unknown
01 00 00 00 Unknown
0d 00 00 00 13 characters (wide) follow, including null
Null terminated wide-character string: malclmgr.dll
01 00 00 00 Unknown
14 00 00 00 20 characters (wide) follow, including null
Null terminated wide-character string: _RAPI_HandleStream2
00 00 00 00 Unknown
01 00 00 00 Unknown

The amount of Unknowns in there is saddening. But I'm hoping *hint hint* someone who actually has a Windows box and the equipment to monitor it can make apps with CeRapiInvoke calls and find out what the rest do.

In any case, we do know that somewhere in there must be the fact that we're talking on a stream socket rather than block mode, and also a "parameter block", which is likely some sort of byte array.

On success, this packet will return 00 00 00 00. From this point onwards, the stream will talk in the Avantgo protocol outlined next. If the request fails, an error code is returned, and the stream aborted with the same error code (see below).

As far as I can tell, the bytes A1 A1 A1 A1 followed by a 32-bit value represents closing the stream. I've seen this when Avantgo closes the stream, when you tell Avantgo something confusing, and when you specify an invalid CeRapiInvoke request. The 32-bit value following seems to be a little-endian 32-bit integer result code (0 being success?). Once this happens, the connection is again in RAPI mode.

Avantgo Stream Protocol

This is the hard part, as it is the main place where behavior deviates from Palm synchronization. It appears to be a simple command-response system. The desktop sends a single byte as a command code, and then parameters as needed. The device responds with a response code, then data if required. There is very minimal state to worry about in the protocol.

I've outlined some of the command codes I've discovered below. It might help to read details on Avantgo's basic datatypes and Avantgo's complex structures, both extracted from malsync code. Thankfully, this protocol seems to borrow some of the same structures, which makes understanding what's happening much easier.

CommandParameters ResponsesDescription
0x00None0x00 - Success Disconnect. You get a single byte response, and eventually the A1 A1 A1 A1 00 00 00 00 that I suspect is a sign that the stream connection has ended with a result code of 0.
0x01 CompactInt (0x02 in traces I've seen) - Purpose Unknown. 0x00 0x02 - Purpose unknown. I have no idea what this is. Possibly a greeting, to trade IDs or something. Regardless of the specified int, you always get back the same result, as far as I can tell, and it seems not to change any state. It seems to be almost like a separator between sections of things. It happens frequently. Perhaps a ping to verify aliveness after a section of unacknowledged commands?
0x02 CompactInt cmd
byte[] cmdData
None, as far as I've seen. MAL code implies that it should return a result code somewhere, but traces and experimentation confirm that nothing comes back. This LOOKS like the command protocol defined in AGProtocol.c/h. The subtypes are specified in great detail there, as well as the format of each individual command. You must have sent 0x07 to open a server before you do this, otherwise, 0x02 will unceremoniously drop your connection. The most common command is 0x10, which is used to add records to the device.
0x03 AGDBConfig record 0x00 result code. Initialize/open a database. The database is automagically closed when a read request finds no more records.
0x04 None0x00 if no record. Otherwise 0x01
If record, AGRecord
Request record from database. Follows a 0x03. Keep doing it until you get a 0x00 back. Don't do this without a 0x03, because it will cause the stream to be aborted with error code 28 04 00 00.
0x05None0x00 Possibly a get-updated-record only thing? As with 0x04, it will abort the stream if you do this without an open database.
0x07CompactInt uid0x00 (result code) I suspect 'Start Server', specifying that we are syncing with a specific server, specified by its uid. This gives me an error if I specify anything except the UID of a server known to the device. If the server is not closed, error code 12 is returned when you end the session. Opening a server multiple times seems not to be a problem.

This conflicts with the old MAL code which indicates that a server and port should be transmitted. Whatever works for you...

0x08None0x00 (result code) Close server? This always responds 0x00 even if nothing's open, and traces indicate that it does happen after a server is processed.
0x09None 0x00 - Success
CompactInt - size of data to follow
Request Avantgo configuration from device. Result is an AGUserConfig, directly from \windows\malconfig.cfg.
0x0A CompactInt - size of data to follow
0x00 - success Push Avantgo configuration to device. This updates the file \windows\malconfig.cfg directly.

Editor's note: Yes, experimenting with this command can be bad! If you mess up (hint: yeah, I messed up once or twice), you can restore the configuration by copying the file "\Documents and Settings\Username\Application Data\Microsoft\ActiveSync\Profiles\devicename\MAL\MALConfig.cfg" on your desktop to ":\windows\malconfig.cfg" on your device.

0x0B None 0x00 - Success
Request information about the device. This includes things like the OS version, device identifiers, etc.
0x0D None 0x00
3xCompactInt. Uses unknown.
Unknown. This is used very early. Nothing seems to really affect it much
Invalid ops NoneThese are SILENTLY IGNORED!

That's not all the commands. These are just the ones I was able to quickly figure out and explain. More will come in time.

Basic Datatypes

boolean 8-bit signed value. 0 == FALSE, > 0 == TRUE, < 0 is invalid.
byte 8-bit unsigned value. Interpretation varies.
int8 Signed 8-bit integer.
int16 Big-endian 16-bit value.
int32 Big-endian 32-bit value.
CompactInt Read byte.
If < 254, value is the byte you just read.
If == 254, read an int16, return that.
IF == 255, read an int32, return that.
Array (note my notation: type[]) Read CompactInt. This is the array length. Read this many following "type" elements.
CString Null-terminated ASCII string.
String byte[] representing an ASCII string

Complex Datatypes

Please note, most of the complex datatypes can be derived from the "Read" functions for them in Malsync, so my summaries will be brief. Also, since most of this is ripped indirectly from malsync, these structures are also released under the Mozilla Public License.

struct AGUserConfigData {
   int16 Signature= 0xDEAA;
   int16 Version= 0x0000;
   CompactInt uid;
   CompactInt flags; // (suspected not used, normally 0x00)
   CompactInt[] uidDeletes;
   AGServerConfig[] servers;
   CompactInt expansion1= 0, expansion2= 0, expansion3= 0, expansion4= 0;
   byte[] reserved= {};

struct AGServerConfig {
    int16 Signature= 0xDEAA;
    int16 Version= 0x0000;
    CompactInt uid;
    CompactInt status;
    CString server;
    CompactInt port;
    CString username;
    CString clearTextPassword; // Thankfully, blank
    // Note: this uses an int8 instead of a compactInt for length
    byte[] nonce;
    boolean disabled;
    CString friendlyName;
    CString serverType; // "AvantGo"
    CString userURL;
    CString description;
    CString serverUri;
    byte[] sequenceCookie; // 5 bytes on my implementation
    AGDBConfig[] databases;
    boolean sendDeviceInfo;
    int8 hashPassword;
    CompactInt connectTimeout;
    CompactInt writeTimeout;
    CompactInt readTimeout
    boolean connectSecurely;
    boolean allowSecureConnection;
    CompactInt flags; // TODO: Discover me!
    CompactInt expansion1= 0, expansion2= 0, expansion3= 0, expansion4= 0;
    byte[] reserved= {};

struct AGDBConfig {
    int16 Signature= 0xD5AA;
    int16 Version= 0x0000;
    CString dbname; /* These can be matched to the subdirectories of the
                       directory specified in
		       HKEY_CURRENT_USER\Software\AvantGo\DatabaseLocation */
    CompactInt type;
    boolean sendRecordPlatformData;
    byte[] platformData;
    int32[] newIds;
    CompactInt expansion1= 0, expansion2= 0, expansion3= 0, expansion4= 0;
    byte[] reserved= {};

struct AGRecord {
    CompactInt uid;
    enum AGRecordStatus status= {UNMODIFIED= 0, UPDATED= 1, NEW= 2,
                                 DELETED= 3, NEW_TEMP= 4};
    byte[] recordData;
    byte[] platformData;

struct AGDeviceInfo {
    int32 availableBytes; // Yup, free storage.
    int32 screenWidth, screenHeight, colorDepth;
    int32 platformDataLen
    Bytes platformData;
    CString osName;
    CString OSVersion;
    CString setLanguage, charset, serialnumber;

Original writing on this page is Copyright (C) 2003 Michael Jarrett. AvantGo is a trademark of its owner. MAL and Avantgo protocols are the intellectual property of their respective owners.