Data transfer over slow, unreliable or Partly disconnected networks

August 30, 2006

Thankfully, we need to face slow and unreliable networks lesser and lesser however with mobility of office workers increasing, the partly disconnected applications appear to be becoming much more mainstream.

The techniques I have seen being used for communication across slow networks are:

RSync: Rsync provides file system synchronization in a highly efficient manner. It detects the differences and sends only the Diff, that to gzipped across the network. I am not sure whether any other method optimizes the network usage so much.

I have also seen companies use Robocopy or some ftp clients which support restart for the same purposes as well, however they are a less optimized than RSync.

DB replication: Products like Sybase SQL Anywhere ( traditionally) or MS SQL Anywhere (more recently) provide two way replication with conflict resolution capabilities between client and servers. This enables having lightweight database at the client which eventually gets synchronized with the servers. The traditional databases also can replicate across internet and slow networks using custom replication scripts. In all cases, for a two way replication ALL tables must have primary key.

Message Queue: Most message queue offer guaranteed delivery between client to the server and also offer replicated queue options so that the messages are never lost.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: