Storage-Message roaming
Goal
Whatever device a user logs in, he could access past messaging history.
For example, Telegram/QQ could support it but WeChat does not support it.
Differences between message roaming and offline message pulling
Offline msgs should not be stored together with normal online msgs because
Offline msgs will contain operation instructions which will not be persisted in online cases. For example, client A deletes a message in device A. When syncing from Client A's device B, the message should still be deleted.
Offline message pulling
Trigger
Whenever a user opens app, the app will pull the offline messags. After network jittery, app will pull the offline messages. Pulling offline message is a high frequency operation.
Storage key
Offline message includes different types of messages including 1-on-1 chat, group chat, system notifications, etc.
So the offline messages should be keyed on each user's identity, such as UserID.
Retention period
The offline messages only have a certain retention period (1 week) or upper limit (1000 messages). Since the number of users' devices is unknown, offline messages could not stored forever. It should be stored in a FIFO basis.
Message roaming
Trigger
Whenever a user opens a conversation and scroll down, app will pull all history conversations. Message roaming is a low frequent operation.
Typical scenarios:
User installs a new IM app and will be able to see history messages.
When user chats on PC device, the history should also be available on mobile app.
Storage key
Historical message should be keyed on a per conversation id.
Storage model
Flowchart
Differences:
Move all offline message storage to cache, keyed by per user.
Move all history message storage separate, keyed by per thread.
Offline message implementation
For each user, created a SortedSet structure keyed by UserID.
Within the SortedSet, offline messages are sorted by their sequence order.
Complexity for adding an entry: O(logN)
Complexity for reading an entry: O(logN + M)
History message implementation
Store history messages according to the group id.
For each user in the group, store a timestamp when the user joined the group.
Timeline model in DingTalk (钉钉)
References: https://www.alibabacloud.com/blog/implementation-of-message-push-and-storage-architectures-of-modern-im-systems_594780
Chinese references:
TODO
http://www.52im.net/forum.php?mod=collection&action=view&ctid=29&fromop=all
Last updated
