Flowchart
Last updated
Last updated
Overall chart
Two requests are sent in parallel: add file metadata and upload the file to cloud storage. Both requests originate from client 1.
Client app computes the file metadata to be uploaded.
File name, file content MD5
Number of blocks (assume each block is 4MB) and MD5 values for each block.
Client app sends the metadata to web server.
Web server computes globally unique block IDs for each block.
Web server sends metadata and block IDs to metadata DB change the file upload status to “pending.”
Web server returns the block IDs to client app.
Client app request block servers to upload blocks.
Block servers connect to metadata DB to verify permissions.
Block servers verify correctness of MD5 values for each block
Block servers store blocks inside object storage.
Web servers notify notification service that a new file is being added.
The notification service notifies relevant clients (client 2) that a file is being uploaded.
Download flow is triggered by
A file is added or edited elsewhere.
User proactively request to sync files
Notification service informs client app that a file is changed somewhere else.
Client app send requests to web servers to download files.
Web servers call metadata DB to fetch metadata of changes and block IDs.
Web servers return block IDs and block servers to client apps.
Client app sends requests to block servers to download blocks.
Block servers fetch metadata from metadata DB and verify permissions.
Block servers fetch blocks from object storage.
Block servers return blocks to client app.
Client app verify MD5 and build the entire file.
How does a client know if a file is added or edited by another client?
Client online: Notification service will inform client A that changes are made somewhere so it needs to pull the latest data.
Client offline: while a file is changed by another client, data will be saved to the cache. When the offline client is online again, it pulls the latest changes.
Even though both options work well, we opt for long polling for the following two reasons:
Communication for notification service is not bi-directional. The server sends information about file changes to the client, but not vice versa.
WebSocket is suited for real-time bi-directional communication such as a chat app.
