# Storage

* [Storage](#storage)
  * [Database criteria](#database-criteria)
  * [Disk write speed](#disk-write-speed)
  * [Disk write latency](#disk-write-latency)
  * [Hard disk IOPS / Throughput](#hard-disk-iops--throughput)
  * [Single MySQL instance](#single-mysql-instance)
  * [SQL index](#sql-index)
  * [Wechat 2016 World Record for MySQL clusters](#wechat-2016-world-record-for-mysql-clusters)
  * [Netflix flash and storage servers](#netflix-flash-and-storage-servers)
  * [Comparison between distributed file storage systems](#comparison-between-distributed-file-storage-systems)

## Storage

### Database criteria

| Deployment                         | Capacity / Performance      | Other criteria                    |
| ---------------------------------- | --------------------------- | --------------------------------- |
| Replication mode                   | Current data size           | Whether query goes through index  |
| Failover strategy                  | Daily incremental data size | Whether there is multi-table join |
| Disaster recovery strategy         | Read per second             | Whether uses opti/pessi lock      |
| Archive strategy                   | Write per second            | Transaction consistency model     |
| Read/Write separation strategy     | Transaction per second      | JDBC config                       |
| Partitioning and sharding strategy |                             | Sharing tool (Proxy/Client)       |
| Caching strategy                   |                             |                                   |

### Disk write speed

* Traditional magnetic hard drives can write data upto 100 MB/second,
* On an average you can write hardly 100 bytes/second in a random write fashion, this limitation basically comes from the design of how the magnetic disk works
* <https://kousiknath.medium.com/data-structures-database-storage-internals-1f5ed3619d43>

### Disk write latency

* Use 10ms as an average value

### Hard disk IOPS / Throughput

* <https://medium.com/naukri-engineering/understanding-disk-i-o-when-should-you-be-worried-naukri-engineering-f0ab332f52d4>

![](https://1010073591-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-Mk8dv8Mfudl_6ziUzDf%2Fuploads%2Fgit-blob-3dfb311bfc422da5512c9637f5f21300444436cc%2Fstorage_harddisk_rpmToIOPS.png?alt=media)

![](https://1010073591-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-Mk8dv8Mfudl_6ziUzDf%2Fuploads%2Fgit-blob-fccb7f948a21aaf8a55be3d1b2693f9d9e69d189%2Fstorage_harddisk_rpmToThroughput.png?alt=media)

![](https://1010073591-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-Mk8dv8Mfudl_6ziUzDf%2Fuploads%2Fgit-blob-887606de9ca355534919f9c460a62d513a403642%2Fstorage_harddisk_rpmToSSD.png?alt=media)

### Single MySQL instance

* Single row size: 1KB
* Physical upper limit of concurrent connections: 16K
* Single table rows: 20M. Single table size: 1GB. Exceeding this number will result in fast degradation in terms of performance.
* A single MySQL 5.6 benchmark on cloud (Aliyun). Use the following for ease of memorization:
  * TPS: 1k TPS
  * QPS: 25k QPS
  * Connection num: 10K
  * Response time: 10ms (Like a lower bound)

![](https://1010073591-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-Mk8dv8Mfudl_6ziUzDf%2Fuploads%2Fgit-blob-7c10e2360b30739cff4934c795f8c166aa4acb73%2Fmysql_scalability_singleMachinePerf.png?alt=media)

### SQL index

* For querying 400 million records
  * With index, around 0.3ms
  * Without indexes, about 1 minute

### Wechat 2016 World Record for MySQL clusters

* TPS (payment transaction for yearly red envelope): 200K
* RPS (number of yearly red envelope): 760K

### Netflix flash and storage servers

* Storage and Flash. These two server types have very different characteristics.
  * Storage servers consist of mostly spinning disks, can hold upwards of 200 TB, and generate \~40 Gbps of throughput.
  * Flash servers (all SSD disks) can generate up to \~100 Gbps but can hold only up to 18 TB of content.
* Reference: <https://netflixtechblog.com/distributing-content-to-open-connect-3e3e391d4dc9>

### Comparison between distributed file storage systems

![](https://1010073591-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-Mk8dv8Mfudl_6ziUzDf%2Fuploads%2Fgit-blob-b365cb14c178d37452a83e9c8f1d1d101c027f24%2Fonline_video_distributedfilesystems.png?alt=media)