Many application workloads are characterized by resource use that varies over time. Usage peaks can vary depending on the time of the day, week, or month, and outside events. Even a big sport game or an online sale can bring usage spikes. With traditional, on-premises technologies, IT departments typically provisioned enough capacity to manage worst-case scenarios. This approach to scalability can leave a significant amount of underutilized resources in the data centers, most of the time—and inefficiency that can impact overall costs. Cloud computing promises answers to such inefficiencies. In the cloud, you can balance costs and performance by deploying new resources when needed and shrinking them during slack periods. This elastic approach matches demand and capacity.
The difficulty in adopting elasticity depends on the various tiers and layers of an application. It’s relatively simple to scale stateless front-end or mid-tier compute nodes depending on the amount of processing resources needed to support a certain workload. For example, you can monitor key node metrics, and apply reactive policies. At Amazon you can use AWS and other related services and in Microsoft Azure, you can use Azure Web Sites, Web Apps, Logic Apps, worker roles, or Azure Virtual Machines.
Application elasticity can become more complex when your application handles persisted storage with relational database management systems (RDBMSs). In the past, the solution was vertical scalability (“Buy a bigger box!”). With this option, application development and management remain relatively simple for the price of a bigger hardware investment, but this ease comes at the cost of a true elastic approach and an efficient use of computing resources.
What if a single database server cannot be scaled up over a certain size? What if its scalability doesn’t improve over a given threshold because of specific resource contention in the system? In such cases, horizontal partitioning and scale-out approaches such as sharding can help. You can apply scale-out techniques across all tiers—from the front end to the application servers. However, for the data tier, a scale-out approach is critical for resolving the issues that vertical scalability can’t adequately address.
Sharding is an established technique for scaling out data tiers by persisting and querying a large data set across a number of partitions, called shards, which can be hosted by one or more nodes in a data store of some type. With both relational and nonrelational data stores, you can use sharding to support large data sets with high-throughput workloads that exceed the capacity and resources of a single server.
In future articles, let me take a shot in understanding what potential Sharding techniques can be and how one can use such techniques with real world applications. Do let me know if you have used any of the sharding techniques in your environments and how efficient have they been?
Reference: Pinal Dave (https://blog.sqlauthority.com)
