Skip to content
he110
Back to blog

Software Architecture

Organizes the concepts of software architecture, sources of complexity, design principles and selection processes, and discusses database clusters and NoSQL solutions.

Updated

The Definition of Architecture

Before understanding the definition of architecture, a few terms need to be understood first.

System and Subsystem

A system generally refers to a group of related individuals that operate according to certain rules and can accomplish work that individual components cannot complete alone. It means "totality", "whole", or "alliance". —— Wikipedia

A subsystem is also a system composed of a group of related individuals, usually part of a larger system. —— Wikipedia

A system has the following characteristics:

  • Association: a system is composed of a group of related individuals; unrelated individuals piled together cannot become a system. For example, putting an engine and a PC together cannot be called a system; combining an engine, chassis, tires, and frame together makes a car.
  • Rules: the individuals within a system need to operate according to specified rules, rather than each individual acting on its own. Rules define how individuals within the system divide work and collaborate. For example, a car engine is responsible for generating power, which is then output to the wheels through the transmission and drive shaft, thereby driving the car forward.
  • Capability: there is an essential difference between system capability and individual capability; system capability is not the sum of individual capabilities, but produces new capabilities. For example, a car can carry loads and move forward, while the engine, transmission, drive shaft, and wheels themselves do not have such capability.

When discussing the architecture of a system, you should focus on the top-level architecture of the current system, not including the architecture at the subsystem level.

Module and Component

A software module is a set of consistent and closely related software organizations. It contains two parts: programs and data structures. Modern software development often uses modules as units of composition. A module's interface expresses the functionality provided by the module and the elements required to call it. Modules are units that can be written separately. This makes them reusable and allows people to collaborate simultaneously, writing and studying different modules. —— Wikipedia

A software component is defined as a self-contained, programmable, reusable, language-independent software unit; software components can easily be used to assemble applications. —— Wikipedia

Both modules and components are parts of a system; they are just the result of splitting the system from different perspectives:

  • Splitting the system from the perspective of business logic, the resulting units are "modules".
  • Splitting the system from the perspective of physical deployment, the resulting units are "components".

The purposes of the two divisions are also different:

  • The main purpose of dividing modules is separation of responsibilities.
  • The main purpose of dividing components is unit reuse.

Suppose we want to build a student information management system.

  • Splitting this system from a logical perspective, it can be divided into a "login and registration module", a "personal information module", and a "personal grades module"
  • Splitting it from a physical perspective, it can be divided into Nginx, a Web server, and MySQL.

Framework and Architecture

A software framework usually refers to a software component specification for implementing a certain industry standard or completing a specific basic task; it also refers to a software product that provides the basic functions required by the specification when implementing a software component specification. —— Wikipedia

  • A framework is a component specification: for example, MVC is one of the most common development specifications; similar frameworks include MVP, MVVM, J2EE, etc.
  • A framework is a product that provides basic functions: for example, Spring MVC is a development framework for MVC; besides satisfying the MVC specification, Spring provides many basic functions to help us implement features, including annotations (@Controller, etc.), Spring Security, Spring JPA, and many other basic functions.

Software architecture refers to the "fundamental structure" of a software system, the guidelines for creating these fundamental structures, and the description of these structures. —— Wikipedia

Comparing the two definitions, you can find: frameworks focus on "specification", while architecture focuses on "structure"

A framework is a complete set of development specifications; architecture is the concrete implementation plan under a certain set of development specifications, including the combination relationships between modules and the operating rules by which they work together to complete functions.

What Is Architecture

Software architecture refers to the top-level (Rank) structure of a software system; it defines which roles (Role) the system consists of, the relationships (Relation) between the roles, and the operating rules (Rule).

Rank

Rank means that software architecture is layered, corresponding to the layered relationship between "system" and "subsystem".

Usually, we only need to focus on the architecture of one layer, showing at most the architecture of two adjacent layers, without mixing the architecture of every layer together. Whether designing architecture or drawing architecture diagrams, the "top-down, stepwise refinement" approach should be adopted.

---
title: WeChat Architecture Example
---
graph TD
  subgraph Wechat[WeChat Architecture L0]
    User[User]
    Chat[Chat]
    Circle[Moments]
    Pay[Payment]
  end
  subgraph FriendCircle[Moments Architecture L1]
    Moving[Feed]
    Comment[Comment]
    Message[Message]
    Advertising[Advertising]
  end
  subgraph FriendComment[Moments Comment Subsystem Architecture L2]
    Publish[Publish]
    Audit[Review]
    Analysis[Analysis]
  end

  Circle --> FriendCircle
  Comment --> FriendComment

Role

Role refers to which roles the software system contains; each role is responsible for part of the system's functionality.

One of the most important tasks in architecture design is splitting the system into multiple roles. The most common microservice splitting is actually splitting the whole complex business system by business domain into multiple microservices, where each microservice is a role of the system.

Relation

Relation refers to the relationships between the roles of a software system.

In an architecture diagram, these are actually the connecting lines. Relationships between roles cannot be connected arbitrarily; every relationship ultimately needs code to implement, including the connection method ( HTTP, TCP, UDP, serial port, etc. ), the data protocol ( JSON, XML, binary, etc. ), and the specific interfaces.

Rule

Rule refers to how the roles of a software system collaborate to complete system functions.

System capability is not the sum of individual capabilities, but produces new capabilities. So how exactly is this new capability accomplished? Which specific roles participate in this new capability?

This is what Rule expresses. When designing architecture, Rules need to be designed for all core business scenarios.

Architecture Diagram Example

A system architecture diagram is generally used to describe Rank, Role, and Relation

A system sequence diagram is used to describe Rule

The architecture diagram of a simple payment system is as follows:

---
title: E-Commerce System Architecture Example
---
graph TD
  TradingCenter[Trading Center]
  LedgerCenter[Ledger Center]
  AccountingCenter[Accounting Center]
  LiquidationCenter[Clearing Center]

  PaymentCenter[Payment Center]
  RiskControlCenter[Risk Control Center]
  MarketingCenter[Marketing Center]
  VIPCenter[Membership Center]

  TradingCenter -->|Transaction records| LedgerCenter
  AccountingCenter -->|Accounting data| LedgerCenter
  LiquidationCenter -->|Accounting data| LedgerCenter

  TradingCenter -->|Payment requests| PaymentCenter
  PaymentCenter -->|Discount information| MarketingCenter
  PaymentCenter -->|Risk control consultation| RiskControlCenter
  MarketingCenter -->|Cards & coupons| VIPCenter
  RiskControlCenter -->|Member information| VIPCenter

The system sequence diagram for the standalone "QR-code payment" scenario is as follows:

---
title: QR-Code Payment Scenario Example
---
sequenceDiagram
  participant Client as Browser
  participant TradingCenter as Trading Center
  participant PaymentCenter as Payment Center
  participant RiskControlCenter as Risk Control Center

  Client ->> TradingCenter: 1: Request payment
  TradingCenter ->> TradingCenter: 1.1: Create order
  TradingCenter ->> PaymentCenter: 1.2: Request payment
  PaymentCenter ->> RiskControlCenter: 1.2.1: Risk control decision
  RiskControlCenter -->> PaymentCenter: 1.2.2: Risk control passed
  PaymentCenter ->> PaymentCenter: 1.2.3: Bank deduction
  PaymentCenter -->> TradingCenter: 1.2.4: Payment succeeded
  TradingCenter -->> Client: 1.2.5: Payment result

The Purpose of Architecture Design

If no architecture design is done, the system can still run, and development may even be faster — so what is the purpose of architecture design?

Simply put, the main purpose of architecture design is to solve the problems brought by software system complexity

Common Questions

Q: Where do you start designing?

A: By becoming familiar with and understanding the requirements, identifying where the system's complexity lies, and then doing architecture design targeting the complex points

Q: What if there is not enough time for architecture design?

A: Architecture design does not need to cover everything; not every architecture needs to have high performance, high availability, high scalability, etc. Instead, identify the complex points and solve the problems in a targeted way.

Q: There are multiple solutions to reference, but they differ greatly — which one should I choose?

A: Understand the complex points each architecture solution is designed to solve, then compare them with the complex points of your own business, and reference solutions with similar complex points.

Architecture Design in Practice

Let's analyze a simple case to see how to apply the guiding principle "the real purpose of architecture design is to solve the problems brought by software system complexity" in practice.

Suppose we need to design a university student management system whose basic functions include login, registration, grade management, course management, etc.

When doing architecture design for such a system, we should first identify where its complexity lies.

  • Performance: a school has roughly 10,000 ~ 20,000 students, and the student management system is not accessed frequently — on average each student accesses it less than once a day. Therefore, performance is not a complex part: MySQL is fully capable for storage, caching is unnecessary, and Nginx is more than sufficient as the web server.
  • Scalability: the functionality of the student management system is fairly stable, and there is not much room for expansion, so scalability is not complex either.
  • High availability: even if the student management system is down for 2 hours, the impact on student management work is not significant, so load balancing can be skipped, let alone complex solutions like multi-site active-active. However, if all student data were lost, recovery would be very troublesome — only manual record-by-record repair — which is hard to accept. Therefore, storage high reliability needs to be considered, and this is somewhat complex. We need to consider multiple abnormal situations: machine failure and data center failure. For machine failure, we need to design a MySQL same-data-center primary-backup solution; for data center failure, we need to design a MySQL cross-data-center synchronization solution.
  • Security: the information stored by the student management system has some privacy, such as students' family situations, but it is not finance-related and does not contain strongly private information. Therefore, in terms of security, doing 3 things basically meets the requirements: Nginx providing ACL control, user account and password management, and database access permission control.
  • Cost: since the system is very simple, a few servers are basically enough, which is not a problem at all for a university, so it does not need much attention.
---
title: Overall Setup Diagram for the School Scenario
---
graph TD
  subgraph LocalDataCenter[School Main Data Center]
    Nginx[Nginx Server]
    WebServer[Web Server]
    PrimaryDB[Primary Database]
    SecondaryDB[Backup Database]

    Nginx --> WebServer
    WebServer --> PrimaryDB
    PrimaryDB --> SecondaryDB
  end

  subgraph RemoteDataCenter[School Backup Data Center]
    RemoteDB[Backup Database]
    PrimaryDB --> RemoteDB
  end

Sources of Complexity

Sources of complexity can be divided into:

  • Business complexity: domain-driven design
  • Technical complexity: high performance, high concurrency, high availability

High Performance

High-performance complexity is mainly reflected in two aspects:

  • The complexity brought by pursuing high performance inside a single computer
  • The complexity brought by pursuing high performance in a cluster of multiple computers

Single-Machine Complexity

The most critical part inside a computer is the operating system, which determines whether hardware performance can be fully utilized.

In an operating system, the things most related to performance are processes and threads.

Processes

Processes were invented to correspond to tasks; each task has its own independent memory space, processes are independent of each other, and they are scheduled by the operating system.

To achieve the goal of running multiple processes in parallel, time-sharing is used: CPU time is divided into many slices, each slice can only execute instructions of a certain process, making it feel like multiple processes are processing in parallel.

Although multiprocessing requires each task to have an independent memory space with processes independent of each other, from the user's perspective, two tasks being able to communicate during operation makes task design more flexible and efficient.

Otherwise, if two tasks cannot communicate during operation, task A would have to write its results to storage, and task B would then read from storage for processing — not only inefficient, but also making task design more complex.

To solve this problem, various methods of inter-process communication were designed, including pipes, message queues, semaphores, shared storage, etc.

Threads

Multiprocessing allows multiple tasks to be processed in parallel, but it has its own drawbacks: within a single process, processing can only be serial, while in reality many internal tasks do not strictly need to be executed in time order and also need parallel processing.

To solve this problem, threads were invented as sub-tasks inside a process, and these sub-tasks share the same process data.

To ensure data correctness, mechanisms such as mutex locks were invented.

With multithreading, the smallest unit scheduled by the operating system is the thread, while the process is the smallest unit to which the operating system allocates resources

Parallel Tasks

Although multiprocessing and multithreading greatly improve the performance of parallel processing of multiple tasks, they are essentially still time-sharing systems and cannot achieve true parallelism in time.

The obvious way to solve this problem is to let multiple CPUs execute computing tasks simultaneously, thereby achieving true multitasking parallelism.

There are currently 3 such solutions:

  • SMP: ( Symmetric Multi-Processor )
  • NUMA: ( Non-Uniform Memory Access )
  • MPP: ( Massive Parallel Processing )

Among them, SMP is the most common one we see; the currently popular multi-core processors are SMP solutions.

Cluster Complexity

Because business develops too fast — faster than the iteration and upgrade speed of hardware performance.

The performance of a single machine can hardly support the increased business volume, so high performance must be achieved through clustering.

Improving performance with a large number of machines is not as simple as just adding machines; making multiple machines work together to achieve high performance is a complex task.

In fact, "task" covers a wide range: it can refer to complete business processing, or just a specific task.

For example, "storage", "computation", "caching", etc. can all be regarded as a task, so storage systems, computing systems, and caching systems can all be architected according to the task allocation approach.

In addition, the "task allocator" does not necessarily have to be a physically existing machine or an independently running program; it can also be an algorithm embedded in other programs, such as Memcache's cluster architecture.

Common approaches include: task allocation, task decomposition

Task Allocation

Task allocation means that each machine can handle complete business tasks, and different tasks are allocated to different machines for execution.

---
title: Upgrading Single-Machine Architecture to Cluster Architecture
---
graph LR
  subgraph Single[Single-Machine Architecture]
    SingleUser[User] --> BusinessServer[Business Server]
  end

  subgraph Cluster[Cluster Architecture]
    ClusterUser[User] --> Task[Task Allocator]
    Task[Task Allocator] --> BusinessServer1[Business Server 1]
    Task[Task Allocator] --> BusinessServer2[Business Server 2]
  end

  Single --> Cluster

Complexity analysis:

  • A task allocator is added.

This allocator may be a hardware network device (e.g., F5, switches, etc.), a software network device (e.g., LVS), load balancing software (e.g., Nginx, HAProxy), or a self-developed system.

Choosing an appropriate task allocator is also a complex matter, requiring comprehensive consideration of performance, cost, maintainability, availability, and other factors.

  • There are connections and interactions between the task allocator and the real business servers (i.e., the connecting lines from the task allocator to the business servers in the diagram), requiring the selection of an appropriate connection method and the management of connections. For example, connection establishment, connection detection, how to handle connection interruptions, etc.

  • The task allocator needs allocation algorithms added. For example, whether to use a round-robin algorithm, weight-based allocation, or load-based allocation. If allocation is based on server load, the business servers also need to be able to report their status to the task allocator.

As business continues to grow, task processing is no longer the bottleneck; instead, the task allocator becomes the performance bottleneck.

To solve the performance problem, the task allocator also needs to be made into multiple devices:

---
title: Upgrading Single Task Allocation to Multiple Task Allocation
---
graph LR
  subgraph Single[Single Task Allocator]
    SingleUser[User] --> Task[Task Allocator]
    Task[Task Allocator] --> BusinessServer1[Business Server 1]
    Task[Task Allocator] --> BusinessServer2[Business Server 2]
  end

  subgraph Cluster[Multiple Task Allocators]
    ClusterUser[User]

    subgraph TaskManager[Task Allocators]
      direction LR
      TaskManager1[Task Allocator 1]
      TaskManager2[Task Allocator 2]
    end

    subgraph ClusterBusinessServer[Business Servers]
      direction LR
      ClusterBusinessServer1[Business Server 1]
      ClusterBusinessServer2[Business Server 2]
    end

    ClusterUser -.User allocation.-> TaskManager
    TaskManager --> ClusterBusinessServer
  end

  Single --> Cluster

Complexity analysis of multiple task allocators:

  • The task allocator changes from 1 device to multiple devices.

The complexity brought by this change is that different users need to be allocated to different task allocators (i.e., the dashed "user allocation" part in the diagram). Common methods include DNS round-robin, intelligent DNS, CDN (Content Delivery Network), GSLB devices (Global Server Load Balance), etc.

  • The connections between task allocators and business servers change from a simple "1-to-many" to a "many-to-many" mesh structure.
  • The number of machines expands from 3 to more, and the complexity of state management and fault handling also increases greatly.

Task Decomposition

Through task allocation, we can break through the processing performance bottleneck of a single machine and meet business performance needs by adding more machines. But if the business itself becomes more and more complex, the returns from expanding performance solely through task allocation will diminish.

The main reason for this phenomenon is that as business becomes more complex, the processing performance of a single machine gets lower and lower. To continue improving performance, we need to adopt the second approach: task decomposition.

---
title: Task Decomposition Architecture Example
---
graph TB
  InsertServer[Access Server]

  subgraph Subserver[Task Sub-Services]
    direction LR
    Login[Registration & Login]
    Message[Messaging]
    LBS
    Shaking[Shake]
    Random[Drift Bottle]
    Other[Other Services]
  end

  InsertServer <--> Subserver

The backend architecture logically splits the sub-services, including: access, registration & login, messaging, LBS, Shake, Drift Bottle, and other services (chat, video, Moments, etc.).

Through this task decomposition approach, the originally monolithic but complex business system can be split into small, simple business systems that require cooperation between multiple systems.

From a business perspective, task decomposition neither reduces functionality nor reduces the amount of code (in fact, the amount of code may increase, because internal code calls are changed to interface calls between servers).

The reasons why the task decomposition approach improves performance are as follows:

  • Simple systems are easier to make high-performance

The simpler the system's functionality, the fewer points affect performance, and the easier it is to perform targeted optimization.

When the system is very complex, first, it is relatively difficult to find the key performance points because there are too many points to consider and verify; second, even after spending great effort to find them, modifying them is not easy, because you may improve key performance point A but inadvertently reduce the performance of point B — the overall system performance not only fails to improve, but may even decline.

  • Individual tasks can be scaled independently

When logical tasks are decomposed into independent subsystems, the performance bottleneck of the whole system is easier to find, and once found, you only need to optimize or improve the performance of the bottlenecked subsystem without changing the entire system — much less risk.

Take the above backend architecture as an example: if the user count grows too fast and the registration & login subsystem hits a performance bottleneck, you only need to optimize the performance of the login & registration subsystem (either code optimization or simply and crudely adding machines); other subsystems such as messaging logic and LBS logic need no changes at all.

Task splitting can improve performance, but unlimited splitting does not improve performance — it actually degrades it.

The reason is: if the system is split too finely, the number of inter-system calls required to complete a certain business rises exponentially, and since inter-system call channels are currently all transmitted over the network, performance is far lower than in-system function calls.

What ultimately determines business processing performance is the business logic itself; without major changes to the business logic itself, there is a theoretical upper limit on performance. System splitting can bring performance closer to this limit, but cannot break through it.

Therefore, the performance benefit brought by task decomposition has its limits — finer decomposition is not always better — and for architecture design, how to grasp this granularity is very critical.

High Availability

The ability of a system to perform its functions without interruption, representing the degree of system availability, is one of the criteria in system design. —— Wikipedia

From the definition, the most critical point is "no interruption", which is also where the difficulty lies.

Hardware gradually ages, and software becomes increasingly complex and bloated as it iterates.

Besides hardware and software being essentially unable to achieve "no interruption", unavailability caused by the external environment is even more inevitable and uncontrollable. For example, power outages, floods, and earthquakes can also make systems unavailable, with more severe impact and greater difficulty in prediction and avoidance.

So, high-availability solutions for systems come in great variety, but they all share the same essence: achieving high availability through "redundancy".

High-availability "redundancy" solutions, purely in form, look the same as the high-performance ones discussed earlier — both achieve the goal by adding more machines — but in essence there is a fundamental difference:

  • High performance adds machines with the goal of "scaling" processing performance.
  • High availability adds machines with the goal of "redundancy" of processing units.

Computation High Availability

Computation refers to the logical processing work of the business.

The characteristic of computation is: no matter in which environment/machine it runs, the same input with the same program always corresponds to the same output.

So the meaning of computation high availability is: when a computing server goes down, other computing servers can take over the task and continue computing — that is, backup computing services.

A simple multi-machine cluster architecture is as follows:

---
title: Multi-Machine Cluster Architecture Example
---
graph LR
  subgraph Single[Single-Machine Architecture]
    SingleUser[User] --> RunServer[Computing Server]
  end

  subgraph Cluster[Cluster Architecture]
    ClusterUser[User] --> TaskManager[Task Allocator]
    TaskManager --> RunServer1[Computing Server 1]
    TaskManager --> RunServer2[Computing Server 2]
  end

  Single --> Cluster

The allocation algorithm for computation high availability is more complex than task allocation.

In task allocation, all servers are equal; tasks only need to be allocated to the corresponding machines according to certain rules.

Computation high availability involves different roles; only one response is ultimately provided to the caller, and who processes, who decides, and how a decision-maker is promoted after it fails are all sources of complexity.

Storage High Availability

For systems that need to store data, the key point and difficulty of the whole system's high-availability design lies in "storage high availability".

Data Synchronization Delay

Compared with computation, storage has an essential difference: moving data from one machine to another requires transmission over a line.

Line transmission speed is on the order of milliseconds; within the same data center, a few milliseconds can be achieved; for data centers distributed in different places, transmission takes tens or even hundreds of milliseconds.

Although milliseconds are almost imperceptible to humans, for high-availability systems they make an essential difference: it means that at any point in time, the data of the entire system is definitely inconsistent.

Applying the formula "data + logic = business", if the data is inconsistent, even with consistent logic, the final business behavior will be different.

Transmission Line Interruption Risk

Besides the physical transmission speed limit, the transmission line itself also has availability problems: the transmission line may be interrupted, may be congested, may be abnormal (corrupted packets, lost packets), and the failure time of transmission lines is generally particularly long — a short one lasts over ten minutes, a long one may last several hours.

When the transmission line is interrupted, it means storage cannot be synchronized, and during that period the data of the entire system is inconsistent.

Summary

Comprehensive analysis: whether it is transmission delay under normal circumstances or transmission interruption under abnormal circumstances, both will cause the system's data to be inconsistent at some point or during some period, and data inconsistency in turn leads to business problems.

But if no redundancy is done at all, the overall high availability of the system cannot be guaranteed.

So the difficulty of storage high availability lies not in how to back up data, but in how to reduce or avoid the impact of data inconsistency on the business.

In the distributed field, there is a famous CAP theorem that theoretically demonstrates the complexity of storage high availability. That is, storage high availability cannot simultaneously satisfy "consistency, availability, partition tolerance" — at most two of them — which requires us to make trade-offs in architecture design based on the business.

High-Availability State Decision-Making

Whether it is computation high availability or storage high availability, their foundation is "state decision-making" — that is, the system needs to be able to judge whether the current state is normal or abnormal, and if an abnormality occurs, take action to ensure high availability.

If the state decision-making itself is wrong or biased, then any subsequent actions and handling, no matter how perfect, are meaningless and worthless.

But in concrete practice, there happens to be an essential contradiction: for high-availability systems implemented through redundancy, state decision-making is essentially impossible to make completely correct.

Common decision-making strategies include: autocratic, negotiated, democratic.

Autocratic

Autocratic decision-making means there is an independent decision-making body, which we may call the "decision-maker", responsible for collecting information and then making decisions; all redundant individuals, which we may call "reporters", send their status information to the decision-maker.

The autocratic decision-making approach does not have the problem of decision-making chaos because there is only one decision-maker, but the problem is precisely that there is only one decision-maker.

When the decision-maker itself fails, the entire system cannot achieve accurate state decision-making.

If the decision-maker itself gets another set of state decision-making, it falls into a recursive infinite loop.

---
title: Autocratic State Decision-Making
---
graph BT
  Master[Decision-Maker]
  Report1[Reporter]
  Report2[Reporter]
  Report3[Reporter]

  Report1 --> Master
  Report2 --> Master
  Report3 --> Master
Negotiated

Negotiated decision-making means two independent individuals exchange information and then make decisions according to rules. The most commonly used negotiated decision-making is primary-backup decision-making.

The basic negotiation rules of this architecture can be designed as:

  • Both servers start as backups
  • The 2 servers establish a connection
  • The 2 servers exchange status information
  • One server makes the decision and becomes the primary; the other server continues to keep the backup identity.

If there is a problem with the information exchange between the two (such as the primary-backup connection being interrupted), state decision-making can easily run into abnormal scenarios:

  • If the backup believes the primary has failed when the connection is interrupted, the backup needs to be promoted to primary, but in fact the primary has not failed at this point, so the system ends up with two primaries, which does not match the design intent (1 primary 1 backup).
  • If the backup does not consider the primary failed when the connection is interrupted, then if the primary really does fail at this point, the system has no primary, which also does not match the design intent (1 primary 1 backup).
  • If, to avoid the impact of connection interruption on state decision-making, more connections can be added — for example, dual connections or triple connections. Although this can reduce the impact of connection interruption on state (note: only reduce, not completely solve), it also introduces the problem of choosing between the information from these connections — that is, if different connections deliver different information, which connection should prevail? In fact, this is also an unsolvable question: no matter which connection prevails, problems may occur in specific scenarios.

Comprehensive analysis: negotiated state decision-making always has some problems in certain scenarios.

Democratic

Democratic decision-making means multiple independent individuals make state decisions by voting. For example, a ZooKeeper cluster uses this approach when electing a leader.

Democratic decision-making is quite similar to negotiated decision-making; both are based on independent individuals exchanging information, each individual making its own decision, and then determining the final state according to the "majority wins" rule.

The difference is that democratic decision-making is much more complex than negotiated decision-making. ZooKeeper's election algorithm ZAB is bewildering to most people, let alone implementing this algorithm in code.

Besides algorithm complexity, democratic decision-making has an inherent flaw: split-brain.

This term comes from medicine, referring to what happens when the connection between the left and right cerebral hemispheres is cut: the left and right brains cannot exchange information, so each makes its own decisions, and then the body is controlled by two brains separately, making all kinds of strange movements.

The root cause of split-brain is that an originally unified cluster, due to connection interruption, becomes two independent, separated sub-clusters, and each sub-cluster conducts its own election, so 2 primaries are elected — equivalent to a human body having two brains.

To solve the split-brain problem, democratic decision-making systems generally adopt the rule that "the number of voting nodes must exceed half of the total number of nodes in the system".

Although this approach solves the split-brain problem, it also reduces the overall availability of the system: if the system has too few voting nodes not because of split-brain but because of genuine node failures, the system will not elect a primary node either, and the whole system is effectively down, even though nodes 4 and 5 are still normal at that point.

Scalability

Scalability refers to an extension capability provided by a system to cope with future requirement changes: when new requirements appear, the system can support them with no or only a few modifications, without re-architecting or rebuilding the entire system.

To design a system with good scalability, there are two basic conditions:

  • Correctly predict changes
  • Perfectly cope with changes

Predicting Changes

The complexity of predicting changes lies in:

  • You cannot consider scalability at every design point.
  • You cannot completely ignore scalability either.
  • All predictions have the possibility of being wrong.

For architects, how to grasp the degree of prediction and improve the accuracy of prediction results is a very complex matter, and there is no universal standard that can simply be applied; it relies more on one's own experience and intuition.

So when designing the overall architecture, mainly consider the situation of the next few years, without needing to consider changes spanning five or ten years.

Coping with Changes

Even if changes can be predicted, how to cope with them is also a complex matter.

Isolation

Isolate the changing parts into a "change layer", while the remaining unchanging parts are called the "stable layer".

  • For the stable layer, the more stable the interface, the better.
  • For the change layer, finding common points among multiple differing implementations while ensuring that when new features are added, the original interfaces do not need major modifications is a very complex matter, so interface design is equally crucial.
Encapsulation

Extract an "abstraction layer" and an "implementation layer": the abstraction layer is stable and unchanging, the "implementation layer" is modified according to business needs, and the overall implementation depends on the "abstraction layer".

/**
 * The base Component interface defines operations that can be altered by decorators.
 */
interface Component {
  operation: () => string
}

/**
 * Concrete Components provide default implementations of the operations.
 * There might be several variations of these classes.
 */
class ConcreteComponent implements Component {
  public operation(): string {
    return 'ConcreteComponent'
  }
}

/**
 * The base Decorator class follows the same interface as the other components.
 * The primary purpose of this class is to define the wrapping interface for all concrete decorators.
 * The default implementation of the wrapping code might include a field for storing a wrapped component and the means to initialize it.
 */
class Decorator implements Component {
  protected component: Component

  constructor(component: Component) {
    this.component = component
  }

  /**
   * The Decorator delegates all work to the wrapped component.
   */
  public operation(): string {
    return this.component.operation()
  }
}

/**
 * Concrete Decorators call the wrapped object and alter its result in some way.
 */
class ConcreteDecoratorA extends Decorator {
  /**
   * Decorators may call parent implementation of the operation, instead of calling the wrapped object directly.
   * This approach simplifies extension of decorator classes.
   */
  public operation(): string {
    return `ConcreteDecoratorA(${super.operation()})`
  }
}

/**
 * Decorators can execute their behavior either before or after the call to a wrapped object.
 */
class ConcreteDecoratorB extends Decorator {
  public operation(): string {
    return `ConcreteDecoratorB(${super.operation()})`
  }
}

/**
 * The client code works with all objects using the Component interface.
 * This way it can stay independent of the concrete classes of components it works with.
 */
function clientCode(component: Component) {
  // ...

  console.log(`RESULT: ${component.operation()}`)

  // ...
}

// This way the client code can support both simple components...
const simple = new ConcreteComponent()
console.log('Client: I\'ve got a simple component:')
clientCode(simple)
console.log('')

// ...as well as decorated ones.
//
// Note how decorators can wrap not only simple components but the other decorators as well.
const decorator1 = new ConcreteDecoratorA(simple)
const decorator2 = new ConcreteDecoratorB(decorator1)
console.log('Client: Now I\'ve got a decorated component:')
clientCode(decorator2)

Cost

When our architecture solution only involves a few or a dozen servers, cost is generally not the focus of our attention. But if the architecture solution involves hundreds, thousands, or even tens of thousands of servers, cost becomes a very important architecture design consideration.

When we design "high-performance" and "high-availability" architectures, the common approach is to add more servers to meet the "high performance" and "high availability" requirements.

Low cost is exactly the opposite: we need to reduce the number of servers to achieve the low-cost goal.

Therefore, low cost essentially conflicts with high performance and high availability, so low cost is often not the primary goal of architecture design, but an additional constraint on architecture design.

In other words, we first set a cost target; when we design a solution according to the high-performance and high-availability requirements, we evaluate whether the solution can meet the cost target. If not, the architecture needs to be redesigned.

The main complexity that low cost brings to architecture design is that often only "innovation" can achieve the low-cost goal.

The "innovation" here includes both pioneering a brand-new technical field and introducing new technologies. If no new technology that can solve your problem can be found, then you really have to create new technology yourself.

Security

Functional Security

From an implementation perspective, functional security is more related to specific coding and has little to do with architecture.

Many development frameworks now embed common security features, which can greatly reduce repeated development of security-related functionality. But frameworks can only prevent common security vulnerabilities and risks; they cannot foresee new security problems, and frameworks themselves often have vulnerabilities too.

So functional security is a process of gradual improvement, and targeted solutions are often proposed only after problems appear. We can never predict where the system's next vulnerability will be, nor dare we claim that our system definitely has no problems.

Functional security is actually also a contradiction between "attack" and "defense"; it can only be gradually improved in this attack-defense battle and cannot be solved once and for all when designing the system architecture.

Architecture Security

Traditional architecture security mainly relies on firewalls. The most basic function of a firewall is network isolation: by dividing the network into different zones, it formulates access control policies between different zones to control the data flow transmitted between zones with different levels of trust.

Although firewalls are powerful in function, their performance is average, so they are more often used in traditional banking and enterprise application fields.

Internet businesses feature massive user access and high concurrency, which firewall performance cannot support.

There is currently no particularly good design approach for the architecture security of Internet systems; it relies more on the powerful bandwidth and traffic scrubbing capabilities of carriers or cloud service providers, and is rarely designed and implemented in-house.

Scale

The main reason scale brings complexity is that "quantitative change leads to qualitative change".

  • Complexity rising due to increased functionality: there are N functions, but there may also be M connections between functions, so the final complexity is N + M
  • Complexity rising due to increased data: after data volume grows, traditional means of data collection, processing, storage, and analysis can hardly adapt, and new technologies are needed to solve it.

The theoretical foundation of big data is Google's three big data papers:

Architecture Design Principles

Architecture design is different from programming, which has concrete programming language rules as constraints: as long as the input is determined, the result is determined.

Architecture design is essentially uncertain; the architectures produced by company A and company B for the same system may differ greatly, yet have the same functionality and both run normally.

Among the many designs, three principles need to be followed: the suitability principle, the simplicity principle, the evolution principle

The Suitability Principle

There is no need to compare with well-known architectures; manpower, knowledge accumulation, and business scenarios are all different.

Truly excellent architectures are designed under the enterprise's current manpower, conditions, business, and other constraints, can reasonably integrate resources and maximize their effectiveness, and can be implemented quickly.

The Simplicity Principle

When most people evaluate the quality of a solution, complexity is one of the important reference indicators.

Complexity comes from: structural complexity, logical complexity

Structural Complexity

Three problems with structural complexity:

  • The more components, the more likely one of them will fail
  • A change to one component affects all associated components
  • Locating a problem in a complex system is more difficult

Logical Complexity

Logical complexity includes two situations:

  • Functional complexity: high modification cost during feature iteration
  • Algorithm complexity: hard to understand, hard to implement, hard to modify, hard to solve problems quickly

The Evolution Principle

Software architecture design is actually more like nature "designing" a creature: through evolution, the creature adapts to the environment and gradually becomes stronger:

  1. The designed architecture must meet the business needs of the time
  2. The architecture must be continuously iterated in actual application: keep excellent designs, fix flawed designs, correct wrong designs, remove useless designs, so that the architecture gradually improves
  3. When the business changes, the architecture must be extended, refactored, or even rewritten; the code may be rewritten, but valuable experiences, lessons, logic, designs, etc. can continue in the new architecture

Architecture Design Process

Identifying Complexity

List the main complexity problems, then rank them according to the combined situation of business, technology, team, etc., and prioritize solving the most significant complexity problem currently faced.

There is a common concern about solving complexity by priority: if complexity is solved by priority, after solving the higher-priority complexity, the solution for subsequent complexity may require the already-implemented solution to be overturned and redone.

This concern is theoretically possible, but almost impossible in reality, due to the plasticity and variability of software systems.

For the same complexity problem, a software system can have multiple solutions, and the one with the highest overall cost performance can always be picked.

Even if the architect decides to overturn and redo, the new solution must also be able to simultaneously solve the complexity problems that have already been solved. Generally speaking, solutions that can achieve this ideal state basically rely on the introduction of new technology.

For unfamiliar business, the exclusion method can be used to determine where the complexity problems lie:

High Performance

For architects, the concern is not a day's data, but one second's data, i.e., TPS ( Transactions Per Second ) and QPS ( Queries Per Second ).

Generally, the following formulas are used for estimation:

  • Peak = average * 3
  • Performance headroom = peak * estimated business growth rate within two years

High Availability

Judge based on the business scenario: what consequences would result from data loss, or a business subsystem reading data one time less — for example, violating national laws and regulations, causing user churn, property loss, etc.

High availability does not necessarily mean the entire system needs high availability; it can be the high availability of a certain system or even a certain module.

High Scalability

Consider whether there is still the possibility of continued change within the next two years, whether the existing architecture can satisfy such changes, etc.

Other Complexity

Based on the specific business, continue analyzing whether there is business complexity, such as: module coupling, business coupling, security, cost, and so on.

Designing Candidate Solutions

New technologies all develop on the basis of existing technologies, which in turn come from earlier technologies. Functionally grouping technologies can greatly simplify the design process — this is the primary reason for the "modularization" of technology. The "combinatorial" and "recursive" characteristics of technology will completely change our understanding of the nature of technology. —— "The Nature of Technology"

When designing solutions, the following points need attention:

  • The number of candidate solutions is generally 3-5. Fewer than 3 solutions may be due to narrow thinking and insufficient consideration; more than 5 requires a lot of energy and time, and the differences between solutions may not be obvious.
  • The differences between candidate solutions should be significant. Different detail configurations of the same solution are not suitable for being independent separate solutions; there need to be obvious differences in mechanism.
  • The technologies of candidate solutions should not be limited to already familiar technologies. When designing architecture, architects need to broaden their horizons and consider more possibilities, rather than being overly inclined toward familiar technologies and ignoring new technology solutions.
  • Focus on technology selection rather than technical details. Focusing on details makes it easy to regard homogeneous solutions as different solutions, and there are too many details to discuss, making it easy to get sidetracked during reviews.
  • Solutions must be feasible. If a solution is simply infeasible, it should not be included in the candidate list; being unimplementable due to factors like manpower and resources does not fall into the infeasible category.

Evaluating and Selecting Solutions

Basic principle: list the quality attribute points we need to focus on, then evaluate each solution along the dimensions of these quality attributes, and comprehensively pick the optimal solution suitable for the situation at the time.

Common solution quality attribute points include:

  • Performance
  • Availability
  • Hardware cost
  • Project investment
  • Complexity
  • Security
  • Scalability

When evaluating these quality attributes, follow the "suitability principle" and the "simplicity principle", avoiding the pursuit of grandiosity and completeness; basically, it is enough for a quality attribute to satisfy business development for a certain period.

When considering problems, follow the "evolution principle": do not consider the impact of low-probability events, avoiding falling into over-design that leads to wasted investment.

When selecting a solution, selection should be made according to the priority of different quality attributes.

For example, BAT-level companies focus more on quality and less on cost, while startups focus more on cost and less on quality; the technical solutions selected by the two for the same scenario are also different.

During selection, follow the quality attributes: first pick out the solutions that satisfy the first-priority quality attribute; if there are multiple solutions, then compare the second priority, and so on, until the single most suitable solution is selected.

Detailed Solution Design

An extreme situation that may be encountered in the detailed design phase is discovering that a candidate solution is infeasible during detailed design. The main reason is usually that a certain key technical point or key quality attribute was omitted when designing the candidate solution.

To avoid this situation, you need to:

  • The architect must not only design and select candidate solutions, but also have a fairly deep understanding of the key details of the candidate solutions.
  • Reduce solution complexity as much as possible through step-by-step, phase-by-phase, system-by-system approaches. The higher the complexity of the solution itself, the higher the possibility that some detail will overturn the whole solution; appropriately reducing complexity can reduce this risk.
  • If the solution itself is very complex, adopt a design team approach, pooling wisdom through division of labor.

The points to consider during refinement can be considered according to the user journey, including:

  • How does data flow?

  • What are the system's callers? How do they interact with the system?

  • How is interface atomicity guaranteed?

  • Transaction rollback

  • What structure / database tables are used for storage?

  • How is data replicated / backed up?

  • What solution is used for primary-backup switching?

  • Selection of the overall communication protocol

  • Whether there is a retry mechanism when the network is abnormal

  • Does communication use duplex or simplex? Long connections or polling?

  • Is there a deduplication mechanism for repeated sends?

  • Are there requirements on data order?

  • Whether permission control risks are involved

Common High-Performance Architectures

Database Cluster

Read-Write Separation

The basic principle of read-write separation is distributing database read and write operations to different nodes.

---
title: Read-Write Separation Architecture Example
---
graph TB
  BusinessServer[Business Server]

  subgraph dataset[Database Cluster]
    Master[Primary]
    Slave1[Replica]
    Slave2[Replica]

    Master -->|Replication| Slave1
    Master -->|Replication| Slave2
  end

  BusinessServer -->|Write| Master
  BusinessServer -->|Read| Slave1
  BusinessServer -->|Read| Slave2

The basic rules are:

  • The database servers form a primary-replica cluster; one primary with one replica or one primary with multiple replicas are both fine.
  • The database primary handles read and write operations; the replicas only handle read operations.
  • The database primary synchronizes data to the replicas through replication; each database server stores all the business data.
  • The business server sends write operations to the database primary and read operations to the database replicas.

Here we use "primary-replica cluster" rather than "primary-backup cluster". The "replica" can be understood as a "servant" — a servant has to work for the master, and a "replica" needs to provide the function of reading data; while a "backup" is generally considered to only provide backup functionality, not access functionality. So whether to use "primary-replica" or "primary-backup" depends on the scenario; these two terms are not completely equivalent.

This architecture gives rise to two major problems: primary-replica replication delay, and the allocation mechanism

Primary-Replica Replication Delay

Primary-replica replication delay brings a problem: if the business server reads immediately after writing data to the database primary server, the read operation accesses a replica, and the primary has not yet replicated the data over, so reading from the replica cannot get the latest data, and business problems may occur.

Solutions include the following:

  • Read from the primary directly after writing
    • Disadvantage: strongly bound to the business, very intrusive and harmful to the business, and difficult to maintain later
  • Second read: read from the replica by default, and only access the primary when delay is detected
    • Advantage: decoupled from business logic, simple to implement
    • Disadvantages:
      • There may be a large number of second reads breaking through to the primary
      • In extreme scenarios it cannot solve the consistency problem, such as a write operation happening between the two reads
  • Use the primary for critical business, read-write separation for non-critical business
Allocation Mechanism

The allocation mechanism considers how to distinguish read and write operations; there are generally two approaches: program code encapsulation, and middleware encapsulation

Program code encapsulation means abstracting a data access layer in the code to implement the separation of read and write operations and the management of database server connections.

---
title: Program Code Encapsulation Architecture
---
graph TB
  subgraph BusinessServer[Business Server]
    Business[Business Layer]
    DataLayer[Data Access Layer]
    Business --> DataLayer
  end

  subgraph dataset[Database Cluster]
    Master[Primary]
    Slave1[Replica]
    Slave2[Replica]

    Master -->|Replication| Slave1
    Master -->|Replication| Slave2
  end

  DataLayer -->|Write| Master
  DataLayer -->|Read| Slave1
  DataLayer -->|Read| Slave2

The program code encapsulation approach has several characteristics:

  • Simple to implement, and can implement fairly customized features according to the business.
  • Each programming language needs its own implementation and cannot be universal; if a business contains multiple subsystems written in multiple programming languages, the repeated development workload is relatively large.
  • In failure situations, if primary-replica switching occurs, all systems may need to modify their configuration and restart.

Middleware encapsulation means building an independent system to implement the separation of read and write operations and the management of database server connections. The middleware provides an SQL-compatible protocol to business servers, so business servers do not need to do read-write separation themselves. For business servers, accessing the middleware is no different from accessing the database; in fact, from the business server's perspective, the middleware is a database server.

---
title: Middleware Encapsulation
---
graph TB
  BusinessServer[Business Server]
  DataMiddleware[Database Middleware]

  BusinessServer -->|Standard SQL Protocol| DataMiddleware

  subgraph dataset[Database Cluster]
    Master[Primary]
    Slave1[Replica]
    Slave2[Replica]

    Master -->|Replication| Slave1
    Master -->|Replication| Slave2
  end

  DataMiddleware -->|Write| Master
  DataMiddleware -->|Read| Slave1
  DataMiddleware -->|Read| Slave2

The characteristics of the database middleware approach are:

  • Can support multiple programming languages, because the database middleware provides a standard SQL interface to business servers.
  • The database middleware must support the complete SQL syntax and the database server protocol; the implementation is fairly complex with a great many details, bugs appear easily, and it takes a long time to stabilize.
  • The database middleware does not execute the real read and write operations itself, but all database operation requests must pass through the middleware, so the performance requirements for the middleware are also high.
  • Database primary-replica switching is imperceptible to business servers; the database middleware can detect the primary-replica status of database servers. For example, write a piece of data to a test table: success means it is the primary, failure means it is a replica.

Database and Table Sharding

Read-write separation distributes the pressure of database read and write operations, but does not distribute storage pressure. When the data volume reaches tens of millions or even hundreds of millions of rows, the storage capacity of a single database server becomes the system's bottleneck, mainly reflected in these aspects:

  • The data volume is too large, so read and write performance declines; even with indexes, the indexes become very large, and performance also declines.
  • Data files become very large, and database backup and recovery take a long time.
  • The larger the data files, the higher the risk of data loss in extreme cases.

To solve the performance problem, databases / tables need to be split.

Business-Based Database Sharding

Business-based database sharding means distributing data to different database servers according to business modules.

---
title: Business-Based Database Sharding Architecture
---
graph TB
  BusinessServer[Business Server]

  subgraph DataCluster[Database Cluster]
    UserDataServer[Server storing user data]
    CommodityDataServer[Server storing commodity data]
    OrderDataServer[Server storing order data]
  end

  BusinessServer -->|Get business data| DataCluster

This solution also has problems, such as:

  • join operation problems: after business-based sharding, tables originally in the same database are scattered into different databases, making SQL join queries unusable.
  • Transaction problems: originally, different tables in the same database could be modified in the same transaction; after business-based sharding, tables are scattered into different databases and cannot be modified uniformly through transactions. Although there are distributed transaction solutions, their performance is too low, inconsistent with the high-performance goal.
  • Cost problems: more servers naturally means higher costs.

Generally speaking, a single database server can support a user scale of about 100k without problems, so startups generally do not do database and table sharding design directly.

For mature large companies in the industry, since there are already mature solutions for business-based database sharding, and even tentative new businesses have massive user scale — fundamentally different from the small companies with startup businesses mentioned earlier — it is best to consider business-based database sharding from the very beginning of business design.

Table Sharding

When the data volume grows, even database sharding can hardly bear business at the scale of millions or even tens of millions of users, and a single table's data will also reach the processing bottleneck of a single server, so it is necessary to split the data tables.

There are two ways to split:

  • Vertical table sharding: split columns, not rows
  • Horizontal table sharding: split rows, not columns

After splitting, whether to store the tables in different databases can be decided according to the business; it is not mandatory.

After a single table is split into multiple tables, the new tables may bring considerable performance improvements even in the same database server; if performance can meet business requirements, it is acceptable not to split them across multiple database servers

Vertical table sharding is suitable for splitting out certain columns of a table that are infrequently used and occupy a lot of space. The introduced complexity is mainly reflected in the increased number of table operations: what was originally one query may need to be changed to multiple queries and then concatenated.

Horizontal table sharding is suitable for tables with a particularly large number of rows. Some companies require that a single table must be sharded once it exceeds 50 million rows; this number can be used as a reference, but it is not an absolute standard — the key still lies in the table's access performance.

After horizontal table sharding, which split sub-table a certain piece of data belongs to needs to be calculated with an additional routing algorithm, which introduces some complexity.

Common routing algorithms include:

  • Range routing: select an ordered column as the condition and store data of different ranges separately. For example, Taobao's historical orders are queried separately, keeping only recent orders with a high lookup probability.
    • Complexity: the rules for selecting segments are fairly complex; factors such as the number of tables after splitting and query performance need to be considered.
    • Advantage: can be smoothly expanded as data grows
    • Disadvantage: data distribution is uneven; some tables may have a large amount of data while others have very little
  • Hash routing: select the value of a certain column or certain columns, perform a Hash calculation, and distribute the data to different database tables based on the result.
    • Complexity: the design of sharding rules, how to compute the hash value, which columns to use, and so on
    • Advantage: even data distribution
    • Disadvantage: hard to expand; all data needs to be redistributed
  • Configuration routing: configuration routing is a routing table, using an independent table to record routing information. Configuration routing is simple to design and very flexible to use, especially when expanding tables: you only need to migrate the specified data and then modify the routing table.
    • Advantages:
      • Simple to implement
      • Convenient for table expansion
    • Disadvantages:
      • One extra query is required, which affects overall performance
      • If the routing table gets too large, it affects performance and needs to be split further — an infinite loop

NoSQL

Because SQL has some business incompatibility problems, such as:

  • Unable to store data structures, only rows
  • Inconvenient schema extension
  • High I/O in big data scenarios
  • Weak full-text search capability

For different problems, different NoSQL ( Not only SQL ) solutions were born

  • K-V storage: solves the problem that relational databases cannot store data structures, represented by Redis.
  • Document databases: solve the problem of strong schema constraints in relational databases, represented by MongoDB.
  • Column-oriented databases: solve the I/O problem of relational databases in big data scenarios, represented by HBase.
  • Full-text search engines: solve the full-text search performance problem of relational databases, represented by Elasticsearch.

K-V Storage

The full name of K-V storage is Key-Value storage.

Redis is a typical representative of K-V storage. It is an open-source (BSD-licensed), high-performance K-V caching and storage system. Redis's Value is a concrete data structure, including string, hash, list, set, sorted set, bitmap, hyperloglog, so it is often called a data structure server.

Redis's disadvantages are mainly reflected in not supporting full ACID transactions. Although Redis provides transaction functionality, Redis transactions can only guarantee isolation and consistency ( I and C ), not atomicity and durability ( A and D ).

Although Redis does not strictly follow the ACID principles, in fact most businesses do not need to strictly follow the ACID principles either.

Document Databases

To solve the problems brought by relational database schemas, document databases emerged. The biggest feature of document databases is no-schema: they can store and read arbitrary data. Currently, most document databases store data in JSON (or BSON) format, because JSON data is self-describing: there is no need to define fields before use, and reading a field that does not exist in a JSON document does not cause a syntax error like SQL would.

The advantages of document databases lie in:

  • Simple to add fields: just read and write the new data format directly.
  • Historical data will not error: when a newly added field does not exist in historical data, it only returns a null value without errors, and the business can handle the compatibility.
  • Convenient complex data storage: JSON can directly store data of any level and any structure. With SQL, you would need to keep splitting tables and then add mapping tables for association, which is quite complex.

The disadvantages of document databases lie in:

  • No transaction support
  • No join support: if data is stored in two places, you need to query twice, and cannot use join like SQL to query only once

Column-Oriented Databases

Corresponding to row-oriented databases (data is stored by row, with the data of the same row stored together), the only change is column-oriented storage (the data of each column is stored in adjacent locations, making it easy to read).

Advantages:

  • High efficiency when the business reads only a few columns: data of the same column is stored adjacently, so one disk operation can read the required columns contiguously into memory without reading entire rows.
  • Data in the same column has the same type and is stored adjacently, yielding higher compression ratios, which further reduces disk I/O and storage usage.

Disadvantage: business scenario limitations; if the business scenario does not fit, it can be even slower than SQL.

Column-oriented databases are suitable for offline big data analysis and statistics, and not suitable for situations requiring row-by-row modification.

Full-Text Search Engines

SQL has the following problems when handling full-text search:

  • The conditions of full-text search can be combined arbitrarily; if satisfied through indexes, the number of indexes would be very large.
  • Indexes cannot satisfy the fuzzy matching of full-text search; only like queries can be used, and like queries are full-table scans with very low efficiency.

Full-text search engines use inverted indexes. The difference between forward indexes and inverted indexes is:

  • A forward index retrieves content by name, such as retrieving an entire article by its title, including additional information like creation time and modification time.
  • An inverted index queries document content by keyword, such as which documents keyword A has appeared in.

The index objects of full-text search engines are words and documents, while the index objects of relational databases are keys and rows. The terminology of the two differs greatly and cannot simply be equated. Therefore, to let full-text search engines support full-text search of relational data, some conversion operations are needed — that is, converting relational data into document data.

The commonly used conversion approach at present is to convert relational data into JSON documents in object form, and then feed the JSON documents into the full-text search engine for indexing.

Common High-Availability Architectures

Common Scalability Architectures