To further understand application scenarios, enterprises’ needs, and improve dev teams’ understanding of Apache ShardingSphere, SphereEx launched the “Enterprise Visits” series. The second stop took us to the leading online streaming platform in Asia, iQiyi.
In November 2021, a team from the ShardingSphere community visited the innovation center of iQiyi for in-depth interactions and discussions with their counterparts from Beijing and Shanghai. During the meetup, Zhang Liang, Apache ShardingSphere PMC Chair and SphereEx Founder provided details on the latest Apache ShardingSphere community initiatives, its future development, and Database Mesh.
During the meetup, iQiyi was especially interested in the capabilities and future plans of ShardingSphere and Database Mesh. here are some key takeaway questions from the visit:
Q: How does ShardingSphere manage its encryption keys?
Encryption key management is a capability subordinate to the encryption algorithm. When ShardingSphere needs an encryption key in its decryption, the key could be passed through properties configured by the encryption algorithm. In principle, encryption key management is not a capability of ShardingSphere but the job of the encryption algorithm.
Q:Is there a built-in cryptographic algorithm in ShardingSphere? Or is it done through outside cooperation?
Strictly speaking, ShardingSphere is not equipped with built-in capability. That’s because the pluggable architecture of ShardingSphere is divided into multiple layers, and the encryption module itself is only a capability of L2 of the pluggable architecture.
The encryption module only defines the top-layer encryption connector and the detailed encryption algorithm achieves the pluggable part of the connector. Currently, ShardingSphere supports several of the most commonly used open source algorithms, but it does not support any national cryptographic algorithm in particular.
Integrating an encryption algorithm is easy for developers: instead of changing source codes, all they need is a ShardingSphere encryption algorithm connector. ShardingSphere’s data encryption process is completely transparent to users and very easy to use.
What’s the future plan for Database Mesh?
Database Mesh is another ShardingSphere initiative. We are now planning to integrate Database Mesh through Sidecar. Sidecar will be used to manage traffic, while ShardingSphere will be used to manage to compute, and the underlying databases’ nodes to manage storage.
Sidecar’s and ShardingSphere functions overlap in part, but not completely. ShardingSphere has more than 190 modules, therefore it’s not possible to completely replace ShardingSphere with Sidecar. That’s why ShardingSphere empowers some lightweight capabilities to Sidecar, and Sidecar is equipped with the capabilities to buffer some traffic and requests.
The database computing capability is still processed by ShardingSphere, as it would consume too many application resources to run heavyweight computation via Sidecar for the distributed transaction, query and optimization of heterogeneous databases.
In the future, other Sidecar capabilities, such as management, SQL auditing and authorities, will be developed. In this way, Sidecar and ShardingSphere will have their own focuses, providing a better solution.
Sidecar is equivalent to a data plane. Will you combine the control plane with Istio or will you build the control plane separately?
ShardingSphere uses DistSQL (Distributed SQL) to create rules for sharding, encryption and so on. ShardingSphere is already equipped with the control plane capability of Sidecar. Any capability achieved on Sidecar can be controlled by ShardingSphere’s DistSQL. In the future, we will consider integrating Sidecar into the Istio ecosystem.
So far, Apache ShardingSphere with the release of a version of 5.0 has officially repositioned itself in a new field with the Database Plus concept at its core. In addition to the powerful incremental capability at the upper layer of databases, the Database Plus architecture also opens a highly extensible database ecosystem to developers and users, charting a new future development course for Apache ShardingSphere.
Apache ShardingSphere Open Source Project Links:
[ShardingSphere Github] (https://github.com/apache/shardingsphere/issues?page=1&q=is%3Aopen+is%3Aissue+label%3A%22project%3A+OpenForce+2022%22)
