Background
To cope with the ever-upgrading Internet business system and keep pace with the technology informatization construction of peers, it is important to build a financial cloud platform with the unique financial architecture of Suzhou Rural Commercial Bank. The high-availability cloud platform should stably support financial production businesses, implement business migration to the cloud, fast application iteration, and high business availability in the production environment, and provide infrastructure and management service capabilities for the fine-tuning, inference, management, evaluation, and services of foundation models of the bank.
Solution
The container cloud platform meets the production standards of Suzhou Rural Commercial Bank in terms of usability, fault tolerance, auto scaling, load balancing, and monitoring alarms. It implements network isolation, resource quota, log collection, multi-tenancy, health check, and monitoring alarms to ensure high availability and security of the production environment. The CI/CD pipeline module is added to accelerate continuous application deployment and release, meeting the requirements for fast iteration and release of applications.
- The foundation model management service platform implements the fine-tuning, inference, management, evaluation, and services of foundation models.
- An Ascend NPU resource pool is built to dynamically allocate NPU resources on demand.
- Ascend 910B is used for training, and Ascend 310 is used for inference, combining high-end and low-end products.
- Unified deployment and O&M simplify O&M.
Benefits
- Lowering requirements for personnel skills and comprehensively improving work efficiency.
- Accelerating environment readiness, ensuring business rollout, and focusing on businesses.
- Effectively improving resource utilization and ensuring business availability.
- Multi-model management: managing large and small models of different types and sizes in a unified manner.
- Multi-NPU-card management: managing and allocating NPU cards in a unified manner.
- Model fine-tuning: performing model fine-tuning training based on private data of the bank, and supporting multi-card fine-tuning to improve training efficiency.
- Model deployment: unifying the deployment and rollout process for multiple models and supporting distributed inference and service quality monitoring.