Overview
InferNex is an end-to-end, atomically decoupled LLM inference acceleration solution. It targets bottlenecks across the LLM inference pipeline with multi-factor intelligent routing, dynamic xPyD disaggregated scheduling and resource management, near-real-time monitoring, and distributed multi-tier KVCache, reducing TPOT and TTFT while improving TPS.
Architecture
Get Application Package
Helm Chartlatest
Use Helm to pull the application chart package locally before installation.
helm pull oci://cr.openfuyao.cn/charts/infernexContribution
Repositories
Browse core repositories of infernex, submit PRs, and collaborate with the community.
View Repositories
SIG
View the SIG for infernex, including meetings, contacts, repositories, and roadmap.
View Details