Several CAPI-Enabled Accelerators for OpenPOWER Servers Revealed

Over a dozen special-purpose accelerators compatible with next-generation OpenPOWER servers that feature the Coherent Accelerator Processor Interface (CAPI) were revealed at the OpenPOWER Summit last week. These accelerators aim to help encourage the use of OpenPOWER based machines for technical and high-performance computing. Most of the accelerators are based on Xilinx high-performance FPGAs, but some feature custom silicon.

IBM’s CAPI port is a PCIe 3.0-based interconnection specifically designed for programmable processors (e.g., ASICs, GPUs, FPGAs, etc.) that enables them to address the same memory address space as the CPU. CAPI requires custom hardware incorporated into IBM’s POWER8 processors, which is called the coherent accelerator processor proxy (CAPP), as well as a POWER service layer (PSL) integrated into CAPI-supporting processors. CAPP maintains a directory of cache lines held by the accelerator and snoops the processor bus for the accelerator. The PSL performs address translations and holds the coherent data for quick access by the accelerating hardware. To work, CAPI has to be supported by the hardware, the operating system and the application in use. At present, IBM’s POWER8 CPUs, a number of accelerators, RedHat Enterprise Linux 7.2 LE (and higher), and Ubuntu LE, as well as select programs, support CAPI.

IBM and the OpenPOWER Foundation need CAPI in order to enable a relatively simple and inexpensive way to build special-purpose accelerators for various workloads. The aim is to make POWER8-based machines viable for a variety of market segments as well as to create platforms that can process modern workloads faster.

While it is possible to enable unified memory for CPUs and co-processors using custom hardware and multiple tweaks in device drivers, this requires huge investments in silicon development, complex drivers and a number of other things. By contrast, programming an FPGA (field-programmable gate array) is considerably cheaper, and the CAPI technology brings them key heterogeneous processing capabilities. While this does not necessarily enable higher bandwidth between the CPU and the accelerator (after all, CAPI is layered on top of PCI Express 3.0 and a specified peak bandwidth), according to IBM they remove overheads, improve performance and can potentially simplify the workflow for programmers. In short, CAPI is an important part of IBM’s POWER strategy in general as well as OpenPOWER initiative.

At this year’s OpenPOWER Summit, IBM and its partners revealed over a dozen of special-purpose CAPI-enabled FPGA-based accelerators. This shows that the OpenPOWER platform is gaining interest and investment from different sources. The list of developers includes such companies as BittWare, DRC, IBM, Mellanox, Xilinx and others, but some decided not to publish details about their accelerators, as it seems from OpenPOWER’s press release. The accelerators revealed at the conference are either available or are set to become available in the coming quarters. The devices come in the form of PCIe 3.0 x8 or x16 cards and are compatible with IBM POWER8-based servers.  Some are also compatible with machines running other processors (and in this case, CAPI is not supported).

One of the important announcements at the summit was Edico Genome’s DRAGEN genomics platform, which uses an accelerator powered by the Xilinx Virtex-7 980T FPGA and is equipped with 16 GB of quad-channel DDR3L-1866 memory. The platform, which is based on a 2-way IBM S822LC server, can analyze an entire genome in 26 minutes, down from approximately 30 hours on general-purpose processors. An earlier prototype was shown at SuperComputing 2015, however this seems to be the announcement of the full product.

Other interesting solutions discussed at the summit include an FPGA-based accelerator for discovering relationships hidden in big data; an FPGA-powered fuzzy search engine for imprecise string searching and matching, which can analyze millions of messages and data streams without indexing; as well as various reconfigurable accelerators for HPC, Big Data, and so on. IBM also mentions that there are companies offering CAPI-enabled building blocks for FPGAs for computer vision, machine learning, and other applications. Some of those companies are startups or working in stealth mode (we do not know whether they developed their building blocks thanks to the SuperVessel program, though this is a possibility), and they may announce their products over time.

While the number of CAPI-enabled accelerators available today is not high, it is growing, which is a good news for the OpenPOWER ecosystem. Positive news (from IBM) is the number of China-based companies developing accelerators featuring CAPI, which shows that local companies in growing markets for servers are expressing interest in such solutions.