跳到正文
AI NEWS JOURNAL
NVIDIA DeveloperAI NEWS / JOURNAL

Integrating QPUs into Supercomputing Centers Using Slurm HRES Plugins

Quantum processors are being integrated into supercomputing centers worldwide. National programs in North America, Europe, and Asia are driving the installation...

翻译服务暂未运行,当前显示原文。

AI-Generated Summary Slurm 26.11 introduces Hierarchical Resources (HRES) plugins that model quantum processing units (QPUs) as native, cluster-level schedulable resources. HRES associates a QPU with the specific nodes allowed to use it, overcoming the limitations of generic resources (GRES) that bind devices to individual nodes and license-based approaches that leave compute nodes idle while waiting for the QPU. A short Lua plugin handles device health checks, job admission before node allocation, and resource release, while keeping site-specific policies in resources.yaml and device logic portable across Slurm sites. Open source communities QRMI and QDMI have released HRES plugins that connect Slurm clusters to quantum computers from multiple providers. Next Steps Deploy the QRMI HRES plugin to connect a Slurm cluster to quantum backends. Explore the QDMI HRES example for provider-managed queue scheduling. Read the HRES documentation to understand the plugin API and resources.yaml configuration. Quantum processors are being integrated into supercomputing centers worldwide. National programs in North America, Europe, and Asia are driving the installation of quantum processing units (QPUs) alongside high-performance computing (HPC) systems. Every supercomputer will eventually be a hybrid quantum-classical device, drawing on quantum processors to expand what’s possible in scientific research. The scientists that will become the first power users of quantum-GPU supercomputing already run their simulations and workflows on HPC systems. Chemistry, weather modeling, engineering simulation, and other traditionally HPC-centric fields stand to benefit from quantum acceleration for certain workloads. For these scientists, the QPU is simply another accelerator, alongside the CPUs and GPUs that supercomputing centers already provide. Making quantum computers readily available, easily programmable, and familiar to access, reserve, and execute at scale is imperative for scientific researchers to be able to adopt quantum processors into workflows. Until now, a key missing piece has been a workload manager. Slurm 26.11 changes that with Hierarchical Resources (HRES) plugins, which allow a site to schedule a QPU as a native, cluster-level resource through a short, portable Lua plugin. This post covers why QPUs have not fit the previous scheduling models Slurm used for CPUs and GPUs. It then explains the solution: how HRES models a QPU, and what HRES plugins add in Slurm 26.11, including the API callbacks and capabilities that middleware teams build on. It also addresses how community-driven interfaces such as QRMI and QDMI have already built open source plugins on these APIs, as well as initial adoption by the quantum ecosystem and supercomputing centers. Building quantum-GPU supercomputing Integrating a QPU at a supercomputing center requires more than just its physical installation. The QPU must be networked with the existing computing infrastructure, and users must be able to request it at job submission the same way CPUs and GPUs are requested. Its usage must be tracked, monitored, and accounted for, and it must abide by site-specific policies. In other words, the scheduler, or workload manager, must manage the QPU like it would any other computing resource. Until now, schedulers have had no native way to model a QPU, so most integrations are repeatedly custom-built for each new supercomputing site or QPU builder. The cost of that fragmentation falls on everyone involved: Users face a different interface at every site. Administrators must maintain custom code that complicates upgrades. QPU vendors rebuild their integration for each customer, with no portability. Global systems integrators (GSIs) cannot deliver a repeatable deployment. On the QPU side, open source efforts such as QRMI and QDMI are unifying how such devices are described to, and controlled by, HPC software. What has been missing is a unified interface in the scheduler used by supercomputing centers for integrating QPUs into HPC deployments. Why existing Slurm resource models don’t accommodate QPUs A QPU is typically a cluster-wide accelerator, not part of a node. Slurm generic resources (GRES), the mechanism used for GPUs, belong to individual compute nodes. Modeling a QPU as a GRES binds it to one node, so only jobs on that node can use it. If that node fails, for example, the QPU stays unreachable until an administrator reconfigures the cluster. Many integrations instead combine licenses, which represent cluster-wide resources, for scheduling, and use SPANK plugins for the user interface and node setup. This approach works, but it has several problems: Idle nodes: Without a license, SPANK only acts after Slurm has allocated nodes, so a job can hold compute nodes idle while it waits for the QPU. Two-flag requests: Nothing stops a job from passing device options without holding the license. Multiple QPUs: Hosting several QPUs on one cluster is difficult, because their SPANK plugins compete for the same command-line options. Upgrade risk: In practice, many SPANK plugins tie third-party code to Slurm internal APIs, so every Slurm upgrade risks breaking the integration. No device awareness: Slurm cannot tell on its own when a QPU is recalibrating or offline, so it keeps starting jobs that cannot use the device until it is back online. How HRES plugins in Slurm 26.11 enable integration Slurm 26.11 introduces an HRES plugin interface to address these gaps, so that the HPC and quantum communities can integrate with Slurm without changing Slurm itself. Its first implementation, hres/lua, runs a short Lua script. The plugin reports device health to the scheduler, accepts device options as part of the resource request, and runs setup and teardown code at each stage of the job. A site describes the QPU as a cluster-level HRES in resources.yaml and points it at the plugin. The two files divide the work. resources.yaml holds site-specific details and policies, such as endpoints and which nodes can reach the device, under the system administrator’s control. The plugin holds the device logic, so the community or a QPU vendor can write it once and ship it to every site that deploys Slurm 26.11. Modeling a QPU as a hierarchical resource HRES was added to Slurm 25.05 as a beta feature and is the evolution of licenses. Like a license, an HRES is a count of a resource that does not belong to any single node. Unlike a license, that count is associated with a specific set of nodes: the ones allowed to use it (Figure 1). Slurm 26.11 also starts to allow more dynamic modeling of the relationship between nodes and HRES. An HRES is defined in resources.yaml, in the same directory as slurm.conf. Each resource has one or more layers, and each layer has a name, a list of nodes, and a count. For a QPU, the simplest definition is a single layer covering the nodes that can reach the device, with a count of one. The plugin block attaches the Lua script and its site-specific settings: - resource: qpu mode: MODE_1 plugin: type: hres/lua config: - name: path value: /etc/slurm/hres_qpu.lua - name: endpoint # site settings, never secrets value: https://qpu-gateway.example.org maintenance_cycle: 30 # seconds between health checks layers: - layer_name: qpu1 nodes: ["node[0001-0123]"] count: 1 With this definition, a request is satisfied when a layer that overlaps the job’s allocated nodes has enough of the resource available. A job that requests the QPU runs only on nodes in that layer. Jobs request the resource with --resources=qpu:1. The middleware is the boundary between HPC and QPU In the model shown in Figure 2, neither Slurm nor the Lua plugin talks to the QPU directly. A third-party access middleware sits between the HPC system and the QPU and provides a unified interface to different quantum hardware APIs. The middleware is therefore the boundary between the HPC system and the QPU. Slurm treats the QPU as a single schedulable unit. It decides whether the unit is available, which job holds it, for how long, and from which nodes. Importantly, Slurm does not orchestrate anything inside the unit: calibration, error correction, and device control stay within the QPU and its own software runtime. The approach therefore applies both to today’s noisy intermediate-scale quantum (NISQ) devices and to future error-corrected systems. The plugin translates between the two sides. It asks the middleware for status and reservations, and reports the results to Slurm in Slurm’s terms: whether a layer is disabled, whether a job can start, and a payload for the job’s nodes. HRES plugin APIs The current implementation of HRES plugins runs a Lua script written by the community or a device vendor. A Lua plugin is a plain text file that vendors can ship and update independently of Slurm’s release cycle. It does not link against libslurm, whose API changes each release, so it does not need rebuilding for every Slurm upgrade. Lua is also fast, embeds cleanly in C, and is already used by Slurm’s job submit and CLI filter plugins. The script must sit at the same path on the controller and on every node, and since most of its functions run as root or as SlurmUser, only root should be able to write to it. Key capabilities include: Stackability: slurmctld creates a separate plugin context for each resource that defines a plugin. One cluster can host several QPUs, and several QPU types, without the option collisions that SPANK plugins run into. Site configuration: Endpoints and other settings live in the plugin’s config list, which Slurm hands to the plugin without interpreting it. The configuration, the job’s arguments, and any payload travel with the job to its nodes, where the job’s user can read them. This means that none of them may hold secrets. User inputs: Users append name=value arguments to the request, for example --resources='qpu:1[shot=1000]'. Slurm passes them to the plugin without interpreting them, which removes the need for a second command-line flag. Health checks: Every maintenance_cycle seconds, a maintenance callback asks the middleware whether the QPU is ready, and sets or clears disable_layer on each layer to stop or resume allocation from it. Job start and end: Once a job is scheduled, and before its nodes are allocated, a controller prerun can admit or hold it, so the job never holds idle nodes while it waits for the QPU. Because the prerun holds the controller’s job lock, slow work, such as reserving the QPU, belongs in an optional asynchronous prerun, whose payload Slurm sends to the job’s nodes. If the job does not start, a revoke callback undoes the reservation, and a controller epilog releases it when the job ends. Failure handling: A callback can return ESLURM_HRES_REMOTE_FAIL to report that the external service is unavailable rather than broken. A failed prerun then leaves the job pending, and a failed node prolog or epilog does not drain the node. A plugin implements the callbacks in Table 1, in the order they run for a job. Most integrations need only a few callbacks. The following skeleton shows the shape of a plugin: a health check, an admission decision before any nodes are allocated, and a release when the job ends. The middleware_* calls stand in for the middleware’s own client library. -- Callback names, arguments, fields, and return codes are Slurm's hres/lua API. -- middleware_* calls are placeholders for your middleware's client library. -- Health: slurmctld, every maintenance_cycle seconds function hres_maintenance(ctx, layers) local ready = middleware_status(ctx.config.endpoint) for _, layer in ipairs(layers) do layer.disable_layer = not ready end return slurm.SUCCESS end -- Admission: once the job is scheduled, before its nodes are allocated function hres_controller_job_prerun(ctx, job, prep) if not middleware_can_admit(ctx, prep.plugin_args) then -- cached, no network return slurm.ESLURM_HRES_REMOTE_FAIL -- the job stays pending end return slurm.SUCCESS end -- Release: after the job's node epilogs function hres_controller_job_epilog(ctx, job, prep) middleware_release(ctx.config.endpoint, prep.payload) return slurm.SUCCESS end -- The remaining required callbacks return slurm.SUCCESS. Community integration examples of HRES plugins The teams that build QPU access middleware usually write the HRES plugins. Two of those communities, QRMI and QDMI, have released open source plugins built on the Slurm 26.11 APIs. Both admit jobs only while the QPU is available, and in both, the layer’s count sets how many jobs can use a QPU at the same time. The QRMI plugin acquires access to the QPU for each job before it runs. The QDMI plugin only admits jobs, and each job then queues its work with the provider. With either plugin, sites running Slurm 26.11 can connect their clusters to quantum computers from multiple providers. QRMI: Acquiring the QPU for each job Each HRES resource is named after a resource in the QRMI configuration file. Before a job runs, the plugin books access to the QPU with the user’s own credentials, in a separate thread outside the scheduler’s critical path, and releases it after the job ends. Depending on the backend, the booking is a reservation or a session. Existing applications run unchanged, because jobs see the same environment variables as with the QRMI SPANK plugin. For the complete configuration, plugin and a containerized test cluster, visit the QRMI HRES plugin repository. QDMI: Leaving the queue to the provider Each HRES resource is one device, named after its QDMI device ID. The plugin’s maintenance callback checks the device’s status and admits jobs only while it reports itself as idle or busy. The plugin reserves nothing: each job submits its work through QDMI, and the provider’s queue schedules it on the device. For the complete configuration and plugin, visit the MQSC Orchestrator. Learn how more organizations in the quantum ecosystem are integrating HRES plugins: Alice & Bob Quantinuum User job submission From the user’s side, the two integrations look and feel the same. A job script requests the QPU by its HRES resource name just as it requests nodes or GPUs, and runs in the partition that contains the HRES assigned nodes. Slurm holds the job until the plugin reports that the QPU is available and, where the middleware supports it, reserved. The template below works for either integration; only the resource name changes. #!/bin/bash #SBATCH --job-name=my-hybrid-job #SBATCH --nodes=4 #SBATCH --time=00:30:00 #SBATCH --resources= srun ./my_hybrid_app Get started with Slurm HRES plugins Slurm 26.11 gives QPUs a native place in the scheduler. An HRES models the QPU as a cluster-level resource tied to the nodes that can reach it, and a Lua plugin connects that resource to the middleware that manages the device. Because the Lua plugin is designed to hold little to nothing site-specific, a vendor or community project can write it once and every Slurm site can reuse it, while administrators keep control of their own policies. By design, this approach is valid for today’s generation of quantum computers and for future fault-tolerant systems, making it a lasting choice for supercomputing centers deploying quantum devices. Ready to get started? Deploy the QRMI HRES plugin or the QDMI HRES example, or use them as starting points for your own integration. To learn more, check out the Slurm 26.11 release notes, and read the HRES documentation and the resources.yaml reference. Acknowledgments We thank the QRMI and QDMI communities for building on HRES plugins and for their contributions to this post. About the Authors Comments

资料来源

阅读原文 ↗

相关主题

← 继续阅读全部动态