Allocating Kernel Resources
Allocating Kernel Resources
Wolfram Enterprise Private Cloud (EPC) handles many types of workloads, from small, focused APIs to complex real-world simulations. Every scenario places its own set of demands on system resources, so administrators are given great flexibility when it comes to configuring their clouds for optimal performance.
One of the most important configuration decisions is the allocation of kernel resources. EPC provides three principal kernel pools for handling different types of computations: the session kernel pool, the deployment kernel pool and the task kernel pool. In addition, an EPC instance can optionally be configured to allocate one or more data paclet kernels.
Each license comes with a number of kernels that can be configured to meet your needs. At least one kernel should be assigned for each of the principal kernel types, and the remainder should be distributed according to business objectives. The following sections describe the function of each pool and include some recommendations about how to allocate them. Although the configuration notebook contains a default allocation, administrators should determine the best allocation according to the business objectives.
Session Kernel Pool
Session Kernel Pool
Session kernels (also known as general kernels) are used for interactive notebook sessions through the web portal, and they are reserved for the duration of the session.
Session kernels (also known as general kernels) are used for interactive notebook sessions through the web portal, and they are reserved for the duration of the session.
The session kernel pool supports fully interactive, stateful evaluation of Wolfram Language code in a cloud notebook session. These kernels are initialized whenever a user signs in and opens a notebook using the cloud, in a manner analogous to the traditional desktop experience. As with the desktop, evaluation history and state accumulate and are preserved for as long as the user continues to interact with the cloud notebook. Each distinct user is assigned a dedicated kernel from the pool, making it unavailable to other users for the duration of that user’s session. The total number of kernels in the session kernel pool dictates the maximum number of simultaneous interactive users that the server can support: the more users are working in parallel, the more general kernels are needed.
Example
Example
In a development organization with a license for 16 kernels expecting a high concurrency of EPC users logged on, most of the kernels should be assigned to the session pool. See slide 7 of the configuration notebook:
$CloudConfiguration["KernelPools"]= <| "SessionPool"->13, "DeploymentPool"->2, "ServicePool"->1 |>;
Because users working in a notebook generally spend more time creating functions for evaluation than expending CPU time to obtain results, the operation of session kernels is usually constrained by memory rather than by available processing power. Thus it often makes sense to allocate more kernels to the session pool than there are actual processor cores available to handle evaluations. However, interactive sessions often use significant blocks of memory, and the memory consumption rate can be expected to grow over time. As a conservative rule of thumb, one should not allocate more session kernels than can be accommodated by the memory available to the servers—after accounting for the resources used by other kernel pools, the cloud application itself and other system processes.
Every notebook opened by a user also creates a server-side instance of that notebook. These consume memory inside the Tomcat application, which may need extra memory to serve large numbers of notebooks. This reduces the likelihood of system memory being exhausted when the server is at full capacity, but close monitoring of memory consumption is recommended. Adjustments to the pool size should be made based on actual usage experience.
Deployment Kernel Pool
Deployment Kernel Pool
Deployment kernels (also known as public kernels) are used for evaluations associated with deployed cloud objects (e.g. APIs, forms, visualizations) and for CloudEvaluate. They are activated on demand and then released back into the kernel pool after evaluation is completed.
Deployment kernels (also known as public kernels) are used for evaluations associated with deployed cloud objects (e.g. APIs, forms, visualizations) and for CloudEvaluate. They are activated on demand and then released back into the kernel pool after evaluation is completed.
The deployment kernel pool processes those Wolfram Language evaluations that are triggered by user requests to deployed objects. Typically, these requests originate in API functions, forms and delayed computation objects, but can also come from cloud notebooks accessed in the /obj view and containing dynamic content. In the case of APIs, a kernel is allocated from the deployment pool to evaluate the associated Wolfram Language computation, and then returned to the pool when the result is computed. Thereafter, that kernel is available to serve any other pending request from any other user.
Deployed notebooks containing dynamic content (such as Manipulate objects, resizable graphics and rotatable 3D graphics) also use kernels from the deployment pool to compute updates. A kernel is allocated to handle the computation and is released back to the pool once the result is calculated. Note also that when a dynamic object, e.g. a Manipulate expression, is deployed, a new notebook is synthesized to contain it. Such interactions therefore also use the deployment pool.
Deployed notebooks do not offer full + evaluation unless they are intentionally opened by the user into his or her own interactive session (e.g. using the Make Your Own Copy button), in which case the notebook engages a session kernel for that user instead of pulling from the deployment pool.
Because kernels in the deployment pool are used for one-and-done computation and may be rapidly recycled among requests, the size of the deployment pool is typically limited by the available processing power of the server. To avoid significant competition for cores when handling requests, the size of this pool should not greatly exceed the number of processor cores on the server. That number may need to be reduced further if the server has a mixed load, or if there are insufficient computational resources for the cloud app itself or for other system processes.
Example
Example
In supporting a deployed API or app (with a license for 32 kernels), the majority of the kernels can be assigned to the deployment pool. See slide 7 of the configuration notebook:
$CloudConfiguration["KernelPools"]= <| "SessionPool"1, "DeploymentPool"30, "ServicePool"->1 |>;
Task Kernel Pool
Task Kernel Pool
Task kernels (also known as service kernels) are used for automated computations executed at set intervals, generally invoked by ScheduledTask. They are activated on demand and then released back into the kernel pool after evaluation is completed.
The task kernel pool is used to process computations associated with scheduled tasks, continuous tasks, automated reports and other automated operations. These computations can be scheduled by both the system and users. Since they occur at predictable intervals, they are not susceptible to variation based on external user demand (barring an extreme use of CloudSubmit). Although processing and memory resources should be considered when provisioning this queue, a significant factor to consider is how much delay is acceptable for processing any scheduled task. When tasks come due, they are added to a queue to be processed by the next free service kernel. A larger pool of service kernels will ensure that tasks are handled more promptly if many are scheduled to execute in a short time period. If latency in the queue is not a significant issue, a smaller pool can be sufficient. At minimum, a pool must have enough kernels to eventually process all the work available before more tasks are scheduled to run.
Example
Example
In supporting a server that generates automated reports (with a license for eight kernels), most of the kernels can be assigned to the service pool. See slide 7 of the configuration notebook:
$CloudConfiguration["KernelPools"]= <| "SessionPool"->1, "DeploymentPool"1, "ServicePool"6 |>;
Data Paclet Kernels
Data Paclet Kernels
A fourth type of kernel, the data paclet kernel, is unusual in that it is not accounted for in the kernel allocation association discussed previously. A data paclet kernel is invoked at setup time like standard kernel types; however, instead of being used directly for computations, it is pre-loaded with commonly retrieved items such as CityData, WordData, ChemicalData, GenomeData, ProteinData, GraphData and some Entity lookups. Assigning a data paclet kernel allows other kernels to route data paclet requests to it, sharing the data to conserve memory and improve paclet retrieval times.
In the default configuration, the data paclet kernel count is 0. If you elect to set it to 1, be aware that this will leave one less kernel to be assigned to one of the main “pools.” There is no need to allocate more than one data paclet kernel.
To dedicate an available kernel to the data paclet kernel pool, open the file /wolframcloud/webapp/WEB-INF/MSPConfiguration.xml. There are four KernelPool configuration nodes in this file. Scroll through the first three, then look for the one with the KernelPoolName of DataPaclets:
