Policies

Policies are used to program how and where your workloads are deployed on Beamlit. Policies can be defined as code, allowing for easy programming and customization of your Global Inference Network.

Policies apply to the entities they are attached to. They can be attached to individual model deployments, or to a whole environment to enforce them across your workspace.

​

Policies overview

Policies essentially describe rules as to how deployments and executions are made on Beamlit.

A policy states all the allowed options for a specific aspect (called the policy type) of the deployment or execution (for example: the execution location).

Example:

• Policy Country: US means that attached workloads will only be able to run in locations that are in the United States.

When no policies are enforced on a type, all options for this type are considered allowed. Workloads are executed using Global Inference Network’s default optimizations.

​

Policy types

Policies have a type, allowing multiple policies to drive various deployment strategies without colliding. Typically, you can easily enforce a policy on the execution location and a policy on the underlying hardware at the same time.

There are currently two types of policies: location-based and flavor-based.

​

Location policies

Location policies give control over which clusters will execute your workloads.

They come in two different formats:

• policies on countries allow to define all physical locations inside of one or several countries at once
  • for example, execute only in the following country: USA
• policies on continents allow to define all physical locations inside of one or several continents at once
  • for example, execute only in North America

Refer to our reference on policy locations below for more information.

​

Flavor policies

Flavor policies give control over which underlying accelerator (GPU) your workloads will be executed on.

They come in two different formats:

• policies on cpu allow to pass a specific list of CPU types
  • for example, execute only on x86
• policies on gpu allow to pass a specific list of GPU types
  • execute only on NVIDIA A100
  • execute only on NVIDIA L4 or NVIDIA T4

Refer to our reference on policy flavors below for more information.

​

Policies target

You can restrict the scope of objects affected by a policy through targets. This is particularly useful when you want to attach a policy to an environment without impacting all resources that depend on it.

Targets specify the resource types the policy should influence. For instance, if you want to enforce the use of specific GPUs in the production environment for models, but allow functions to run on CPU, you can define a target on models for that policy.

By default, a policy without a specified target applies to all resource types.

​

Create a policy

Policies can be created from the Beamlit console, or from the Beamlit APIs and CLI.

Read our complete reference on policies.

​

Attach a policy

Attaching a policy to a workload enforces it on the workload. Policies can be attached either to environments, or to specific model deployments (provided they do not collide with the environment-level policies).

​

Attaching multiple policies

When attaching multiple policies to a resource, it’s crucial to understand their combined effect. This effect varies depending on whether you’re attaching multiple policies to the same workload (e.g., one environment, or one model deployment), or applying policies to a workload that already inherits policies from its linked environment.

If you are attaching multiple policies to the same resource.

In this situation, their combined effect is the UNION of all of their effects for the same type of policy (a.k.a OR clause), and INTERSECTION across all types of policies (a.k.a AND clause).

If you are attaching policies to a workload directly that already inherits policies from a linked environment

In this situation, a user cannot allow more options on a policy type regulated by environment-level policies by using deployment-specific policies. In other words, the combined effect is the INTERSECTION of all of their effects for the same type.

The combined effect of policies of different types is the same as if all those policies where attached to the same resource (cf. above)

For example:

• Let’s assume the following policies:
  • Policy A: Country is: USA
  • Policy B: Continent is: North America, or Europe
  • Policy C: GPU is: NVIDIA T4
• if a single environment has the following combined policies:
  • A and B: then workloads will only execute in any location in either North America (including USA) or Europe — on any kind of hardware available there.
  • B and C: then workloads will only execute in any location in either North America or Europe, only on T4 GPUs.
  • A and C: then workloads will only execute in any location in the USA, only on T4 GPUs.
  • A and B and C: then workloads will only execute in any location in either North America (including USA) or Europe — only on T4 GPUs.
• If an environment has certain policies, and additional policies are attached directly to a deployment linked to that environment:
  • Environment (policy A) and deployment (policy B): then workloads will only execute in any location in the USA — on any kind of hardware available there.
  • Environment (policy B) and deployment (policy A): same result
  • Environment (policy B) and deployment (policy C): then workloads will only execute in any location in either North America or Europe, only on T4 GPUs.
  • Environment (policy A) and deployment (policies B and C): then workloads will only execute in any location in the USA, only on T4 GPUs.

​

Attaching a policy to an environment

Attaching a policy to an environment enforces it on all workloads that are deployed on this environment, and that are in the scope of the target of the policy.

A platform admin can thus apply policies at the environment level, ensuring they will be enforced for all deployments linked to this environment. This approach prevents other platform users from escalating their privileges.

This is because as mentioned above, combining of policies at environment-level and policies at deployment-level has the following result:

​

Attaching a policy to a model deployment

Attaching a policy to a model deployment directly enforces it on this specific model deployment, provided it is in the scope of the target of the policy.

If the deployment is already inheriting policies from its environment, the resulting effect is as described above. You will not be able to allow more options than what is restricted at environment level.

​

Policy reference

Below is the list of official names to build policies.

​

Flavors

Code	Type	Flavor Name
CPU x86	cpu	x86
t4	gpu	NVIDIA T4

​

Locations

Code	Type	Name
eu	continent	Europe
na	continent	North America
us	country	United States
fr	country	France