On-premises Marblerun deployment
This guide walks you through setting up Marblerun for your on-premises deployment.
Hardware and firmware
To deploy Marblerun with Intel SGX, the machine or VM has to support Intel SGX. Particularly, Marblerun requires support for the SGX Datacenter Attestation Primitives (DCAP). You can verify if your CPU supports DCAP.
For more information read the Properly Detecting Intel Software Guard Extensions (Intel® SGX) in Your Applications article.
BIOS support is required for Intel SGX to provide the capability to enable and configure the Intel SGX feature in the system. Currently, most of the SGX capable systems have SGX disabled by default in the BIOS. This default setting might change but for now, you need to manually enable it if it’s not already enabled.
As with any modern technology, Intel SGX has been affected by security vulnerabilities. Intel addresses these vulnerabilities by updating the microcode of CPUs, changing the hardware of new CPUs, and updating the system software. Each microcode update that patches an SGX vulnerability requires a BIOS update. During remote attestation, it is checked that the microcode of the CPU which is deployed by the BIOS is up to date. The microcode and platform enclaves are commonly called the platform
Trusted Computing Base (TCB).
If your BIOS/firmware is outdated, you will see errors as
Platform TCB (2) is not up-to-date (oe_result_t=OE_TCB_LEVEL_INVALID) during remote attestation procedures.
If you are using VMs for your Marblerun deployment, you need to make sure your hypervisor has SGX enabled. Most of the popular hypervisors support SGX:
You need to install the SGX Driver for systems supporting the Launch Control Configuration. Azure provides the instructions on how to install this driver that you can use for your on-premises machines.
SGX Datacenter Attestation Primitives (DCAP)
DCAP is the new attestation mechanism for SGX replacing EPID. You can find an overview of DCAP in the official Intel docs. Marblerun only supports DCAP and requires DCAP libraries installed and configured on your system.
From the perspective of Marblerun and your workloads DCAP is accessed with a Quote Generation Library (QGL) and a Quote Verification Library (QVL) for generating and verifying quotes respectively. The QGL and QVL libraries need to be configured to talk to a Provisioning Certificate Caching Service (PCCS). You currently have two options regarding PCCS for your on-premises machines and clusters:
Use a public PCCS service by configuring your QGL and QVL to point to the public endpoints. Currently, only Azure provides such a service that can be used by installing the Azure-DCAP-Client package on your system:
echo 'deb [arch=amd64] https://download.01.org/intel-sgx/sgx_repo/ubuntu bionic main' | sudo tee /etc/apt/sources.list.d/intel-sgx.list wget -qO - https://download.01.org/intel-sgx/sgx_repo/ubuntu/intel-sgx-deb.key | sudo apt-key add - echo "deb [arch=amd64] https://packages.microsoft.com/ubuntu/18.04/prod bionic main" | sudo tee /etc/apt/sources.list.d/msprod.list wget -qO - https://packages.microsoft.com/keys/microsoft.asc | sudo apt-key add - sudo apt update && sudo apt install -y libsgx-dcap-ql libsgx-dcap-ql-dev az-dcap-client
Follow these steps to set up your machines for your PCCS:
The PCCS is a cache, so you need to make sure it stays up to date. In case your cache is outdated, you might see error messages as:
coordinator-enclave.signed:mbedtls_x509_crt_verify failed with The CRL is expired (flags=0x20) (oe_result_t=OE_VERIFY_CRL_EXPIRED)
You can inspect the Intel Root CA CRL of your PCCS:
curl --insecure --request GET --url https://<YOUR_PCCS_DOMAIN>:<YOUR_PCCS_PORT>/sgx/certification/v2/rootcacrl > rootca.crl openssl crl -inform PEM -text -noout -in rootca.crl
You can refresh all SGX collaterals for your PCCS:
curl --insecure --request GET -H "admin-token: <my password>" --url https://<YOUR_PCCS_DOMAIN>:<YOUR_PCCS_PORT>/sgx/certification/v2/refresh
If refreshing CRL fails, you can manually delete the
pckcache.dbdatabase (default location
/opt/intel/sgx-dcap-pccs/pckcache.db) and restart your PCCS.
NoteCurrently, the Marblerun Coordinator and EGo docker images are configured to always use the public Azure PCCS. In the future, we will make this configurable, so you can define your own PCCS for our images.
You have made sure your hardware supports SGX, updated all firmware, installed the SGX driver, and configured DCAP on all your machines and VMs? Great! Now it is time to install Marblerun and get going.
You can either follow our guide and use Marblerun in standalone mode .
Or you follow our instructions to install Marblerun in your Kubernetes cluster.