Docs

Command Line Interface (CLI)

We provide a command-line interface (CLI) for Marblerun. This CLI allows you to install Marblerun on your cluster and interacts with the control plane through the Client API for all administrative tasks in the service mesh.

To install the Marblerun CLI on your machine you can use our pre-built binaries.

For the current user

wget -P ~/.local/bin/marblerun https://github.com/edgelesssys/marblerun/releases/latest/download/marblerun
chmod +x ~/.local/bin/marblerun

Global install (requires root)

sudo wget -O /usr/local/bin/marblerun https://github.com/edgelesssys/marblerun/releases/latest/download/marblerun
sudo chmod +x /usr/local/bin/marblerun

To build the Marblerun CLI, Edgeless RT needs to be installed on your machine.

go build -o marblerun github.com/edgelesssys/marblerun/cli

To list all available commands, either run marblerun with no commands or execute marblerun help The output is the following:

Usage:
  marblerun [command]

Available Commands:
  certificate      Retrieves the certificate of the Marblerun coordinator
  check            Check the status of Marbleruns control plane
  graphene-prepare Modifies a Graphene manifest for use with Marblerun
  help             Help about any command
  install          Installs marblerun on a kubernetes cluster
  manifest         Manages manifest for the Marblerun coordinator
  namespace        Manages namespaces associated with Marblerun installations
  precheck         Check if your kubernetes cluster supports SGX
  recover          Recovers the Marblerun coordinator from a sealed state
  status           Gives information about the status of the marblerun Coordinator
  uninstall        Removes Marblerun from a kubernetes cluster
  version          Display version of this CLI and (if running) the Marblerun coordinator

Flags:
  -h, --help   help for marblerun

Use "marblerun [command] --help" for more information about a command.

Command certificate

Get the root and/or intermediate certificates of the Marblerun coordinator.

Flags These flags apply to all certificate subcommands

Name, shorthandDefaultDescription
–era-configPath to remote attestation config file in json format, if none provided the newest configuration will be loaded from github
–help, -hhelp for certificate
–insecure, -isimulation mode
–output, -oFile to save the certificate to
  • root

    Gets the root certificate of the Marblerun coordinator.

    Usage

    marblerun certificate root <IP:PORT> [flags]
    
  • intermediate

    Gets the intermediate certificate of the Marblerun coordinator.

    Usage

    marblerun certificate intermediate <IP:PORT> [flags]
    
  • chain

    Gets the certificate chain of the Marblerun coordinator.

    Usage

    marblerun certificate chain <IP:PORT> [flags]
    

Command check

Check the status of Marbleruns control plane. This command will check if the Marblerun coordinator and/or the Marblerun webhook are deployed on a Kubernetes cluster and wait until all replicas of the deployment have the available status.

Usage

marblerun check

Flags

Name, shorthandDefaultDescription
–timeout60Time to wait before aborting in seconds

Command graphene-prepare

This command helps you if you want to add Graphene-based services to your Marblerun service mesh. It prepares your Graphene project to be used as a Marble. Given your Graphene manifest template, it will suggest the required adjustments needed and adds our bootstrapping data-plane code to your Graphene image. See Building a service: Graphene for detailed information on Marblerun’s Graphene integration and our changes in your Graphene manifest.

Please note that this only works on a best-effort basis and may not instantly work correctly. While suggestions should be made for every valid TOML Graphene configuration, changes can only be performed for non-hierarchically sorted configurations. as the official Graphene examples. The unmodified manifest is saved as a backup under the old path with an added “.bak” suffix, allowing you to try out and roll back any changes performed.

Remember, you need to create a Marblerun manifest in addition to the Graphene manifest. Adding Graphene packages to your manifest is straightforward and follows the same principles as any other SGX enclave.

This command supports two modes, spawn and preload.

  • spawn

    Replaces the original entrypoint of your application with the bootstrapping Marble premain process which eventually spawns your application. Dedicates argv provisioning to Marblerun. If you configured the arguments to your Graphene application through the Graphene manifest before, you need to transfer those to the Marblerun manifest.

    Usage

    marblerun graphene-prepare spawn <path>
    

    Examples

    marblerun graphene-prepare spawn nginx.manifest.template
    

    Output:

    Reading file: nginx.manifest.template
    
    Marblerun suggests the following changes to your Graphene manifest:
    libos.entrypoint = "file:premain-graphene"
    loader.argv0_override = "$(INSTALL_DIR)/sbin/nginx"
    loader.insecure__use_host_env = 1
    sgx.allowed_files.marblerun_uuid = "file:uuid"
    sgx.enclave_size = "1024M"
    sgx.remote_attestation = 1
    sgx.thread_num = 16
    sgx.trusted_files.marblerun_premain = "file:premain-graphene"
    Do you want to automatically apply the suggested changes [y/n]? y
    Applying changes...
    Saving original manifest as nginx.manifest.template.bak...
    Saving changes to nginx.manifest.template...
    Downloading Marblerun premain from GitHub...
    Successfully downloaded premain-graphene.
    
    Done! You should be good to go for Marblerun!
    
  • preload

    Adds the premain as a shared library loaded during the launch of your application via LD_PRELOAD. Allows for a faster launch than spawn and delegates more features to Graphene (but restricts Marblerun’s functionalities), making it the quickest way to adapt your existing application. However, features such as argv provisioning cannot be used in Marblerun anymore in this mode.

    Usage

    marblerun graphene-prepare preload <path>
    

    Examples

    marblerun graphene-prepare preload nginx.manifest.template
    

    Output:

    Reading file: nginx.manifest.template
    
    Marblerun suggests the following changes to your Graphene manifest:
    loader.env.LD_PRELOAD = "./premain-graphene.so"
    loader.insecure__use_host_env = 1
    sgx.allowed_files.marblerun_uuid = "file:uuid"
    sgx.enclave_size = "1024M"
    sgx.remote_attestation = 1
    sgx.thread_num = 16
    sgx.trusted_files.marblerun_premain = "file:premain-graphene.so"
    Do you want to automatically apply the suggested changes [y/n]? y
    Applying changes...
    Saving original manifest as nginx.manifest.template.bak...
    Saving changes to nginx.manifest.template...
    Downloading Marblerun premain from GitHub...
    Successfully downloaded premain-graphene.so.
    
    Done! You should be good to go for Marblerun!
    

Command install

Install Marblerun on your Kubernetes cluster. This command will add Marblerun to your local helm repository if it is not present yet, optionally you can provide a path to your own helm chart.

Usage

marblerun install [flags]

Flags

Name, shorthandDefaultDescription
–client-server-port4433Set the client server port. Needs to be configured to the same port as in your client tool stack
–disable-auto-injectionDisable automatic injection of selected namespaces
–domainlocalhostSets the CNAME for the coordinator certificate
–help, -hhelp for install
–marblerun-chart-pathPath to marblerun helm chart
–mesh-sever-port2001Set the mesh server port. Needs to be configured to the same port as in the data-plane marbles
–no-sgx-device-pluginDisables the installation of an sgx device plugin
–simulationSet Marblerun to start in simulation mode, needed when not running on an SGX enabled cluster

Examples

  • Install Marblerun on a cluster with SGX Support

    marblerun install --domain=mycluster.uksouth.cloudapp.azure.com
    

    The output is similar to the following:

    Did not find marblerun helm repository on system, adding now...
    edgeless has been added to your helm repositories
    Setting up Marblerun Webhook... Done
    Marblerun installed successfully
    
  • Install Marblerun on a cluster without SGX Support (simulation mode)

    marblerun install --simulation
    

    The output is similar to the following:

    Setting up Marblerun Webhook... Done
    Marblerun installed successfully
    

Command manifest

Set or update a manifest, or retrieve the signature of the manifest in place.

Flags These flags apply to all sub commands of manifest

Name, shorthandDefaultDescription
–era-configPath to remote attestation config file in json format, if none provided the newest configuration will be loaded from github
–help, -hhelp for manifest
–insecure, -isimulation mode
  • set

    Uploads a manifest in json or yaml format to the Marblerun coordinator. If a recovery key was set in the manifest, a recovery secret will be sent back.

    Usage

    marblerun manifest set <manifest.json> <IP:PORT> [flags]
    

    Flags

    Name, shorthandDefaultDescription
    –recovery-data, -rFile to write recovery data to, print to stdout if not set

    Examples

    marblerun manifest set manifest.json $MARBLERUN --recovery-data=recovery-secret.json --era-config=era.json
    

    The output is similar to the following:

    Successfully verified coordinator, now uploading manifest
    Manifest successfully set, recovery data saved to: recovery-secret.json
    
  • update

    Update a manifest by uploading an update manifest to the Marblerun coordinator. The original manifest has to define one or multiple Admins who are allowed to update the manifest. For more information see Update

    Usage

    marblerun manifest update <manifest.json> <IP:PORT> --cert=admin-cert.pem --key=admin-key.pem [flags]
    

    Flags

    Name, shorthandDefaultDescription
    –cert, -cPEM encoded admin certificate file (required)
    –key, -kPEM encoded admin key file (required)

    Examples

    marblerun manifest update update-manifest.json $MARBLERUN --cert=admin-cert.pem --key=admin-key.pem --era-config=era.json
    

    The ouput is the following:

    Successfully verified coordinator, now uploading manifest
    Manifest successfully updated
    
  • get

    Retrieves the signature of an uploaded manifest. This allows a user to verify what manifest is running on the coordinator.

    Usage

    marblerun manifest get <IP:PORT> [flags]
    

    Flags

    Name, shorthandDefaultDescription
    –output, -osignature.jsonDefine file to write to

    Examples

    marblerun manifest get $MARBLERUN --output=manifest-signature.json --era-config=era.json
    

    The output is the following:

    Successfully verified coordinator, now requesting manifest signature
    Manifest written to: manifest-signature.json
    
  • signature

    Print the signature of a Marblerun manifest. The manifest can be in either json or yaml format.

    Usage

    marblerun manifest signature manifest.json
    

    The output is the sha256 hash in base64 encoding of the manifest as it would be interpreted by the Marblerun coordinator. Note, that Internally, the coordinator handles the manifest in JSON format. Hence, the signature is always based on the JSON format of your manifest. You can quickly verify the integrity of the installed manifest by comparing the output of marblerun manifest signature on your local version and the signature returned by marblerun manifest get of the coordinator’s version.

Command namespace

Add namespaces to Marblerun. If the auto-injection feature is enabled. All new pods in those namespaces will get their Marblerun configuration automatically injected.

  • add

    Add a namespace to the Marblerun mesh by creating a new label

    Usage

    marblerun namespace add NAMESPACE ... [flags]
    

    Flags

    Name, shorthandDefaultDescription
    –inject-sgxSet to enable automatic injection of SGX tolerations for namespace

    Examples

    marblerun namespace add default testspace
    

    The output is the following:

    Added namespace [default] to Marblerun mesh
    Added namespace [testspace] to Marblerun mesh
    
  • remove

    Remove a namespace from the Marblerun mesh

    Usage

    marblerun namespace remove NAMESPACE [flags]
    

    Examples

    marblerun namespace remove default
    

    The output is the following:

    Namespace [default] succesfully removed from the Marblerun mesh
    
  • list

    List all namespaces currently associated with the Marblerun mesh

    Usage

    marblerun namespace list
    

    Examples

    marblerun namespace list
    

    The output is the following:

    testspace
    

Command precheck

Check if your kubernetes cluster supports SGX. More precisely the command will check if any nodes in the cluster define SGX resources through the use of Device Plugins. Currently supported are:

Usage

marblerun precheck
  • If your cluster does not support SGX the output is the following:
Cluster does not support SGX, you may still run Marblerun in simulation mode
To install Marblerun run [marblerun install --simulation]
  • If you cluster does support SGX the output is similar to the following
Cluster supports SGX on 2 nodes
To install Marblerun run [marblerun install]

Command recover

Recover the Marblerun coordinator from a sealed state by uploading a recovery key. For more information about coordinator recovery see Recovery

Usage

marblerun recover <recovery_key_decrypted> <IP:PORT> [flags]

Flags

Name, shorthandDefaultDescription
–era-configPath to remote attestation config file in json format, if none provided the newest configuration will be loaded from github
–help, -hhelp for recover
–insecure, -iSet to skip quote verification, needed when running in simulation mode

Examples

marblerun recover $MARBLERUN recovery_key_decrypted --era-config=era.json

The output is similar to the following:

Successfully verified coordinator, now uploading key
Successfully uploaded recovery key and unsealed the Marblerun coordinator

Command status

Checks on the current status of the coordinator.

Usage

marblerun status <IP:PORT> [flags]

Flags

Name, shorthandDefaultDescription
–era-configPath to remote attestation config file in json format, if none provided the newest configuration will be loaded from github
–help, -hhelp for status
–insecure, -iSet to skip quote verification, needed when running in simulation mode

Examples

marblerun status $MARBLERUN

The output is similar to the following:

No era config file specified, getting latest config from github.com/edgelesssys/marblerun/releases/latest/download/coordinator-era.json
Got latest config
2: Coordinator is ready to accept a manifest.

Command uninstall

Remove Marblerun from your kubernetes cluster. This command will remove all resources added by the installation command.

Usage

marblerun uninstall

The output is the following:

Marblerun successfully removed from your cluster

Command version

Display version information of CLI, and the Marblerun coordinator running on a Kubernetes cluster.

Usage

marblerun version

The output is similar to the following:

CLI Version: v0.3.0
Commit: 689787ea6f3ea3e047a68e2d4deaf095d1d84db9
Coordinator Version: v0.3.0