Usage Basics

Ciris is a configuration as code library for compile-time safe configurations. This means that your configuration models and data is part of your code, in contrast to configuration files (and libraries like Lightbend Config), where the configuration data resides in configuration files with specialized syntax. Writing configurations as code is most often an easy, safe, and secure alternative to configuration files, suitable in situations where it’s easy to change and deploy software. But before we begin to explore Ciris and configurations as code, let’s take a brief moment to discuss why and when you might consider configuration files, and when configurations as code could be a better fit.

Consider configuration files in the following situation.

  • You want to be able to change most configuration values without releasing a new version of the software. Often these changes are made separate to the version control repository of your software. This can be desirable if it’s difficult to release and deploy software, or if you’re not sure in which environments the software will run. End-users or stakeholders might need to be able to configure certain aspects of your software, and recompiling the software to make configuration changes is not a viable alternative.

Consider configuration as code in the following situations.

  • You want to load configuration values from various sources in a uniform way. With configuration as code, and Ciris specifically, you can load values from, for example, both environment variables and vault services in the same way. With configuration files, you might still need to fetch secret values from a vault service as a separate ad-hoc step.
  • You want a single place for your configuration, including multiple environments, while avoiding duplicated values. Configuration as code gives you the ability to deal with multiple environments explicitly, making it clear when and what is different across environments, while avoiding any sort of duplication of configuration values.
  • You want to check that the constant values in your configuration are valid at compile-time. Constant configuration values are literals in code, and will therefore be checked at compile-time. This is especially powerful when used together with refinement types. This means you no longer have to rely on tests to confirm that your default configuration values are valid and useable.
  • You want unused configuration values to be easy to spot. With configuration as code, unused configuration values are dead code, making it easy to spot when certain values are not being used anymore. In contrast with configuration files, unused configuration values are often harder to spot, and get noticed first much later on.
  • You want to avoid having to use a separate configuration syntax, and would rather just write code. There’s no need to learn and use a different syntax, and there is no need to have a library for parsing that configuration file syntax. Having configurations as code also means you’re able to refactor your configurations with more confidence.
  • You want a flexible configuration loading process, without being limited to a configuration syntax. Due to the limited nature of configuration files and their syntax, some configuration loading is more inherently difficult, or even impossible, to express. With Ciris and configuration as code, you have more flexibility around configuration loading.

While it’s possible to not use any libraries when writing configurations as code, loading values from the environment typically means dealing with: different environments and configuration sources, type conversions, error handling, and validation. This is where Ciris comes in: a small library, dependency-free at its core, helping you to deal with all of that.

Configuration Values

Ciris includes functions env (for reading environment variables), prop (for reading system properties), and file and fileWithName (for reading file contents). In a similar fashion, if you would be using any of the external libraries like ciris-kubernetes or ciris-aws-ssm, they respectively provide functions secret (for reading Kubernetes secrets) and param (for reading AWS SSM parameters). These functions have in common that they accept a type to which to convert the value, for example String or Int, and then the key to read (or in case of file and fileWithName, the file which contents should be read). The result is a key-value pair, represented by ConfigEntry, like in the following example.

import ciris.{env, prop, file}
// import ciris.{env, prop, file}

// Read environment variable LANG as a String
env[String]("LANG")
// res1: ciris.ConfigEntry[ciris.api.Id,String,String,String] = ConfigEntry(LANG, Environment)

// Read system property file.encoding as a String
prop[String]("file.encoding")
// res3: ciris.ConfigEntry[ciris.api.Id,String,String,String] = ConfigEntry(file.encoding, Property)

// Read the file, trim the file contents, and convert to Int
file[Int](tempFile, _.trim)
// res5: ciris.ConfigEntry[ciris.api.Id,(java.io.File, java.nio.charset.Charset),String,Int] = ConfigEntry((/var/folders/ff/tg7g7zh52_g9s5_djb5z81w00000gn/T/temp-256420455339564896.tmp,UTF-8), ConfigKeyType(file))

Ciris handles errors when reading values, for example if the environment variable or file doesn’t exist, or if the value couldn’t be converted to the specified type. In the background, these functions are loading values from a configuration source (represented by ConfigSource) and converting the value to the specified type with a configuration decoder (represented by ConfigDecoder). For a list of currently supported types, refer to the current supported types section.

If you want a value to be optional, you can use Option.

// Read environment variable FILE_ENCODING as a String
// but if it has not been set, return None instead of
// an error saying it is missing
val fileEncoding = env[Option[String]]("FILE_ENCODING")
// fileEncoding: ciris.ConfigEntry[ciris.api.Id,String,String,Option[String]] = ConfigEntry(FILE_ENCODING, Environment)

// The unmodified source value is available in the entry,
// and here we can see that the environment variable has
// not been set
fileEncoding.sourceValue
// res12: ciris.api.Id[Either[ciris.ConfigError,String]] = Left(MissingKey(FILE_ENCODING, Environment))

// We get None back as the value, since the environment
// variable has not been set
fileEncoding.value
// res15: ciris.api.Id[Either[ciris.ConfigError,Option[String]]] = Right(None)

// If the key has been set, but could not be decoded
// to the specified type, we keep the error as it is
prop[Option[Int]]("file.encoding").value
// res18: ciris.api.Id[Either[ciris.ConfigError,Option[Int]]] = Left(WrongType(file.encoding, Property, Right(UTF8), UTF8, Int, java.lang.NumberFormatException: For input string: "UTF8"))

Alternatively, you can use orElse to fall back to other values if keys are missing.
Note that you do not have to specify the type to decode again in the orElse.

// Uses the value of the file.encoding system property as
// the FILE_ENCODING environment variable has not been set
env[String]("FILE_ENCODING").
  orElse(prop("file.encoding"))
// res21: ciris.ConfigValue[ciris.api.Id,String] = ConfigValue$1288683671

When using orElse, we get a ConfigValue back, since we’ve combined the values of multiple ConfigEntrys.

You can also combine orElse and orNone to fall back to other values, but use None if all keys are missing.

env[String]("API_KEY").
  orElse(prop("api.key")).
  orNone
// res22: ciris.ConfigValue[ciris.api.Id,Option[String]] = ConfigValue$271496048

It is also possible to read values with type Secret, in order to prevent values from being output in logs. The Secret String representation includes a short SHA1 hash of the value, so you can check that the expected value is being used.

import ciris.Secret
// import ciris.Secret

prop[Secret[String]]("file.encoding").value
// res23: ciris.api.Id[Either[ciris.ConfigError,ciris.Secret[String]]] = Right(Secret(7fa9ad7))

For more information on Secret, refer to the logging configurations section.

Suspending Effects

Most functions we’ve seen so far return values wrapped in a context Id, which is a way to say that there is no context. Since Id is defined as type Id[A] = A, you simply get the values without any context. However, you might have noticed that certain functions, while being safe, are not pure in the sense that, if the function is called more than once with the same arguments, it might return different values. This applies to, for example, reading system properties with prop (properties being mutable), and file and fileWithName (file contents may change).

One way to deal with these functions not being pure, is to model the effects explicitly with effect types. Instead of returning the result of reading a file, for example, we merely describe how to read the file, by suspending the reading of the file in a context F[_] for which there is a Sync instance (for example, IO from cats-effect with the cats-effect module).

Ciris provides pure functions argF, envF, propF, and fileSync and fileWithNameSync, which suspend the reading in a context F[_] for which there is a Sync instance. (Note that envF is merely a convenience method which lifts the value into F without suspending, since environment variables are immutable.) If you’re using any of the external libraries, like ciris-kubernetes or ciris-aws-ssm, they also provide pure functions which suspend reading, like secretF (for reading Kubernetes secrets) and paramF (for reading AWS SSM parameters).

import ciris.{propF, fileSync}
// import ciris.{propF, fileSync}

import ciris.cats.effect._
// import ciris.cats.effect._

import cats.effect.IO
// import cats.effect.IO

// Suspend reading of system property file.encoding as a String
propF[IO, String]("file.encoding")
// res25: ciris.ConfigEntry[cats.effect.IO,String,String,String] = ConfigEntry(file.encoding, Property)

// Suspend reading the file, trim the file contents, and convert to Int
fileSync[IO, Int](tempFile, _.trim)
// res27: ciris.ConfigEntry[cats.effect.IO,(java.io.File, java.nio.charset.Charset),String,Int] = ConfigEntry((/var/folders/ff/tg7g7zh52_g9s5_djb5z81w00000gn/T/temp-256420455339564896.tmp,UTF-8), ConfigKeyType(file))

Loading Configurations

We’re now able to represent configuration entries (with ConfigEntry) and configuration values (with ConfigValue) in our application, so now it’s time to combine multiple values into a configuration. We’ll start by modelling our configuration with nested case classes, like in the following example. If you haven’t already separated your application from your configuration, now is a good time to do so, to be able to load the configuration seperately. In the example below, we’re using refinement types to encode validation in the types of the configuration.

import eu.timepit.refined.api.Refined
import eu.timepit.refined.string.MatchesRegex
import eu.timepit.refined.types.net.UserPortNumber
import eu.timepit.refined.types.numeric.PosInt
import eu.timepit.refined.types.string.NonEmptyString
import eu.timepit.refined.W

type ApiKey = String Refined MatchesRegex[W.`"[a-zA-Z0-9]{25,40}"`.T]

final case class ApiConfig(
  key: Secret[ApiKey],
  port: UserPortNumber,
  timeoutSeconds: PosInt
)

final case class Config(
  appName: NonEmptyString,
  api: ApiConfig
)

The API key is secret, and we’ve wrapped it in Secret to denote that it shouldn’t be included in log output. For the port, we need it to be dynamic depending on the environment, and default to a fixed port if it’s not specified. In order to combine multiple configuration values, Ciris provides the loadConfig function, which accepts a number of configuration values (ConfigEntrys or ConfigValues) and the function with which to create the configuration.

import ciris.loadConfig
// import ciris.loadConfig

import ciris.refined._
// import ciris.refined._

import eu.timepit.refined.auto._
// import eu.timepit.refined.auto._

val config =
  loadConfig(
    env[Secret[ApiKey]]("API_KEY").
      orElse(prop("api.key")),
    prop[Option[UserPortNumber]]("http.port")
  ) { (apiKey, port) =>
    Config(
      appName = "my-api",
      api = ApiConfig(
        key = apiKey,
        timeoutSeconds = 10,
        port = port getOrElse 4000
      )
    )
  }
// config: ciris.api.Id[Either[ciris.ConfigErrors,Config]] = Left(ConfigErrors(Combined(MissingKey(API_KEY, Environment), MissingKey(api.key, Property))))

Note that the literal values above (the name, timeout, and default port) are validated at compile-time. If there are errors for the configuration values, Ciris will deal with them and accumulate them as ConfigErrors, before finally returning an Either[ConfigErrors, Config]. Note that the result is wrapped in Id, which is to say that no context (for example, effect type) was used. We could just as well have described the configuration loading with, for example, IO from cats-effect instead, using envF and propF, as seen in the suspending effects section.

import ciris.{envF, propF}
// import ciris.{envF, propF}

val configF =
  loadConfig(
    envF[IO, Secret[ApiKey]]("API_KEY").
      orElse(propF("api.key")),
    propF[IO, Option[UserPortNumber]]("http.port")
  ) { (apiKey, port) =>
    Config(
      appName = "my-api",
      api = ApiConfig(
        key = apiKey,
        timeoutSeconds = 10,
        port = port getOrElse 4000
      )
    )
  }
// configF: cats.effect.IO[Either[ciris.ConfigErrors,Config]] = <function1>

At this point, we can see that both the API_KEY environment variable and api.key system property are missing, so we’ve gotten a ConfigErrors back. We can use the messages function to retrieve error messages which are a bit more readable. Alternatively, with a syntax import, you can use orThrow to throw an exception with the error messages, if there are any, or return the configuration if it could be loaded successfully.

config.left.map(_.messages)
// res28: scala.util.Either[Vector[String],Config] = Left(Vector(Missing environment variable [API_KEY] and missing system property [api.key]))
{
  import ciris.syntax._
  config.orThrow()
}
// ciris.ConfigException: configuration loading failed with the following errors.
// 
//   - Missing environment variable [API_KEY] and missing system property [api.key].
// 
//   at ciris.ConfigException$.apply(ConfigException.scala:34)
//   at ciris.ConfigErrors$.toException$extension(ConfigErrors.scala:142)
//   at ciris.syntax$EitherConfigErrorsSyntax$.$anonfun$orThrow$1(syntax.scala:27)
//   at ciris.syntax$EitherConfigErrorsSyntax$.$anonfun$orThrow$1$adapted(syntax.scala:27)
//   at scala.util.Either.fold(Either.scala:189)
//   at ciris.syntax$EitherConfigErrorsSyntax$.orThrow$extension(syntax.scala:28)
//   ... 43 elided

Dynamic Configuration Loading

Sometimes it’s necessary to change how the configuration is loaded depending on some configuration value. For example, you might want to load configurations differently depending on in which environment the application is being run. You might want to use a fixed configuration in the local and testing environments, while loading secret values from a vault service in the production environment. One way to represent environments is with enumeratum enumerations, like in the following example. Refer to the multiple environments section for more information.

object environments {
  import enumeratum._

  sealed abstract class AppEnvironment extends EnumEntry

  object AppEnvironment extends Enum[AppEnvironment] {
    case object Local extends AppEnvironment
    case object Testing extends AppEnvironment
    case object Production extends AppEnvironment

    val values = findValues
  }
}

import environments._
import AppEnvironment._

Ciris provides the withValues (and withValue for a single value) function for being able to change how the configuration is loaded depending on the specified values. For example, following is an example of how to use a fixed configuration in the local and testing environments, while keeping the previously seen configuration loading in the production environment. Note that when using withValues, errors for the specified values (in this case, the APP_ENV environment variable) means we will not continue to try to load the configuration, meaning potential further errors are not included.

import ciris.withValue
// import ciris.withValue

import ciris.enumeratum._
// import ciris.enumeratum._

val config =
  withValue(env[AppEnvironment]("APP_ENV")) {
    case Local | Testing =>
      loadConfig {
        Config(
          appName = "my-api",
          api = ApiConfig(
            key = Secret("RacrqvWjuu4KVmnTG9b6xyZMTP7jnX"),
            timeoutSeconds = 10,
            port = 4000
          )
        )
      }

    case Production =>
      loadConfig(
        env[Secret[ApiKey]]("API_KEY").
          orElse(prop("api.key")),
        prop[Option[UserPortNumber]]("http.port")
      ) { (apiKey, port) =>
        Config(
          appName = "my-api",
          api = ApiConfig(
            key = apiKey,
            timeoutSeconds = 10,
            port = port getOrElse 4000
          )
        )
      }
  }
// config: ciris.api.Id[Either[ciris.ConfigErrors,Config]] = Left(ConfigErrors(MissingKey(APP_ENV, Environment)))

In many cases it’s not necessary to have this kind of dynamic configuration loading. If you want to keep the configuration loading process the same across environments, but want to have some values be different depending on the environment, that is also possible, like in the following example. You can also mix the two approaches as you see necessary.

val config =
  loadConfig(
    env[AppEnvironment]("APP_ENV"),
    env[Secret[ApiKey]]("API_KEY").
      orElse(prop("api.key")),
    prop[Option[UserPortNumber]]("http.port")
  ) { (environment, apiKey, port) =>
    Config(
      appName = "my-api",
      api = ApiConfig(
        key = apiKey,
        timeoutSeconds = 10,
        port = port getOrElse (environment match {
          case Local | Testing => 4000
          case Production      => 9000
        })
      )
    )
  }
// config: ciris.api.Id[Either[ciris.ConfigErrors,Config]] = Left(ConfigErrors(MissingKey(APP_ENV, Environment), Combined(MissingKey(API_KEY, Environment), MissingKey(api.key, Property))))