Using compile-env to manage environment variables
January 2023, by Maarten Nieber
The dev, prod and deploy environment

The role of the dev, prod and deploy environments

In this section I will describe an approach that helps to bridge the gap between dev and prod. Feel free to skip this if you are only interested in the practical approach for managing environment variables.

The dev environment

The role of the dev environment is to help the developer to create deployable code in an efficient and pleasant manner. This means that workflows for starting, stopping and debugging services are fast and uncomplicated, and that the system is easy to reason about so that the diagnosis of problems is made as easy as possible. For this purpose, the dev environment usually contains various debugging, testing and formatting tools.

The prod environment

The role of the prod environment is to execute the system in a safe, efficient and scaleable manner. Usually, developers don't deal with this environment, but dev-ops people do.

The deploy environment

In addition to dev and prod, I find it useful to have a deploy environment. The role of this environment to make a convincing case that it's possible to deploy the code to production. The deploy environment will usually be more minimal than the dev environment, because it won't have any of the dev tools. It will also be more minimal than prod, because it merely serves as an example starting point, and therefore it doesn't need to have all the prod functionality.

The deploy environment is created by (mainly) the developers to make life easier for the dev-ops. For example, while it's typically the case that the dev environment runs a development server, we can replace this with a minimally configured production server in the deploy environment. This makes it easier for the dev-ops person to verify that a version of the system that runs the production server is viable. In general, when the dev-ops person looks at the deploy environment (and reads its documentation) then they will be able to imagine the deployment to production.

The relation between the different environments

Dev can be different from prod

It was suggested above that the dev environment should be easy to reason about. This is important because developers continually see their code behave in unexpected ways. To figure out what's going on, it helps if the system's environment is relatively simple. For this reason, I don't support the idea that the dev environment should be as close as possible to the production environment.

However, in some cases dev should be similar to prod

Now, I'm not arguing that the developer should remain completely unaware of dev-ops related concerns. There may be particular reasons to reflect a production concern in the dev environment. For example, in particular systems, the omission of a load balancer in the dev environment may lead to a code base that does not lend itself well for load-balancing. However, since including it adds an extra burden on the developers, it should be conscious decision that is backed by analysis and arguments, not a reflex.

Having one set of artifacts

Having multiple environments is not at odds with deploying the same artifacts everywhere, and using the environment variables to tune it. These "one and only" artifacts are based on the prod environment. In dev you don't need artifacts because you can use the source code directly, so there is no contradiction there.

Security in each environment

Dev and deploy should have no sensitive data

The dev and deploy environments should have no sensitive data at all. They require some security, because we wouldn't want a hacker to gain access and see the source code. However, if a hacker obtained access, then they wouldn't be able to get any useful secrets. Although the deploy environment doesn't have sensitive data, it will reflect some of the security measures of the prod environment. This is because the deploy environment needs to be relatively close to the prod environment (since it must make a convincing case that the system can be deployed).

Prod has sensitive data

Finally, the prod environment obviously contains sensitive data that must be protected.