-
Attempt to catch problematic configurations
-
Breaking changes via deprecation
-
Preference of Secrets in initContainer over Environment
-
Sub-charts are deployed from global chart
-
Template partials for
gitlab/*
should be global whenever possible
-
Forked charts
-
Redis
-
Redis HA
-
MinIO
-
registry
-
NGINX Ingress
-
Kubernetes version used throughout chart
-
Image variants shipped with CNG
Design Decisions
This documentation collects reasoning and decisions made
regarding the design of the Helm charts in this repository.
Attempt to catch problematic configurations
Due to the complexity of these charts and their level of flexibility, there are some
overlaps where it is possible to produce a configuration that would lead to an
unpredictable, or entirely non-functional deployment. In an effort
to prevent known problematic settings combinations, we have implemented template logic
designed to detect and warn the user that their configuration will not work.
This replicates the behavior of deprecations, but is specific to ensuring functional configuration.
Introduced in !757 checkConfig: add methods to test for known errors
Breaking changes via deprecation
During the development of these charts, we occasionally make improvements that require
alterations to the properties of existing deployments. Two examples were the centralization
of configuring the use of MinIO, and the migration of external object storage configuration
from properties to secrets (in observance of our preference).
As a means of preventing a user from accidentally deploying an updated version of these
charts which includes a breaking change against a configuration that would not function, we
have chosen to implement deprecation notifications. These are designed to detect
properties have been relocated, altered, replaced, or removed entirely, then inform
the user of what changes need to be made to the configuration. This may include informing
the user to see documentation on how to replace a property with a secret. These notifications
will cause the Helm
install
or
upgrade
commands to stop with a parse error, and output a complete list of items that need to be addressed. We have taken care to ensure a user will not be placed into a loop of error, fix, repeat.
All deprecations must be addressed in order for a successful deployment to occur. We believe
the user would prefer to be informed of a breaking change over experiencing unexpected
behavior or complete failure that requires debugging.
Introduced in !396 Deprecations: implement buffered list of deprecations
Preference of Secrets in initContainer over Environment
Much of the container ecosystem has, or expects, the capability to be configured
through environment variables. This configuration practice
stems from the concept of The Twelve-Factor App. This
greatly simplifies configuration across multiple deployment environments, but there
remains a security concern with passing connection secrets such as passwords and
private keys via the container’s environment.
Most container ecosystems provide a simple method to inspect the state of a running
container, which usually includes the environment. Using Docker
as an example, any process capable of communicating with the daemon can query the
state of all running containers. This means that if you have a privileged container
such as
dind
, that container can then inspect the environment of
any
container
on a given node, and expose
all
secrets contained within.
As a part of the complete DevOps lifecycle,
dind
is regularly
used for building containers that will be pushed to a registry and subsequently
deployed.
This concern is why we’ve decided to prefer the population of sensitive information
via initContainers.
Related issues:
Sub-charts are deployed from global chart
All sub-charts of this repository are designed to be deployed via the global chart.
Each component can still be deployed individually, but make use of a common set of
properties facilitated by the global chart.
This decision simplifies both the use and maintenance of the repository as a whole.
Related issue:
Template partials for
gitlab/*
should be global whenever possible
All template partials of the
gitlab/*
sub-charts should be a part of the global or
GitLab sub-chart
templates/_helpers.tpl
whenever possible. Templates from
forked charts will remain a part of those charts. This reduces
the maintenance impact of these forks.
The benefits of this are straight-forward:
-
Increased DRY behavior, leading to easier maintenance. There should be no reason
to have duplicates of the same function across multiple sub-charts when a single
entry will suffice.
-
Reduction of template naming conflicts. All partials throughout a chart are compiled together,
and thus we can treat them like the global behavior they are.
Related issue:
Forked charts
The following charts have been forked or re-created in this repository following
our guidelines for forking
Redis
With the
3.0
release of the GitLab Helm chart, we no longer fork the upstream Redis chart,
and instead include it as a dependency.
Redis HA
Redis-HA was a chart we included in our releases prior to
3.0
. It has now been removed,
and replaced with upstream Redis chart
which has added optional HA support.
MinIO
Our MinIO chart was altered from the upstream MinIO.
-
Make use of pre-existing Kubernetes secrets instead of creating new ones from properties.
-
Remove providing the sensitive keys via Environment.
-
Automate the creation of multiple buckets via
defaultBuckets
in place of
defaultBucket.*
properties.
registry
Our registry chart was altered from the upstream
docker-registry
.
-
Enable the use of in-chart MinIO services automatically.
-
Automatically hook authentication to the GitLab services.
NGINX Ingress
Our NGINX Ingress chart was altered from the upstream NGINX Ingress.
-
Add feature to allow for the TCP ConfigMap to be external to the chart
-
Add feature to allow Ingress class to be templated based on release name
Kubernetes version used throughout chart
To maximize support for different Kubernetes versions, use a
kubectl
that’s
one minor version lower than the current stable release of Kubernetes.
This should allow support for at least three, and quite possibly more
Kubernetes minor versions. For further discussion on
kubectl
versions, see
issue 1509.
Related Issues:
-
charts/gitlab#1509
-
charts/gitlab#1583
Related Merge Requests:
-
charts/gitlab!1053
-
build/CNG!329
-
gitlab-build-images!251
Image variants shipped with CNG
Date: 2022-02-10
The CNG project ships images based on both Debian and UBI. The decision to maintain configuration
for both distributions was based upon the following:
-
Why we ship Debian-based images:
-
Track record, precedent
-
Familiarity with distribution
-
Community vs “enterprise”
-
Lack of perceived vendor lock-in
-
Why we ship UBI-based images:
-
Required in some customer environments
-
Required for RHEL certification and inclusion into the OpenShift Marketplace / RedHat Catalog
Further discussion on this topic can be found in issue #3095.
