Blog
Token
About Us
AiDOOS Wiki
Project Cost Estimation

Welcome to Knowledge Base!

KB at your finger tips

This is one stop global knowledge base where you can learn about all the products, solutions and support features.

Categories
All
CRM
Salesforce
ERP
SAP
Storage and Backups
Purestorage Netapp Nutanix
Cloud
AWS
Computing
IBM
Database
MongoDB MySQL PostgreSQL
DevOps
Docker Kubernetes Jenkins GitLab
Mobile
React Native Android
Web
React Angular PHP
Programming
Python
Products
CNC Software (Mastercam) 3D perception 7shifts ABB EV Charging Infrastructure Accelo Aconex (by Oracle) Act-On Actenum Scheduler Actico Compliance Suite ActiveCampaign Adapt Addepar Adobe Captivate Adyen AG Leader Agrivi AgriWebb Agworld Ahrefs Aidoc AIMS360 Akamai Kona Site Defender Algolia Alis by TalentGuard ALLDATA Repair Allscripts Alpaca Trading Altair HyperWorks Altium Designer AMOS Amplitude Anaplan Anessa Biofuel Optimization AppDynamics Applied Epic Applied Materials E3 Aptean ERP Aptean Food & Beverage ERP Aptima Aquatic Informatics’ WaterTrax Arable Arena Solutions PLM Articulate 360 Asana ASML Twinscan Aspen Plus Biochemical Process Modeling Aurora Solar AspenTech APC Athenahealth Atomwise Audiam Audiomovers AspenTech for Refining Autodesk AutoCAD Autosoft AspenTech Pharma Suite Asset Panda Aveva Marine AssetMark Aveva Pipeline Management AVEVA Process Engineering Suite AVEVA Refining Solutions AVEVA Unified Operations Center Avid Media Composer AVL eSuite AWS Ground Station AWS IoT Greengrass Azure IoT Central Azure Space Backbase BatchMaster ERP Benchling Bentley OpenFlows Bentley OpenRoads Bentley OpenUtilities Bentley Systems AssetWise Bentley Systems – OpenRail beonic BevSpot Bigbelly Smart Waste & Recycling System BigCommerce Automotive Bitly Bitmovin Blackbaud BlackBerry AtHoc Blackboard Learn BlackLynx Blend Blue Yonder WMS Bluebeam Revu Bohemia Interactive Simulations (BISim) Bosch Esitronic Diagnostic Bosch IoT Suite Braintree Bright Biomethane Brightpearl Bringg Browzwear Builder Prime Buildertrend BuildTools BWX Technologies' Nuclear Security Cabinet Vision Cadence OrCAD Cadence Virtuoso CAE Canvas LMS Cargowise One Carrier i-Vu CaterZen CattleMax CDK Global Celtx Cerner Pharmacy ChargePoint Cloud Services Charles River Development Check Point CloudGuard Checkout.com Cin7 Circularise Cisco Umbrella Classcraft ClearDATA ClearMetal ClickFunnels ClickUp Climate FieldView Clir Renewables Cloud Lending (Q2 Software) Cobb Accessport ComplianceQuest ComplyAdvantage Conducttr Conservis Continental Connected Vehicle Copper Coupa by LLamasoft CQG Integrated Client Creatio Creatio marketing CropNutrition CrowdStrike Falcon CyberSource Damarel Darktrace Dassault Systèmes CATIA Dassault Systèmes ENOVIA Dataminr DaVinci Resolve DealerSocket DealHub Decisive Farming Deliverect Deltek Costpoint Demandbase Denso EV Thermal Systems DeSL (Digital Enterprise Software) DigitalOcean DJI Ground Station Pro Docebo DrillPlan by Schlumberger DroneDeploy e-Builder Ecochain EcoVadis Edmentum Edsby EFI Live Tuning EFI Optitex EFI PrintSmith Vision Ellie Mae Encompass Ellucian Banner Elucidat Emerson Ovation Emerson Plantweb eMoney Advisor Enact Systems Enfusion Envestnet | Tamarac Epicor ERP Epicor Parts Network Esko ArtiosCAD Esko WebCenter ETQ Reliance QMS Everbridge eVinci™ Micro Reactor by Westinghouse Exostar SCM Fanuc Automation Fanuc Robotics FarmERP FarmLogs FieldAware Finacle Finastra Fusion FinQuery Flatiron Health FoodLogiQ Fortinet FortiGate FourKites Freshworks GDC Technology GE Digital APM GE Digital Plant Applications GE Digital Predix GE Healthcare FlexFactory GE Vernova Genscape Pipeline Solutions Geofeedia GeoSoftware Geotab GEP SMART Gerber Technology GetAccept Gorgias Greenbyte Energy Cloud Greenlight Guru GreenPrinter GreenSlate Guidewire Software Handshake HawkEye 360 Health Catalyst Herdwatch Hexagon Intergraph CAD Hexagon Manufacturing Intelligence Hexagon MSC Software Hexagon PPM Hexagon Safety & Infrastructure Hexcel HexPly Honeywell Enraf Tank Gauging Honeywell Forge Honeywell Forge for Chemicals Honeywell UOP Ecofining Houzz Pro HP Indigo Digital Press HP Tuners VCM Suite HubSpot CRM IBM i2 IBM Watson Health IFS Applications Illuminate Education Imprivata Indra Air Automation InEight Infor CloudSuite Food & Beverage Infor EAM Infor LN Infor WMS Inmarsat Fleet Data Instapage Insurity Intellect QMS Intergraph SmartMarine 3D Intex ERP Intuitive Surgical da Vinci IONOS Ironclad iSpring Suite John Deere Operations Center Johnson Controls Metasys Juro JustFood ERP Kahoot Kahua Kareo KBC Petro-SIM by Yokogawa Keap Kiinaxis RapidResponse Kinaxis RapidResponse Kisters WISKI KLA YieldStar Klaviyo Knewton Alta Kratos OpenSpace KUKA Robotics KUKA Smart Production Kustomer Landr LeadAngel LeanData Lectra Fashion PLM Leidos Integrated ISR Solutions Linode Lithium Balance BMS Lockheed Martin DiamondShield LogicGate Logility LogiNext Mile LuciadLightspeed (Hexagon) Luma Health Mahle Thermal Management Majesco Mallcomm MAM Software Autopart Manhattan TMS Manhattan WMS Mark43 MASS (Cohort plc) MasterControl Manufacturing Excellence MasterControl Quality Excellence Material Exchange McKesson Pharmacy Management MedeAnalytics Medidata Solutions Medtronic CareLink Medtronic Hugo RAS Mentor Graphics (Siemens) MetricStream Microsoft D365 SCM Microsoft Project MilliporeSigma BioContinuum MindTickle Mirakl Mitchell1 ProDemand MODS (Military Operational Decision Support) Moodle Moody’s Analytics Morningstar Direct MRI Facilities Management MuleSoft MyPath Nautilus Labs Navitaire nCino Nearpod Netafim NetApp Netwrix Auditor NextGen Healthcare NICE Actimize NICE Public Safety Nimble Nutshell Nuvation Energy BMS Nuvolo Office Furniture by Haworth Omnisend OneTrust OneWeb Satellite Services Onfleet Optibus Optimizely ONYX InSight Oracle E-Business Suite Oracle FLEXCUBE Oracle TMS Orion Advisor Tech OSISoft PI System Outgrow OVHcloud PagerDuty Palo Alto Networks Cortex XSOAR Palo Alto Networks Prisma Cloud PandaDoc Panorama Education Parrot Analytics Petrel by Schlumberger Philips HealthSuite PipelineManager by Atmos International Pipeliner CRM PitchBook Pix4Dfields Planet Labs Plex Systems PowerSchool PredictHQ Printavo Procore Project44 Proposify Prospera PTC Windchill Quantifi QuarkXPress Quipli Quorum Distribution Software R1 RCM Rackspace Rapyd ReCharge Reynolds and Reynolds Rightsline Riskalyze Roostify Runway ML S&P Capital IQ Sabre AirCentre Safefood 360 Saipher ATC SAP Defense & Security Sapiens CoreSuite Oracle Cloud SCM Oracle NetSuite Plex Manufacturing Cloud QbDVision Ramco Aviation MRO Suite RedTeam SAP EWM SAP Supply Chain Management SAS Analytics Schoology ScreenCloud Sensia Oilfield Monitoring and Control SentinelOne Serpstat Shift Technology Shift72 ShipBob ShipServ Shopify Plus Shopify Plus for Automotive Shotgun (by Autodesk) Siemens Simcenter Signal Sciences SimCentric Technologies SimCorp Dimension SimScale Simutech Multimedia Skedulo SkillMill by Interplay Learning Skyfii SkyWatch EarthCache Slingshot Orbital Smartsheet for Compliance Snapsheet Solutionreach Soyang Europe Green Materials Sparta Systems TrackWise Sphera Sustainability Management Spire Space Services Splunk Enterprise Security Splunk Industrial IoT Square for Retail StormGeo Stratasys 3D Printers Stripe STRIVR StudioBinder SUSE Swissport Symantec Endpoint Security (Broadcom) Synopsys TCAD Syntasa TalentLMS TankVision by Endress+Hauser Teladoc Health Tenable.io Terminus TexGen Thales CipherTrust Thales Manpack EW Suite Thales TopSky-ATC Thinkorswim Thryv Toast POS TotalEnergies Bio Lubricants TraceGains TraceLink Trimble Construction Trove TurnKey Lender Udemy for Business Unbounce Updox Validic Valmet DNA Automation System Varsity Tutors Veeva CRM Veeva MedTech Veeva Quality Veeva Vault Vendasta Veson Nautical IMOS Platform Viewpoint (by Trimble) Vimeo OTT Virbela Vizient VTS VWO Waste Management Open API by AMCS Waystar Wealthfront Wherefour WindRiver Helix Virtualization Platform WoodCAD|CAM Workday Student Worldpay Wrike Xactly Xerox Paper Recycling XOLV Xtime YCharts Yopto Zendesk Marketplace Zoho Zoho Inventory Zonos Zscaler Web-to-Print by Printful Zuppler ABB Robotics Solutions Alibaba Cloud ANSYS Composite PrepPost Apollo Autodesk BIM 360 Aventri BigCommerce Bizzabo Bloomberg Terminal BookBrush Brella Canva Pro Chartmetric Contentful edX EnergyHub EventPro Granular Helix Core iVvy Venue Management Joomag ManageArtworks MathWorks MATLAB Mobalytics Moocall On24 Pluralsight PressPad Prismm (All Seated) Reedsy RegFox SAP S/4HANA Amwell ANSYS Engineering Simulation Software Autodesk CFD AutoGrid Flex BetterUp Dassault Systèmes SIMULIA Evisort FlippingBook LabWare LIMS SAP TMS

DevOps-Docker

Configuring builds

Configuring builds

Bake supports loading build definition from files, but sometimes you need even more flexibility to configure this definition.

For this use case, you can define variables inside the bake files that can be set by the user with environment variables or by attribute definitions in other bake files. If you wish to change a specific value for a single invocation you can use the --set flag from the command line.

Global scope attributes

You can define global scope attributes in HCL/JSON and use them for code reuse and setting values for variables. This means you can do a “data-only” HCL file with the values you want to set/override and use it in the list of regular output files. 

# docker-bake.hcl
variable "FOO" {
  default = "abc"
}

target "app" {
  args = {
    v1 = "pre-${FOO}"
  }
}

You can use this file directly: 

$ docker buildx bake --print app

{
  "group": {
    "default": {
      "targets": [
        "app"
      ]
    }
  },
  "target": {
    "app": {
      "context": ".",
      "dockerfile": "Dockerfile",
      "args": {
        "v1": "pre-abc"
      }
    }
  }
}

Or create an override configuration file: 

# env.hcl
WHOAMI="myuser"
FOO="def-${WHOAMI}"

And invoke bake together with both of the files: 

$ docker buildx bake -f docker-bake.hcl -f env.hcl --print app

{
  "group": {
    "default": {
      "targets": [
        "app"
      ]
    }
  },
  "target": {
    "app": {
      "context": ".",
      "dockerfile": "Dockerfile",
      "args": {
        "v1": "pre-def-myuser"
      }
    }
  }
}

From command line

You can also override target configurations from the command line with the --set flag: 

# docker-bake.hcl
target "app" {
  args = {
    mybuildarg = "foo"
  }
}

$ docker buildx bake --set app.args.mybuildarg=bar --set app.platform=linux/arm64 app --print

{
  "group": {
    "default": {
      "targets": [
        "app"
      ]
    }
  },
  "target": {
    "app": {
      "context": ".",
      "dockerfile": "Dockerfile",
      "args": {
        "mybuildarg": "bar"
      },
      "platforms": [
        "linux/arm64"
      ]
    }
  }
}

Pattern matching syntax defined in https://golang.org/pkg/path/#Match is also supported: 

$ docker buildx bake --set foo*.args.mybuildarg=value  # overrides build arg for all targets starting with "foo"
$ docker buildx bake --set *.platform=linux/arm64      # overrides platform for all targets
$ docker buildx bake --set foo*.no-cache               # bypass caching only for targets starting with "foo"

Complete list of overridable fields:

  • args
  • cache-from
  • cache-to
  • context
  • dockerfile
  • labels
  • no-cache
  • output
  • platform
  • pull
  • secrets
  • ssh
  • tags
  • target

Using variables in variables across files

When multiple files are specified, one file can use variables defined in another file. 

# docker-bake1.hcl
variable "FOO" {
  default = upper("${BASE}def")
}

variable "BAR" {
  default = "-${FOO}-"
}

target "app" {
  args = {
    v1 = "pre-${BAR}"
  }
}

# docker-bake2.hcl
variable "BASE" {
  default = "abc"
}

target "app" {
  args = {
    v2 = "${FOO}-post"
  }
}

$ docker buildx bake -f docker-bake1.hcl -f docker-bake2.hcl --print app

{
  "group": {
    "default": {
      "targets": [
        "app"
      ]
    }
  },
  "target": {
    "app": {
      "context": ".",
      "dockerfile": "Dockerfile",
      "args": {
        "v1": "pre--ABCDEF-",
        "v2": "ABCDEF-post"
      }
    }
  }
}

Color output controls

Color output controls

BuildKit and Buildx have support for modifying the colors that are used to output information to the terminal. You can set the environment variable BUILDKIT_COLORS to something like run=123,20,245:error=yellow:cancel=blue:warning=white to set the colors that you would like to use:

Progress output custom colors

Setting NO_COLOR to anything will disable any colorized output as recommended by no-color.org:

Progress output no color

Note

Parsing errors will be reported but ignored. This will result in default color values being used where needed.

See also the list of pre-defined colors.

Read article

Configure BuildKit

Configure BuildKit

If you create a docker-container or kubernetes builder with Buildx, you can apply a custom BuildKit configuration by passing the --config flag to the docker buildx create command.

Registry mirror

You can define a registry mirror to use for your builds. Doing so redirects BuildKit to pull images from a different hostname. The following steps exemplify defining a mirror for docker.io (Docker Hub) to mirror.gcr.io .

  1. Create a TOML at /etc/buildkitd.toml with the following content: 

    debug = true
[registry."docker.io"]
  mirrors = ["mirror.gcr.io"]

    Note

    debug = true turns on debug requests in the BuildKit daemon, which logs a message that shows when a mirror is being used.

  2. Create a docker-container builder that uses this BuildKit configuration: 

    $ docker buildx create --use --bootstrap \
  --name mybuilder \
  --driver docker-container \
  --config /etc/buildkitd.toml

  3. Build an image: 

    docker buildx build --load . -f - <<EOF
FROM alpine
RUN echo "hello world"
EOF

The BuildKit logs for this builder now shows that it uses the GCR mirror. You can tell by the fact that the response messages include the x-goog-* HTTP headers. 

$ docker logs buildx_buildkit_mybuilder0

...
time="2022-02-06T17:47:48Z" level=debug msg="do request" request.header.accept="application/vnd.docker.container.image.v1+json, */*" request.header.user-agent=containerd/1.5.8+unknown request.method=GET spanID=9460e5b6e64cec91 traceID=b162d3040ddf86d6614e79c66a01a577
time="2022-02-06T17:47:48Z" level=debug msg="fetch response received" response.header.accept-ranges=bytes response.header.age=1356 response.header.alt-svc="h3=\":443\"; ma=2592000,h3-29=\":443\"; ma=2592000,h3-Q050=\":443\"; ma=2592000,h3-Q046=\":443\"; ma=2592000,h3-Q043=\":443\"; ma=2592000,quic=\":443\"; ma=2592000; v=\"46,43\"" response.header.cache-control="public, max-age=3600" response.header.content-length=1469 response.header.content-type=application/octet-stream response.header.date="Sun, 06 Feb 2022 17:25:17 GMT" response.header.etag="\"774380abda8f4eae9a149e5d5d3efc83\"" response.header.expires="Sun, 06 Feb 2022 18:25:17 GMT" response.header.last-modified="Wed, 24 Nov 2021 21:07:57 GMT" response.header.server=UploadServer response.header.x-goog-generation=1637788077652182 response.header.x-goog-hash="crc32c=V3DSrg==" response.header.x-goog-hash.1="md5=d0OAq9qPTq6aFJ5dXT78gw==" response.header.x-goog-metageneration=1 response.header.x-goog-storage-class=STANDARD response.header.x-goog-stored-content-encoding=identity response.header.x-goog-stored-content-length=1469 response.header.x-guploader-uploadid=ADPycduqQipVAXc3tzXmTzKQ2gTT6CV736B2J628smtD1iDytEyiYCgvvdD8zz9BT1J1sASUq9pW_ctUyC4B-v2jvhIxnZTlKg response.status="200 OK" spanID=9460e5b6e64cec91 traceID=b162d3040ddf86d6614e79c66a01a577
time="2022-02-06T17:47:48Z" level=debug msg="fetch response received" response.header.accept-ranges=bytes response.header.age=760 response.header.alt-svc="h3=\":443\"; ma=2592000,h3-29=\":443\"; ma=2592000,h3-Q050=\":443\"; ma=2592000,h3-Q046=\":443\"; ma=2592000,h3-Q043=\":443\"; ma=2592000,quic=\":443\"; ma=2592000; v=\"46,43\"" response.header.cache-control="public, max-age=3600" response.header.content-length=1471 response.header.content-type=application/octet-stream response.header.date="Sun, 06 Feb 2022 17:35:13 GMT" response.header.etag="\"35d688bd15327daafcdb4d4395e616a8\"" response.header.expires="Sun, 06 Feb 2022 18:35:13 GMT" response.header.last-modified="Wed, 24 Nov 2021 21:07:12 GMT" response.header.server=UploadServer response.header.x-goog-generation=1637788032100793 response.header.x-goog-hash="crc32c=aWgRjA==" response.header.x-goog-hash.1="md5=NdaIvRUyfar8201DleYWqA==" response.header.x-goog-metageneration=1 response.header.x-goog-storage-class=STANDARD response.header.x-goog-stored-content-encoding=identity response.header.x-goog-stored-content-length=1471 response.header.x-guploader-uploadid=ADPycdtR-gJYwC7yHquIkJWFFG8FovDySvtmRnZBqlO3yVDanBXh_VqKYt400yhuf0XbQ3ZMB9IZV2vlcyHezn_Pu3a1SMMtiw response.status="200 OK" spanID=9460e5b6e64cec91 traceID=b162d3040ddf86d6614e79c66a01a577
time="2022-02-06T17:47:48Z" level=debug msg=fetch spanID=9460e5b6e64cec91 traceID=b162d3040ddf86d6614e79c66a01a577
time="2022-02-06T17:47:48Z" level=debug msg=fetch spanID=9460e5b6e64cec91 traceID=b162d3040ddf86d6614e79c66a01a577
time="2022-02-06T17:47:48Z" level=debug msg=fetch spanID=9460e5b6e64cec91 traceID=b162d3040ddf86d6614e79c66a01a577
time="2022-02-06T17:47:48Z" level=debug msg=fetch spanID=9460e5b6e64cec91 traceID=b162d3040ddf86d6614e79c66a01a577
time="2022-02-06T17:47:48Z" level=debug msg="do request" request.header.accept="application/vnd.docker.image.rootfs.diff.tar.gzip, */*" request.header.user-agent=containerd/1.5.8+unknown request.method=GET spanID=9460e5b6e64cec91 traceID=b162d3040ddf86d6614e79c66a01a577
time="2022-02-06T17:47:48Z" level=debug msg="fetch response received" response.header.accept-ranges=bytes response.header.age=1356 response.header.alt-svc="h3=\":443\"; ma=2592000,h3-29=\":443\"; ma=2592000,h3-Q050=\":443\"; ma=2592000,h3-Q046=\":443\"; ma=2592000,h3-Q043=\":443\"; ma=2592000,quic=\":443\"; ma=2592000; v=\"46,43\"" response.header.cache-control="public, max-age=3600" response.header.content-length=2818413 response.header.content-type=application/octet-stream response.header.date="Sun, 06 Feb 2022 17:25:17 GMT" response.header.etag="\"1d55e7be5a77c4a908ad11bc33ebea1c\"" response.header.expires="Sun, 06 Feb 2022 18:25:17 GMT" response.header.last-modified="Wed, 24 Nov 2021 21:07:06 GMT" response.header.server=UploadServer response.header.x-goog-generation=1637788026431708 response.header.x-goog-hash="crc32c=ZojF+g==" response.header.x-goog-hash.1="md5=HVXnvlp3xKkIrRG8M+vqHA==" response.header.x-goog-metageneration=1 response.header.x-goog-storage-class=STANDARD response.header.x-goog-stored-content-encoding=identity response.header.x-goog-stored-content-length=2818413 response.header.x-guploader-uploadid=ADPycdsebqxiTBJqZ0bv9zBigjFxgQydD2ESZSkKchpE0ILlN9Ibko3C5r4fJTJ4UR9ddp-UBd-2v_4eRpZ8Yo2llW_j4k8WhQ response.status="200 OK" spanID=9460e5b6e64cec91 traceID=b162d3040ddf86d6614e79c66a01a577
...

Setting registry certificates

If you specify registry certificates in the BuildKit configuration, the daemon copies the files into the container under /etc/buildkit/certs . The following steps show adding a self-signed registry certificate to the BuildKit configuration.

  1. Add the following configuration to /etc/buildkitd.toml : 

    # /etc/buildkitd.toml
debug = true
[registry."myregistry.com"]
  ca=["/etc/certs/myregistry.pem"]
  [[registry."myregistry.com".keypair]]
    key="/etc/certs/myregistry_key.pem"
    cert="/etc/certs/myregistry_cert.pem"

    This tells the builder to push images to the myregistry.com registry using the certificates in the specified location ( /etc/certs ).

  2. Create a docker-container builder that uses this configuration: 

    $ docker buildx create --use --bootstrap \
  --name mybuilder \
  --driver docker-container \
  --config /etc/buildkitd.toml

  3. Inspect the builder’s configuration file ( /etc/buildkit/buildkitd.toml ), it shows that the certificate configuration is now configured in the builder. 

    $ docker exec -it buildx_buildkit_mybuilder0 cat /etc/buildkit/buildkitd.toml

    debug = true

[registry]

  [registry."myregistry.com"]
    ca = ["/etc/buildkit/certs/myregistry.com/myregistry.pem"]

    [[registry."myregistry.com".keypair]]
      cert = "/etc/buildkit/certs/myregistry.com/myregistry_cert.pem"
      key = "/etc/buildkit/certs/myregistry.com/myregistry_key.pem"

  4. Verify that the certificates are inside the container: 

    $ docker exec -it buildx_buildkit_mybuilder0 ls /etc/buildkit/certs/myregistry.com/
myregistry.pem    myregistry_cert.pem   myregistry_key.pem

Now you can push to the registry using this builder, and it will authenticate using the certificates: 

$ docker buildx build --push --tag myregistry.com/myimage:latest .

CNI networking

CNI networking for builders can be useful for dealing with network port contention during concurrent builds. CNI is not yet available in the default BuildKit image. But you can create your own image that includes CNI support.

The following Dockerfile example shows a custom BuildKit image with CNI support. It uses the CNI config for integration tests in BuildKit as an example. Feel free to include your own CNI configuration. 

# syntax=docker/dockerfile:1

ARG BUILDKIT_VERSION=v{{ site.buildkit_version }}
ARG CNI_VERSION=v1.0.1

FROM --platform=$BUILDPLATFORM alpine AS cni-plugins
RUN apk add --no-cache curl
ARG CNI_VERSION
ARG TARGETOS
ARG TARGETARCH
WORKDIR /opt/cni/bin
RUN curl -Ls https://github.com/containernetworking/plugins/releases/download/$CNI_VERSION/cni-plugins-$TARGETOS-$TARGETARCH-$CNI_VERSION.tgz | tar xzv

FROM moby/buildkit:${BUILDKIT_VERSION}
ARG BUILDKIT_VERSION
RUN apk add --no-cache iptables
COPY --from=cni-plugins /opt/cni/bin /opt/cni/bin
ADD https://raw.githubusercontent.com/moby/buildkit/${BUILDKIT_VERSION}/hack/fixtures/cni.json /etc/buildkit/cni.json

Now you can build this image, and create a builder instance from it using the --driver-opt image option: 

$ docker buildx build --tag buildkit-cni:local --load .
$ docker buildx create --use --bootstrap \
  --name mybuilder \
  --driver docker-container \
  --driver-opt "image=buildkit-cni:local" \
  --buildkitd-flags "--oci-worker-net=cni"

Resource limiting

Max parallelism

You can limit the parallelism of the BuildKit solver, which is particularly useful for low-powered machines, using a BuildKit configuration while creating a builder with the --config flags. 

# /etc/buildkitd.toml
[worker.oci]
  max-parallelism = 4

Now you can create a docker-container builder that will use this BuildKit configuration to limit parallelism. 

$ docker buildx create --use \
  --name mybuilder \
  --driver docker-container \
  --config /etc/buildkitd.toml

TCP connection limit

TCP connections are limited to 4 simultaneous connections per registry for pulling and pushing images, plus one additional connection dedicated to metadata requests. This connection limit prevents your build from getting stuck while pulling images. The dedicated metadata connection helps reduce the overall build time.

More information: moby/buildkit#2259

Read article

Docker driver

Docker driver

The Buildx Docker driver is the default driver. It uses the BuildKit server components built directly into the Docker engine. The Docker driver requires no configuration.

Unlike the other drivers, builders using the Docker driver can’t be manually created. They’re only created automatically from the Docker context.

Images built with the Docker driver are automatically loaded to the local image store.

Synopsis 

# The Docker driver is used by buildx by default
docker buildx build .

It’s not possible to configure which BuildKit version to use, or to pass any additional BuildKit parameters to a builder using the Docker driver. The BuildKit version and parameters are preset by the Docker engine internally.

If you need additional configuration and flexibility, consider using the Docker container driver.

Further reading

For more information on the Docker driver, see the buildx reference.

Read article

Bake file definition

Bake file definition

buildx bake supports HCL, JSON and Compose file format for defining build groups, targets as well as variables and functions. It looks for build definition files in the current directory in the following order:

  • docker-compose.yml
  • docker-compose.yaml
  • docker-bake.json
  • docker-bake.override.json
  • docker-bake.hcl
  • docker-bake.override.hcl

Specification

Inside a bake file you can declare group, target and variable blocks to define project specific reusable build flows.

Target

A target reflects a single docker build invocation with the same options that you would specify for docker build : 

# docker-bake.hcl
target "webapp-dev" {
  dockerfile = "Dockerfile.webapp"
  tags = ["docker.io/username/webapp:latest"]
}

$ docker buildx bake webapp-dev

Note

In the case of compose files, each service corresponds to a target. If compose service name contains a dot it will be replaced with an underscore.

Complete list of valid target fields available for HCL and JSON definitions:

Name Type Description
inherits List Inherit build options from other targets
args Map Set build-time variables (same as --build-arg flag)
cache-from List External cache sources (same as --cache-from flag)
cache-to List Cache export destinations (same as --cache-to flag)
context String Set of files located in the specified path or URL
contexts Map Additional build contexts (same as --build-context flag)
dockerfile String Name of the Dockerfile (same as --file flag)
dockerfile-inline String Inline Dockerfile content
labels Map Set metadata for an image (same as --label flag)
no-cache Bool Do not use cache when building the image (same as --no-cache flag)
no-cache-filter List Do not cache specified stages (same as --no-cache-filter flag)
output List Output destination (same as --output flag)
platforms List Set target platforms for build (same as --platform flag)
pull Bool Always attempt to pull all referenced images (same as --pull flag)
secret List Secret to expose to the build (same as --secret flag)
ssh List SSH agent socket or keys to expose to the build (same as --ssh flag)
tags List Name and optionally a tag in the format name:tag (same as --tag flag)
target String Set the target build stage to build (same as --target flag)

Group

A group is a grouping of targets: 

# docker-bake.hcl
group "build" {
  targets = ["db", "webapp-dev"]
}

target "webapp-dev" {
  dockerfile = "Dockerfile.webapp"
  tags = ["docker.io/username/webapp:latest"]
}

target "db" {
  dockerfile = "Dockerfile.db"
  tags = ["docker.io/username/db"]
}

$ docker buildx bake build

Variable

Similar to how Terraform provides a way to define variables, the HCL file format also supports variable block definitions. These can be used to define variables with values provided by the current environment, or a default value when unset: 

# docker-bake.hcl
variable "TAG" {
  default = "latest"
}

target "webapp-dev" {
  dockerfile = "Dockerfile.webapp"
  tags = ["docker.io/username/webapp:${TAG}"]
}

$ docker buildx bake webapp-dev          # will use the default value "latest"
$ TAG=dev docker buildx bake webapp-dev  # will use the TAG environment variable value

Tip

See also the Configuring builds page for advanced usage.

Functions

A set of generally useful functions provided by go-cty are available for use in HCL files: 

# docker-bake.hcl
target "webapp-dev" {
  dockerfile = "Dockerfile.webapp"
  tags = ["docker.io/username/webapp:latest"]
  args = {
    buildno = "${add(123, 1)}"
  }
}

In addition, user defined functions are also supported: 

# docker-bake.hcl
function "increment" {
  params = [number]
  result = number + 1
}

target "webapp-dev" {
  dockerfile = "Dockerfile.webapp"
  tags = ["docker.io/username/webapp:latest"]
  args = {
    buildno = "${increment(123)}"
  }
}

Note

See User defined HCL functions page for more details.

Built-in variables

  • BAKE_CMD_CONTEXT can be used to access the main context for bake command from a bake file that has been imported remotely.
  • BAKE_LOCAL_PLATFORM returns the current platform’s default platform specification (e.g. linux/amd64 ).

Merging and inheritance

Multiple files can include the same target and final build options will be determined by merging them together: 

# docker-bake.hcl
target "webapp-dev" {
  dockerfile = "Dockerfile.webapp"
  tags = ["docker.io/username/webapp:latest"]
}

# docker-bake2.hcl
target "webapp-dev" {
  tags = ["docker.io/username/webapp:dev"]
}

$ docker buildx bake -f docker-bake.hcl -f docker-bake2.hcl webapp-dev

A group can specify its list of targets with the targets option. A target can inherit build options by setting the inherits option to the list of targets or groups to inherit from: 

# docker-bake.hcl
target "webapp-dev" {
  dockerfile = "Dockerfile.webapp"
  tags = ["docker.io/username/webapp:${TAG}"]
}

target "webapp-release" {
  inherits = ["webapp-dev"]
  platforms = ["linux/amd64", "linux/arm64"]
}

default target/group

When you invoke bake you specify what targets/groups you want to build. If no arguments is specified, the group/target named default will be built: 

# docker-bake.hcl
target "default" {
  dockerfile = "Dockerfile.webapp"
  tags = ["docker.io/username/webapp:latest"]
}

$ docker buildx bake

Definitions

HCL definition

HCL definition file is recommended as its experience is more aligned with buildx UX and also allows better code reuse, different target groups and extended features. 

# docker-bake.hcl
variable "TAG" {
  default = "latest"
}

group "default" {
  targets = ["db", "webapp-dev"]
}

target "webapp-dev" {
  dockerfile = "Dockerfile.webapp"
  tags = ["docker.io/username/webapp:${TAG}"]
}

target "webapp-release" {
  inherits = ["webapp-dev"]
  platforms = ["linux/amd64", "linux/arm64"]
}

target "db" {
  dockerfile = "Dockerfile.db"
  tags = ["docker.io/username/db"]
}

JSON definition 

{
  "variable": {
    "TAG": {
      "default": "latest"
    }
  },
  "group": {
    "default": {
      "targets": [
        "db",
        "webapp-dev"
      ]
    }
  },
  "target": {
    "webapp-dev": {
      "dockerfile": "Dockerfile.webapp",
      "tags": [
        "docker.io/username/webapp:${TAG}"
      ]
    },
    "webapp-release": {
      "inherits": [
        "webapp-dev"
      ],
      "platforms": [
        "linux/amd64",
        "linux/arm64"
      ]
    },
    "db": {
      "dockerfile": "Dockerfile.db",
      "tags": [
        "docker.io/username/db"
      ]
    }
  }
}

Compose file 

# docker-compose.yml
services:
  webapp:
    image: docker.io/username/webapp:latest
    build:
      dockerfile: Dockerfile.webapp

  db:
    image: docker.io/username/db
    build:
      dockerfile: Dockerfile.db

Note

See Building from Compose file page for more details.

Remote definition

You can also build bake files directly from a remote Git repository or HTTPS URL: 

$ docker buildx bake "https://github.com/docker/cli.git#v20.10.11" --print
#1 [internal] load git source https://github.com/docker/cli.git#v20.10.11
#1 0.745 e8f1871b077b64bcb4a13334b7146492773769f7       refs/tags/v20.10.11
#1 2.022 From https://github.com/docker/cli
#1 2.022  * [new tag]         v20.10.11  -> v20.10.11
#1 DONE 2.9s

{
  "group": {
    "default": {
      "targets": [
        "binary"
      ]
    }
  },
  "target": {
    "binary": {
      "context": "https://github.com/docker/cli.git#v20.10.11",
      "dockerfile": "Dockerfile",
      "args": {
        "BASE_VARIANT": "alpine",
        "GO_STRIP": "",
        "VERSION": ""
      },
      "target": "binary",
      "platforms": [
        "local"
      ],
      "output": [
        "build"
      ]
    }
  }
}

As you can see the context is fixed to https://github.com/docker/cli.git even if no context is actually defined in the definition.

If you want to access the main context for bake command from a bake file that has been imported remotely, you can use the BAKE_CMD_CONTEXT built-in var. 

$ cat https://raw.githubusercontent.com/tonistiigi/buildx/remote-test/docker-bake.hcl

target "default" {
  context = BAKE_CMD_CONTEXT
  dockerfile-inline = <<EOT
FROM alpine
WORKDIR /src
COPY . .
RUN ls -l && stop
EOT
}

$ docker buildx bake "https://github.com/tonistiigi/buildx.git#remote-test" --print

{
  "target": {
    "default": {
      "context": ".",
      "dockerfile": "Dockerfile",
      "dockerfile-inline": "FROM alpine\nWORKDIR /src\nCOPY . .\nRUN ls -l \u0026\u0026 stop\n"
    }
  }
}

$ touch foo bar
$ docker buildx bake "https://github.com/tonistiigi/buildx.git#remote-test"

...
 > [4/4] RUN ls -l && stop:
#8 0.101 total 0
#8 0.102 -rw-r--r--    1 root     root             0 Jul 27 18:47 bar
#8 0.102 -rw-r--r--    1 root     root             0 Jul 27 18:47 foo
#8 0.102 /bin/sh: stop: not found

$ docker buildx bake "https://github.com/tonistiigi/buildx.git#remote-test" "https://github.com/docker/cli.git#v20.10.11" --print
#1 [internal] load git source https://github.com/tonistiigi/buildx.git#remote-test
#1 0.429 577303add004dd7efeb13434d69ea030d35f7888       refs/heads/remote-test
#1 CACHED

{
  "target": {
    "default": {
      "context": "https://github.com/docker/cli.git#v20.10.11",
      "dockerfile": "Dockerfile",
      "dockerfile-inline": "FROM alpine\nWORKDIR /src\nCOPY . .\nRUN ls -l \u0026\u0026 stop\n"
    }
  }
}

$ docker buildx bake "https://github.com/tonistiigi/buildx.git#remote-test" "https://github.com/docker/cli.git#v20.10.11"

...
 > [4/4] RUN ls -l && stop:
#8 0.136 drwxrwxrwx    5 root     root          4096 Jul 27 18:31 kubernetes
#8 0.136 drwxrwxrwx    3 root     root          4096 Jul 27 18:31 man
#8 0.136 drwxrwxrwx    2 root     root          4096 Jul 27 18:31 opts
#8 0.136 -rw-rw-rw-    1 root     root          1893 Jul 27 18:31 poule.yml
#8 0.136 drwxrwxrwx    7 root     root          4096 Jul 27 18:31 scripts
#8 0.136 drwxrwxrwx    3 root     root          4096 Jul 27 18:31 service
#8 0.136 drwxrwxrwx    2 root     root          4096 Jul 27 18:31 templates
#8 0.136 drwxrwxrwx   10 root     root          4096 Jul 27 18:31 vendor
#8 0.136 -rwxrwxrwx    1 root     root          9620 Jul 27 18:31 vendor.conf
#8 0.136 /bin/sh: stop: not found
Read article

User defined HCL functions

User defined HCL functions

Using interpolation to tag an image with the git sha

As shown in the File definition page, bake supports variable blocks which are assigned to matching environment variables or default values: 

# docker-bake.hcl
variable "TAG" {
  default = "latest"
}

group "default" {
  targets = ["webapp"]
}

target "webapp" {
  tags = ["docker.io/username/webapp:${TAG}"]
}

alternatively, in json format: 

{
  "variable": {
    "TAG": {
      "default": "latest"
    }
  },
  "group": {
    "default": {
      "targets": ["webapp"]
    }
  },
  "target": {
    "webapp": {
      "tags": ["docker.io/username/webapp:${TAG}"]
    }
  }
}

$ docker buildx bake --print webapp

{
  "group": {
    "default": {
      "targets": [
        "webapp"
      ]
    }
  },
  "target": {
    "webapp": {
      "context": ".",
      "dockerfile": "Dockerfile",
      "tags": [
        "docker.io/username/webapp:latest"
      ]
    }
  }
}

$ TAG=$(git rev-parse --short HEAD) docker buildx bake --print webapp

{
  "group": {
    "default": {
      "targets": [
        "webapp"
      ]
    }
  },
  "target": {
    "webapp": {
      "context": ".",
      "dockerfile": "Dockerfile",
      "tags": [
        "docker.io/username/webapp:985e9e9"
      ]
    }
  }
}

Using the add function

You can use go-cty stdlib functions. Here we are using the add function. 

# docker-bake.hcl
variable "TAG" {
  default = "latest"
}

group "default" {
  targets = ["webapp"]
}

target "webapp" {
  args = {
    buildno = "${add(123, 1)}"
  }
}

$ docker buildx bake --print webapp

{
  "group": {
    "default": {
      "targets": [
        "webapp"
      ]
    }
  },
  "target": {
    "webapp": {
      "context": ".",
      "dockerfile": "Dockerfile",
      "args": {
        "buildno": "124"
      }
    }
  }
}

Defining an increment function

It also supports user defined functions. The following example defines a simple an increment function. 

# docker-bake.hcl
function "increment" {
  params = [number]
  result = number + 1
}

group "default" {
  targets = ["webapp"]
}

target "webapp" {
  args = {
    buildno = "${increment(123)}"
  }
}

$ docker buildx bake --print webapp

{
  "group": {
    "default": {
      "targets": [
        "webapp"
      ]
    }
  },
  "target": {
    "webapp": {
      "context": ".",
      "dockerfile": "Dockerfile",
      "args": {
        "buildno": "124"
      }
    }
  }
}

Only adding tags if a variable is not empty using an notequal

Here we are using the conditional notequal function which is just for symmetry with the equal one. 

# docker-bake.hcl
variable "TAG" {default="" }

group "default" {
  targets = [
    "webapp",
  ]
}

target "webapp" {
  context="."
  dockerfile="Dockerfile"
  tags = [
    "my-image:latest",
    notequal("",TAG) ? "my-image:${TAG}": "",
  ]
}

$ docker buildx bake --print webapp

{
  "group": {
    "default": {
      "targets": [
        "webapp"
      ]
    }
  },
  "target": {
    "webapp": {
      "context": ".",
      "dockerfile": "Dockerfile",
      "tags": [
        "my-image:latest"
      ]
    }
  }
}

Using variables in functions

You can refer variables to other variables like the target blocks can. Stdlib functions can also be called but user functions can’t at the moment. 

# docker-bake.hcl
variable "REPO" {
  default = "user/repo"
}

function "tag" {
  params = [tag]
  result = ["${REPO}:${tag}"]
}

target "webapp" {
  tags = tag("v1")
}

$ docker buildx bake --print webapp

{
  "group": {
    "default": {
      "targets": [
        "webapp"
      ]
    }
  },
  "target": {
    "webapp": {
      "context": ".",
      "dockerfile": "Dockerfile",
      "tags": [
        "user/repo:v1"
      ]
    }
  }
}

Using typed variables

Non-string variables are also accepted. The value passed with env is parsed into suitable type first. 

# docker-bake.hcl
variable "FOO" {
  default = 3
}

variable "IS_FOO" {
  default = true
}

target "app" {
  args = {
    v1 = FOO > 5 ? "higher" : "lower" 
    v2 = IS_FOO ? "yes" : "no"
  }
}

$ docker buildx bake --print app

{
  "group": {
    "default": {
      "targets": [
        "app"
      ]
    }
  },
  "target": {
    "app": {
      "context": ".",
      "dockerfile": "Dockerfile",
      "args": {
        "v1": "lower",
        "v2": "yes"
      }
    }
  }
}
Read article