Abstract Kaniko image building to be simpler for users.

[zjrgov]

Building images with the runner is currently somewhat complicated and could be made less so. Here's an example of the current process:

package image:
  stage: package
  image: cloudfoundry/cli:8.7.10
  services:
    - name: gcr.io/kaniko-project/executor:debug
      alias: kaniko
      entrypoint: ["/busybox/sh"]
  script:
    - cf api https://api.fr.cloud.gov
    - cf auth $CLOUD_GOV_USER $CLOUD_GOV_PASS
    - cf target -o gsa-tts-devtools-prototyping -s zjr-gl-test
    - tar -C $CI_PROJECT_DIR -czf bundle.tar.gz .
    - ci_auth=$(echo -n $CI_REGISTRY_USER:$CI_REGISTRY_PASSWORD | base64)
    - >-
      kaniko_cfg="{\"auths\": {\"${CI_REGISTRY}\": {\"auth\": \"${ci_auth}\"}}}"
    - >-
      echo $kaniko_cfg |
      cf ssh "${SERVICE_PREFIX}kaniko" --command
      "/busybox/cat > /kaniko/.docker/config.json"
    - >-
      cat bundle.tar.gz |
      cf ssh "${SERVICE_PREFIX}kaniko" --command
      "/kaniko/executor --context tar://stdin --destination=$APP_IMAGE_TAG"

First the user bundles their project into a tarball.

tar -C $CI_PROJECT_DIR -czf bundle.tar.gz .

Then they create a Docker style auth config in scary escaped inline JSON and send it to the Kaniko service through ssh & cat. I split this up into multiple sections in the job configuration above for the sake of readability. This step could be mitigated by fixing #23.

ci_auth=$(echo -n $CI_REGISTRY_USER:$CI_REGISTRY_PASSWORD | base64)
kaniko_cfg="{\"auths\": {\"${CI_REGISTRY}\": {\"auth\": \"${ci_auth}\"}}}"

echo $kaniko_cfg | 
cf ssh "${SERVICE_PREFIX} kaniko" --command "/busybox/cat > /kaniko/.docker/config.json"

Finally the user ssh/cats the bundle tarball over to the Kaniko service where the kaniko executor reads it from stdin, builds the image, and pushes it to the destination.

cat bundle.tar.gz |
cf ssh "${SERVICE_PREFIX}kaniko" \
--command "/kaniko/executor --context tar://stdin --destination=$APP_IMAGE_TAG"

We could perhaps provide a callable script, or a flag in the service, or something along those lines to run these commands automatically because for the most part these aren't really a user concern—could always allow manual operation for advanced use cases.

[rahearn]

This should all still work with egress proxying, though cf ssh calls need to be switched over to use the cf_ssh style call from the manager

[rahearn]

also probably needed: https://docs.gitlab.com/ee/ci/docker/using_kaniko.html#building-an-image-with-kaniko-behind-a-proxy