DockerClientException: Could not build image when FROM image isn't present

[dbyron0]

On a freshly 'Reset to factory defaults' docker for mac Version 18.03.1-ce-mac65 (24312), I get an DockerClientException with both 1.8.1 and 1.8.2.

$ docker version
Client:
 Version:      18.03.1-ce
 API version:  1.37
 Go version:   go1.9.5
 Git commit:   9ee9f40
 Built:        Thu Apr 26 07:13:02 2018
 OS/Arch:      darwin/amd64
 Experimental: false
 Orchestrator: swarm

Server:
 Engine:
  Version:      18.03.1-ce
  API version:  1.37 (minimum version 1.12)
  Go version:   go1.9.5
  Git commit:   9ee9f40
  Built:        Thu Apr 26 07:22:38 2018
  OS/Arch:      linux/amd64
  Experimental: true

Here's a snippet of the stack trace (same for both 1.8.1 and 1.8.2):

org.testcontainers.containers.ContainerFetchException: Can't get Docker image: org.testcontainers.images.builder.ImageFromDockerfile@477ec507
	at org.testcontainers.containers.GenericContainer.getDockerImageName(GenericContainer.java:940)
	at org.testcontainers.containers.GenericContainer.logger(GenericContainer.java:335)
	at org.testcontainers.containers.GenericContainer.doStart(GenericContainer.java:214)
	at org.testcontainers.containers.GenericContainer.start(GenericContainer.java:209)
Caused by: com.github.dockerjava.api.exception.DockerClientException: Could not build image
	at com.github.dockerjava.core.command.BuildImageResultCallback.getImageId(BuildImageResultCallback.java:76)
	at com.github.dockerjava.core.command.BuildImageResultCallback.awaitImageId(BuildImageResultCallback.java:51)
	at org.testcontainers.images.builder.ImageFromDockerfile.resolve(ImageFromDockerfile.java:141)
	at org.testcontainers.images.builder.ImageFromDockerfile.resolve(ImageFromDockerfile.java:30)
	at org.testcontainers.utility.LazyFuture.getResolvedValue(LazyFuture.java:20)
	at org.testcontainers.utility.LazyFuture.get(LazyFuture.java:27)
	at org.testcontainers.containers.GenericContainer.getDockerImageName(GenericContainer.java:938)
	... 49 more

I don't see anything relevant in the log.

Apparently, when the python image is present before running my tests (via gradle + junit 4), everything works. In other words, after the failure, if I try again, everything works.

$ docker images python
REPOSITORY                     TAG                 IMAGE ID            CREATED             SIZE
python                         2.7                 17c0fe4e76a5        8 days ago          908MB

and when that image isn't there, I get the exception.

My code for building the GenericContainer is:

      final GenericContainer<?> retval = new GenericContainer<>(
            new ImageFromDockerfile()
               .withFileFromClasspath("Dockerfile", dockerfile)
               .withFileFromClasspath("listener.py", listener))
         .withNetwork(network)
         .withNetworkAliases(statsdNetworkAlias)
         .waitingFor(new LogMessageWaitStrategy()
                     .withRegEx(".*listener is up.*\\s")
                     .withStartupTimeout(Duration.of(5, SECONDS)));
      retval.withCreateContainerCmdModifier(cmd -> cmd.withExposedPorts(new ExposedPort(8125, InternetProtocol.UDP)));

      return retval;

and Dockerfile is:

FROM python:2.7

ADD listener.py /

CMD [ "python", "-u", "./listener.py" ]

I went ahead and updated docker for mac to Version 18.06.0-ce-mac70 (26399),

$ docker version
Client:
 Version:           18.06.0-ce
 API version:       1.38
 Go version:        go1.10.3
 Git commit:        0ffa825
 Built:             Wed Jul 18 19:05:26 2018
 OS/Arch:           darwin/amd64
 Experimental:      false

Server:
 Engine:
  Version:          18.06.0-ce
  API version:      1.38 (minimum version 1.12)
  Go version:       go1.10.3
  Git commit:       0ffa825
  Built:            Wed Jul 18 19:13:46 2018
  OS/Arch:          linux/amd64
  Experimental:     true

and see a different caused by in the stack trace (and a log message with the stack trace too) with 1.8.1:

Caused by: com.github.dockerjava.api.exception.InternalServerErrorException: {"message":"archive/tar: cannot encode header: filename may not have trailing slash"}

	at com.github.dockerjava.netty.handler.HttpResponseHandler.channelRead0(HttpResponseHandler.java:109)
	at com.github.dockerjava.netty.handler.HttpResponseHandler.channelRead0(HttpResponseHandler.java:33)
	at org.testcontainers.shaded.io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105)
	at org.testcontainers.shaded.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
	at org.testcontainers.shaded.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
	at org.testcontainers.shaded.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)
	at org.testcontainers.shaded.io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:86)
	at org.testcontainers.shaded.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
	at org.testcontainers.shaded.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
	at org.testcontainers.shaded.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)
	at org.testcontainers.shaded.io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.fireChannelRead(CombinedChannelDuplexHandler.java:438)
	at org.testcontainers.shaded.io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:310)
	at org.testcontainers.shaded.io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:284)
	at org.testcontainers.shaded.io.netty.channel.CombinedChannelDuplexHandler.channelRead(CombinedChannelDuplexHandler.java:253)
	at org.testcontainers.shaded.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
	at org.testcontainers.shaded.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
	at org.testcontainers.shaded.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)
	at org.testcontainers.shaded.io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1342)
	at org.testcontainers.shaded.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
	at org.testcontainers.shaded.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
	at org.testcontainers.shaded.io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:934)
	at org.testcontainers.shaded.io.netty.channel.kqueue.AbstractKQueueStreamChannel$KQueueStreamUnsafe.readReady(AbstractKQueueStreamChannel.java:593)
	at org.testcontainers.shaded.io.netty.channel.kqueue.KQueueDomainSocketChannel$KQueueDomainUnsafe.readReady(KQueueDomainSocketChannel.java:131)
	at org.testcontainers.shaded.io.netty.channel.kqueue.AbstractKQueueChannel$AbstractKQueueUnsafe.readReady(AbstractKQueueChannel.java:367)
	at org.testcontainers.shaded.io.netty.channel.kqueue.KQueueEventLoop.processReady(KQueueEventLoop.java:198)
	at org.testcontainers.shaded.io.netty.channel.kqueue.KQueueEventLoop.run(KQueueEventLoop.java:270)
	at org.testcontainers.shaded.io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:858)
	at org.testcontainers.shaded.io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:138)
	... 1 more

Although with this new docker for mac, there's no python image present after the failure, so subsequent executions still fail. At this point this looks like a dup of #680 / fixed by #808 as the InternalServerErrorException goes away with 1.8.2. Unfortunately, I'm still back to the original DockerClientException with 1.8.2 and docker for mac 18.06.0-ce-mac70 (26399). The python image is again present with 1.8.2, so a subsequent execution works.

[kiview]

I would think the error message is because of wrong copying of the file in the Dockerfile?
ADD listener.py /
would need to be
ADD listener.py listener.py
I think.

[kiview]

Oh, it was something different, probably fixed by #808.

[dbyron0]

Confirmed that this no longer happens with 1.9.1 + docker for mac 18.06.1-ce-mac73 (26764).

[kiview]

Thanks for the feedback 🙂