Duplicate DB containers with Tomcat DB connection pool and Testcontainers JDBC driver

[postalservice14]

When running a very simple test on a new Spring Boot project, the project is creating 10 postgresql containers. This appears to be because of the connection pooling. Below is the configuration used to reproduce the 10 container issue in Spring Boot.

To fix this:
Add spring.datasource.max-active=1 to your tests application.properties file.

Is this an issue? Seems like it should just create 10 connections to the same container...

My properties file

spring.datasource.url=jdbc:tc:postgresql://hostname/databasename
spring.datasource.driver-class-name=org.testcontainers.jdbc.ContainerDatabaseDriver
spring.jpa.database=postgresql

spring.jpa.hibernate.ddl-auto=create-drop

My Test

package com.mycompany.controller;

import com.mycompany.LeadServiceApplication;
import com.mycompany.entity.User;
import com.mycompany.entity.Lead;
import com.mycompany.repository.UserRepository;
import org.junit.Assert;
import org.junit.Before;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.SpringApplicationConfiguration;
import org.springframework.http.MediaType;
import org.springframework.http.converter.HttpMessageConverter;
import org.springframework.http.converter.json.MappingJackson2HttpMessageConverter;
import org.springframework.mock.http.MockHttpOutputMessage;
import org.springframework.test.context.junit4.SpringJUnit4ClassRunner;
import org.springframework.test.context.web.WebAppConfiguration;
import org.springframework.test.web.servlet.MockMvc;
import org.springframework.web.context.WebApplicationContext;

import java.io.IOException;
import java.util.Arrays;

import static org.springframework.test.web.servlet.request.MockMvcRequestBuilders.post;
import static org.springframework.test.web.servlet.result.MockMvcResultMatchers.status;
import static org.springframework.test.web.servlet.setup.MockMvcBuilders.webAppContextSetup;

@RunWith(SpringJUnit4ClassRunner.class)
@SpringApplicationConfiguration(classes = LeadServiceApplication.class)
@WebAppConfiguration
public class LeadsRestControllerTest {

    private MediaType contentType = new MediaType(MediaType.APPLICATION_JSON.getType(),
        MediaType.APPLICATION_JSON.getSubtype());

    private MockMvc mockMvc;

    @Autowired
    private WebApplicationContext webApplicationContext;

    @Autowired
    private UserRepository userRepository;

    private HttpMessageConverter mappingJackson2HttpMessageConverter;

    @Autowired
    private void setConverters(HttpMessageConverter<?>[] converters) {
        this.mappingJackson2HttpMessageConverter = Arrays.asList(converters).stream().filter(
            hmc -> hmc instanceof MappingJackson2HttpMessageConverter).findAny().get();

        Assert.assertNotNull("the JSON message converter must not be null",
            this.mappingJackson2HttpMessageConverter);
    }

    @Before
    public void setUp() {
        this.mockMvc = webAppContextSetup(webApplicationContext).build();
    }

    @Test
    public void postInitialLeadSuccess() throws Exception {
        // Setup test
        userRepository.save(new User("1234", "Bob", "Jones"));

        String leadJson = json(new Lead(
            "1234", "Ford", "12345"));
        mockMvc.perform(post("/leads/initial")
            .contentType(contentType)
            .content(leadJson))
            .andExpect(status().isCreated());
    }

    private String json(Lead lead) throws IOException {
        MockHttpOutputMessage mockHttpOutputMessage = new MockHttpOutputMessage();
        this.mappingJackson2HttpMessageConverter.write(
            lead, MediaType.APPLICATION_JSON, mockHttpOutputMessage
        );
        return mockHttpOutputMessage.getBodyAsString();
    }

}

mvn test output:

...
-------------------------------------------------------
 T E S T S
-------------------------------------------------------
objc[9542]: Class JavaLaunchHelper is implemented in both /Library/Java/JavaVirtualMachines/jdk1.8.0_92.jdk/Contents/Home/jre/bin/java and /Library/Java/JavaVirtualMachines/jdk1.8.0_92.jdk/Contents/Home/jre/lib/libinstrument.dylib. One of the two will be used. Which one is undefined.
08:56:40.278 [main] INFO  org.springframework.test.context.support.AbstractContextLoader - Could not detect default resource locations for test class [com.mycompany.controller.LeadsRestControllerTest]: no resource found for suffixes {-context.xml, Context.groovy}.
08:56:40.302 [main] INFO  org.springframework.test.context.web.WebTestContextBootstrapper - Loaded default TestExecutionListener class names from location [META-INF/spring.factories]: [org.springframework.test.context.web.ServletTestExecutionListener, org.springframework.test.context.support.DirtiesContextBeforeModesTestExecutionListener, org.springframework.test.context.support.DependencyInjectionTestExecutionListener, org.springframework.test.context.support.DirtiesContextTestExecutionListener, org.springframework.test.context.transaction.TransactionalTestExecutionListener, org.springframework.test.context.jdbc.SqlScriptsTestExecutionListener]
08:56:40.331 [main] INFO  org.springframework.test.context.web.WebTestContextBootstrapper - Using TestExecutionListeners: [org.springframework.test.context.web.ServletTestExecutionListener@c2e3264, org.springframework.test.context.support.DirtiesContextBeforeModesTestExecutionListener@107f4980, org.springframework.test.context.support.DependencyInjectionTestExecutionListener@75a118e6, org.springframework.test.context.support.DirtiesContextTestExecutionListener@1d540566, org.springframework.test.context.transaction.TransactionalTestExecutionListener@6014a9ba, org.springframework.test.context.jdbc.SqlScriptsTestExecutionListener@acdcf71]
Running com.mycompany.controller.LeadsRestControllerTest
08:56:40.345 [main] INFO  org.springframework.test.context.support.AbstractContextLoader - Could not detect default resource locations for test class [com.mycompany.controller.LeadsRestControllerTest]: no resource found for suffixes {-context.xml, Context.groovy}.
08:56:40.346 [main] INFO  org.springframework.test.context.web.WebTestContextBootstrapper - Loaded default TestExecutionListener class names from location [META-INF/spring.factories]: [org.springframework.test.context.web.ServletTestExecutionListener, org.springframework.test.context.support.DirtiesContextBeforeModesTestExecutionListener, org.springframework.test.context.support.DependencyInjectionTestExecutionListener, org.springframework.test.context.support.DirtiesContextTestExecutionListener, org.springframework.test.context.transaction.TransactionalTestExecutionListener, org.springframework.test.context.jdbc.SqlScriptsTestExecutionListener]
08:56:40.347 [main] INFO  org.springframework.test.context.web.WebTestContextBootstrapper - Using TestExecutionListeners: [org.springframework.test.context.web.ServletTestExecutionListener@59e43e8c, org.springframework.test.context.support.DirtiesContextBeforeModesTestExecutionListener@2caa5d7c, org.springframework.test.context.support.DependencyInjectionTestExecutionListener@5e671e20, org.springframework.test.context.support.DirtiesContextTestExecutionListener@3eabe84a, org.springframework.test.context.transaction.TransactionalTestExecutionListener@46c3a14d, org.springframework.test.context.jdbc.SqlScriptsTestExecutionListener@38fc5554]

  .   ____          _            __ _ _
 /\\ / ___'_ __ _ _(_)_ __  __ _ \ \ \ \
( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
 \\/  ___)| |_)| | | | | || (_| |  ) ) ) )
  '  |____| .__|_| |_|_| |_\__, | / / / /
 =========|_|==============|___/=/_/_/_/
 :: Spring Boot ::        (v1.3.5.RELEASE)

08:56:40.683 [main] INFO  com.mycompany.controller.LeadsRestControllerTest - Starting LeadsRestControllerTest on Johns-MacBook-Pro.local with PID 9542 (/Users/myname/Projects/MyCompany/lead-service/target/test-classes started by myname in /Users/myname/Projects/MyCompany/lead-service)
08:56:40.683 [main] INFO  com.mycompany.controller.LeadsRestControllerTest - No active profile set, falling back to default profiles: default
08:56:40.752 [main] INFO  org.springframework.web.context.support.GenericWebApplicationContext - Refreshing org.springframework.web.context.support.GenericWebApplicationContext@6ef037e4: startup date [Wed Jun 15 08:56:40 EDT 2016]; root of context hierarchy
08:56:41.226 [background-preinit] INFO  org.hibernate.validator.internal.util.Version - HV000001: Hibernate Validator 5.2.4.Final
08:56:41.847 [main] INFO  org.springframework.context.support.PostProcessorRegistrationDelegate$BeanPostProcessorChecker - Bean 'org.springframework.transaction.annotation.ProxyTransactionManagementConfiguration' of type [class org.springframework.transaction.annotation.ProxyTransactionManagementConfiguration$$EnhancerBySpringCGLIB$$b5577074] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying)
08:56:42.178 [main] INFO  org.springframework.orm.jpa.LocalContainerEntityManagerFactoryBean - Building JPA container EntityManagerFactory for persistence unit 'default'
08:56:42.205 [main] INFO  org.hibernate.jpa.internal.util.LogHelper - HHH000204: Processing PersistenceUnitInfo [
    name: default
    ...]
08:56:42.289 [main] INFO  org.hibernate.Version - HHH000412: Hibernate Core {4.3.11.Final}
08:56:42.291 [main] INFO  org.hibernate.cfg.Environment - HHH000206: hibernate.properties not found
08:56:42.293 [main] INFO  org.hibernate.cfg.Environment - HHH000021: Bytecode provider name : javassist
08:56:42.557 [main] INFO  org.hibernate.annotations.common.Version - HCANN000001: Hibernate Commons Annotations {4.0.5.Final}
08:56:42.696 [main] INFO  org.testcontainers.dockerclient.DockerConfigurationStrategy - Looking for Docker environment. Trying Environment variables, system properties and defaults. Resolved: 
    uri=https://localhost:2376
    sslConfig='LocalDirectorySSLConfig{dockerCertPath=/Users/myname/.docker}'
    version='{UNKNOWN_VERSION}'
    username='myname'
    password='null'
    email='null'
    serverAddress='https://index.docker.io/v1/'
    dockerCfgPath='/Users/myname/.dockercfg'

08:56:43.339 [main] INFO  org.testcontainers.shaded.org.apache.http.impl.execchain.RetryExec - I/O exception (org.testcontainers.shaded.org.apache.http.conn.UnsupportedSchemeException) caught when processing request: https protocol is not supported
08:56:43.340 [main] INFO  org.testcontainers.shaded.org.apache.http.impl.execchain.RetryExec - Retrying request
08:56:43.340 [main] INFO  org.testcontainers.shaded.org.apache.http.impl.execchain.RetryExec - I/O exception (org.testcontainers.shaded.org.apache.http.conn.UnsupportedSchemeException) caught when processing request: https protocol is not supported
08:56:43.340 [main] INFO  org.testcontainers.shaded.org.apache.http.impl.execchain.RetryExec - Retrying request
08:56:43.341 [main] INFO  org.testcontainers.shaded.org.apache.http.impl.execchain.RetryExec - I/O exception (org.testcontainers.shaded.org.apache.http.conn.UnsupportedSchemeException) caught when processing request: https protocol is not supported
08:56:43.341 [main] INFO  org.testcontainers.shaded.org.apache.http.impl.execchain.RetryExec - Retrying request
08:56:43.341 [main] INFO  org.testcontainers.dockerclient.DockerConfigurationStrategy - Looking for Docker environment. Trying docker-machine
08:56:43.396 [main] INFO  org.testcontainers.dockerclient.DockerConfigurationStrategy - Found docker-machine, and will use machine named default
08:56:43.743 [main] INFO  org.testcontainers.dockerclient.DockerConfigurationStrategy - Docker daemon IP address for docker machine default is 192.168.99.100
08:56:44.877 [main] INFO  org.testcontainers.DockerClientFactory - Disk utilization in Docker environment is 17% (14450 MB available)
08:56:45.084 [main] INFO  🐳 [postgres:latest] - Creating container for image: postgres:latest
08:56:45.221 [main] INFO  🐳 [postgres:latest] - Starting container with ID: cefed6498a55eb7673d69c054b8f7a1e054731041af7e07ff822fb16f9da0567
08:56:45.369 [main] INFO  🐳 [postgres:latest] - Container postgres:latest is starting: cefed6498a55eb7673d69c054b8f7a1e054731041af7e07ff822fb16f9da0567
08:56:45.515 [main] INFO  🐳 [postgres:latest] - Waiting for database connection to become available at jdbc:postgresql://192.168.99.100:32788/test using query 'SELECT 1'
08:56:49.954 [pool-2-thread-1] INFO  🐳 [postgres:latest] - Obtained a connection to container (jdbc:postgresql://192.168.99.100:32788/test)
08:56:49.954 [main] INFO  🐳 [postgres:latest] - Container postgres:latest started
08:56:50.024 [main] INFO  🐳 [postgres:latest] - Creating container for image: postgres:latest
08:56:50.091 [main] INFO  🐳 [postgres:latest] - Starting container with ID: 4693d7d35d3cd10465c3c96603b26154e16f80ea024ccb03e2936ee4b0388e55
08:56:50.273 [main] INFO  🐳 [postgres:latest] - Container postgres:latest is starting: 4693d7d35d3cd10465c3c96603b26154e16f80ea024ccb03e2936ee4b0388e55
08:56:50.393 [main] INFO  🐳 [postgres:latest] - Waiting for database connection to become available at jdbc:postgresql://192.168.99.100:32789/test using query 'SELECT 1'
08:56:54.920 [pool-2-thread-1] INFO  🐳 [postgres:latest] - Obtained a connection to container (jdbc:postgresql://192.168.99.100:32789/test)
08:56:54.920 [main] INFO  🐳 [postgres:latest] - Container postgres:latest started
08:56:54.984 [main] INFO  🐳 [postgres:latest] - Creating container for image: postgres:latest
08:56:55.047 [main] INFO  🐳 [postgres:latest] - Starting container with ID: 5de3bac95c1d6e51b138d02a2ce8a00a1d8a250b3bdb8434a3eaac3d42c5493c
08:56:55.186 [main] INFO  🐳 [postgres:latest] - Container postgres:latest is starting: 5de3bac95c1d6e51b138d02a2ce8a00a1d8a250b3bdb8434a3eaac3d42c5493c
08:56:55.292 [main] INFO  🐳 [postgres:latest] - Waiting for database connection to become available at jdbc:postgresql://192.168.99.100:32790/test using query 'SELECT 1'
08:57:00.206 [pool-2-thread-1] INFO  🐳 [postgres:latest] - Obtained a connection to container (jdbc:postgresql://192.168.99.100:32790/test)
08:57:00.206 [main] INFO  🐳 [postgres:latest] - Container postgres:latest started
08:57:00.273 [main] INFO  🐳 [postgres:latest] - Creating container for image: postgres:latest
08:57:00.352 [main] INFO  🐳 [postgres:latest] - Starting container with ID: 1d08f8ed2f9e3062744afdadc88e5f0a1f64fad01c305667be170188bd1fe37f
08:57:00.478 [main] INFO  🐳 [postgres:latest] - Container postgres:latest is starting: 1d08f8ed2f9e3062744afdadc88e5f0a1f64fad01c305667be170188bd1fe37f
08:57:00.581 [main] INFO  🐳 [postgres:latest] - Waiting for database connection to become available at jdbc:postgresql://192.168.99.100:32791/test using query 'SELECT 1'
08:57:05.020 [pool-2-thread-1] INFO  🐳 [postgres:latest] - Obtained a connection to container (jdbc:postgresql://192.168.99.100:32791/test)
08:57:05.020 [main] INFO  🐳 [postgres:latest] - Container postgres:latest started
08:57:05.084 [main] INFO  🐳 [postgres:latest] - Creating container for image: postgres:latest
08:57:05.153 [main] INFO  🐳 [postgres:latest] - Starting container with ID: f32877c30cef4e779eeebca125c96a01cb6b98258dac810250d6f8c4fdc0113a
08:57:05.328 [main] INFO  🐳 [postgres:latest] - Container postgres:latest is starting: f32877c30cef4e779eeebca125c96a01cb6b98258dac810250d6f8c4fdc0113a
08:57:05.452 [main] INFO  🐳 [postgres:latest] - Waiting for database connection to become available at jdbc:postgresql://192.168.99.100:32792/test using query 'SELECT 1'
08:57:09.168 [pool-2-thread-1] INFO  🐳 [postgres:latest] - Obtained a connection to container (jdbc:postgresql://192.168.99.100:32792/test)
08:57:09.168 [main] INFO  🐳 [postgres:latest] - Container postgres:latest started
08:57:09.266 [main] INFO  🐳 [postgres:latest] - Creating container for image: postgres:latest
08:57:09.328 [main] INFO  🐳 [postgres:latest] - Starting container with ID: 0c516deef1d52b3e88cb012127af3e76f9795cf218752e47611909ea50611574
08:57:09.475 [main] INFO  🐳 [postgres:latest] - Container postgres:latest is starting: 0c516deef1d52b3e88cb012127af3e76f9795cf218752e47611909ea50611574
08:57:09.591 [main] INFO  🐳 [postgres:latest] - Waiting for database connection to become available at jdbc:postgresql://192.168.99.100:32793/test using query 'SELECT 1'
08:57:14.186 [pool-2-thread-1] INFO  🐳 [postgres:latest] - Obtained a connection to container (jdbc:postgresql://192.168.99.100:32793/test)
08:57:14.187 [main] INFO  🐳 [postgres:latest] - Container postgres:latest started
08:57:14.261 [main] INFO  🐳 [postgres:latest] - Creating container for image: postgres:latest
08:57:14.328 [main] INFO  🐳 [postgres:latest] - Starting container with ID: 46d64d372be22d09c8b074adab58aa19bd117f0318dced3155d8275a00b7d50e
08:57:14.469 [main] INFO  🐳 [postgres:latest] - Container postgres:latest is starting: 46d64d372be22d09c8b074adab58aa19bd117f0318dced3155d8275a00b7d50e
08:57:14.597 [main] INFO  🐳 [postgres:latest] - Waiting for database connection to become available at jdbc:postgresql://192.168.99.100:32794/test using query 'SELECT 1'
08:57:19.245 [pool-2-thread-1] INFO  🐳 [postgres:latest] - Obtained a connection to container (jdbc:postgresql://192.168.99.100:32794/test)
08:57:19.246 [main] INFO  🐳 [postgres:latest] - Container postgres:latest started
08:57:19.313 [main] INFO  🐳 [postgres:latest] - Creating container for image: postgres:latest
08:57:19.376 [main] INFO  🐳 [postgres:latest] - Starting container with ID: d6989c9106a4ac5a4e2707aaef2df05ee2af84e697ce220ec4c68a0e8c084f5c
08:57:19.520 [main] INFO  🐳 [postgres:latest] - Container postgres:latest is starting: d6989c9106a4ac5a4e2707aaef2df05ee2af84e697ce220ec4c68a0e8c084f5c
08:57:19.620 [main] INFO  🐳 [postgres:latest] - Waiting for database connection to become available at jdbc:postgresql://192.168.99.100:32795/test using query 'SELECT 1'
08:57:24.062 [pool-2-thread-1] INFO  🐳 [postgres:latest] - Obtained a connection to container (jdbc:postgresql://192.168.99.100:32795/test)
08:57:24.062 [main] INFO  🐳 [postgres:latest] - Container postgres:latest started
08:57:24.125 [main] INFO  🐳 [postgres:latest] - Creating container for image: postgres:latest
08:57:24.189 [main] INFO  🐳 [postgres:latest] - Starting container with ID: 91225307f29d09769bbe30c35b0d837c136162bbf9f0b97fad522c7b2421706f
08:57:24.335 [main] INFO  🐳 [postgres:latest] - Container postgres:latest is starting: 91225307f29d09769bbe30c35b0d837c136162bbf9f0b97fad522c7b2421706f
08:57:24.431 [main] INFO  🐳 [postgres:latest] - Waiting for database connection to become available at jdbc:postgresql://192.168.99.100:32796/test using query 'SELECT 1'
08:57:29.464 [pool-2-thread-1] INFO  🐳 [postgres:latest] - Obtained a connection to container (jdbc:postgresql://192.168.99.100:32796/test)
08:57:29.464 [main] INFO  🐳 [postgres:latest] - Container postgres:latest started
08:57:29.524 [main] INFO  🐳 [postgres:latest] - Creating container for image: postgres:latest
08:57:29.593 [main] INFO  🐳 [postgres:latest] - Starting container with ID: 6f5d5fbcfff4530ed8584cff83edad568af4580cce2f806a598c0c01a8a05e24
08:57:29.735 [main] INFO  🐳 [postgres:latest] - Container postgres:latest is starting: 6f5d5fbcfff4530ed8584cff83edad568af4580cce2f806a598c0c01a8a05e24
08:57:29.833 [main] INFO  🐳 [postgres:latest] - Waiting for database connection to become available at jdbc:postgresql://192.168.99.100:32797/test using query 'SELECT 1'
08:57:34.464 [pool-2-thread-1] INFO  🐳 [postgres:latest] - Obtained a connection to container (jdbc:postgresql://192.168.99.100:32797/test)
08:57:34.464 [main] INFO  🐳 [postgres:latest] - Container postgres:latest started
08:57:34.671 [main] INFO  org.hibernate.dialect.Dialect - HHH000400: Using dialect: org.hibernate.dialect.PostgreSQLDialect
08:57:34.680 [main] INFO  org.hibernate.engine.jdbc.internal.LobCreatorBuilder - HHH000424: Disabling contextual LOB creation as createClob() method threw error : java.lang.reflect.InvocationTargetException
08:57:34.773 [main] INFO  org.hibernate.hql.internal.ast.ASTQueryTranslatorFactory - HHH000397: Using ASTQueryTranslatorFactory
08:57:35.014 [main] INFO  org.hibernate.tool.hbm2ddl.SchemaExport - HHH000227: Running hbm2ddl schema export
08:57:35.022 [main] INFO  org.hibernate.tool.hbm2ddl.SchemaExport - HHH000230: Schema export complete
08:57:35.655 [main] INFO  org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter - Looking for @ControllerAdvice: org.springframework.web.context.support.GenericWebApplicationContext@6ef037e4: startup date [Wed Jun 15 08:56:40 EDT 2016]; root of context hierarchy
08:57:35.725 [main] INFO  org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerMapping - Mapped "{[/leads/initial],methods=[POST]}" onto public org.springframework.http.ResponseEntity<?> com.mycompany.controller.LeadsRestController.addInitialLead(com.mycompany.entity.Lead)
08:57:35.727 [main] INFO  org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerMapping - Mapped "{[/error]}" onto public org.springframework.http.ResponseEntity<java.util.Map<java.lang.String, java.lang.Object>> org.springframework.boot.autoconfigure.web.BasicErrorController.error(javax.servlet.http.HttpServletRequest)
08:57:35.727 [main] INFO  org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerMapping - Mapped "{[/error],produces=[text/html]}" onto public org.springframework.web.servlet.ModelAndView org.springframework.boot.autoconfigure.web.BasicErrorController.errorHtml(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse)
08:57:35.747 [main] INFO  org.springframework.web.servlet.handler.SimpleUrlHandlerMapping - Mapped URL path [/webjars/**] onto handler of type [class org.springframework.web.servlet.resource.ResourceHttpRequestHandler]
08:57:35.747 [main] INFO  org.springframework.web.servlet.handler.SimpleUrlHandlerMapping - Mapped URL path [/**] onto handler of type [class org.springframework.web.servlet.resource.ResourceHttpRequestHandler]
08:57:35.775 [main] INFO  org.springframework.web.servlet.handler.SimpleUrlHandlerMapping - Mapped URL path [/**/favicon.ico] onto handler of type [class org.springframework.web.servlet.resource.ResourceHttpRequestHandler]
08:57:35.939 [main] INFO  com.mycompany.controller.LeadsRestControllerTest - Started LeadsRestControllerTest in 55.48 seconds (JVM running for 56.812)
08:57:35.971 [main] INFO  org.springframework.boot.test.SpringBootMockServletContext - Initializing Spring FrameworkServlet ''
08:57:35.971 [main] INFO  org.springframework.test.web.servlet.TestDispatcherServlet - FrameworkServlet '': initialization started
08:57:35.981 [main] INFO  org.springframework.test.web.servlet.TestDispatcherServlet - FrameworkServlet '': initialization completed in 10 ms
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 55.843 sec - in com.mycompany.controller.LeadsRestControllerTest
08:57:36.183 [Thread-7] INFO  org.springframework.web.context.support.GenericWebApplicationContext - Closing org.springframework.web.context.support.GenericWebApplicationContext@6ef037e4: startup date [Wed Jun 15 08:56:40 EDT 2016]; root of context hierarchy
08:57:36.195 [Thread-7] INFO  org.springframework.orm.jpa.LocalContainerEntityManagerFactoryBean - Closing JPA EntityManagerFactory for persistence unit 'default'
08:57:36.195 [Thread-7] INFO  org.hibernate.tool.hbm2ddl.SchemaExport - HHH000227: Running hbm2ddl schema export
08:57:36.198 [Thread-7] INFO  org.hibernate.tool.hbm2ddl.SchemaExport - HHH000230: Schema export complete
08:57:36.474 [Thread-6] INFO  org.testcontainers.utility.ContainerReaper - Removed container and associated volume(s): postgres:latest
08:57:36.513 [Thread-7] INFO  org.testcontainers.utility.ContainerReaper - Removed container and associated volume(s): postgres:latest
08:57:36.686 [Thread-6] INFO  org.testcontainers.utility.ContainerReaper - Removed container and associated volume(s): postgres:latest
08:57:36.716 [Thread-7] INFO  org.testcontainers.utility.ContainerReaper - Removed container and associated volume(s): postgres:latest
08:57:36.894 [Thread-6] INFO  org.testcontainers.utility.ContainerReaper - Removed container and associated volume(s): postgres:latest
08:57:37.094 [Thread-6] INFO  org.testcontainers.utility.ContainerReaper - Removed container and associated volume(s): postgres:latest
08:57:37.175 [Thread-7] INFO  org.testcontainers.utility.ContainerReaper - Removed container and associated volume(s): postgres:latest
08:57:37.294 [Thread-6] INFO  org.testcontainers.utility.ContainerReaper - Removed container and associated volume(s): postgres:latest
08:57:37.489 [Thread-6] INFO  org.testcontainers.utility.ContainerReaper - Removed container and associated volume(s): postgres:latest
08:57:37.564 [Thread-7] INFO  org.testcontainers.utility.ContainerReaper - Removed container and associated volume(s): postgres:latest

Results :

Tests run: 1, Failures: 0, Errors: 0, Skipped: 0

[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 1:00.353s
[INFO] Finished at: Wed Jun 15 08:57:37 EDT 2016
[INFO] Final Memory: 33M/400M
[INFO] ------------------------------------------------------------------------

[rnorth]

Hi John
Thanks for reporting this - it definitely sounds like a bug. There is code
in the Testcontainers JDBC proxy driver that's supposed to prevent exactly
this problem. Two connections with the same connection string should be
mapped to the same container.

Clearly there's something not working as it should in this case - apologies.

I'll try and have a look into this in the next few days and get back to
you. The code samples should help a lot, so thank you.

Richard

[robshep]

The Driver is being instantiated multiple times, per pool connection.
And the cache is an instance member, so one container per Driver instance.

The tomcat connection pool has PooledConnection which holds a reference to a Driver instance which it creates using ...newInstance();

One option might be to make the jdbcUrlContainerCache static but I don't know how you would clean it up between tests etc.
If it can cleaned via the Rule, then it may be worthwhile looking at keying on the hascode of the url +":"+ the hashcode of the Properties that are passed in. For tomcat connection pool this is an instance shared amongst all the pooledconnections.

[robshep]

Setting the pool size to 1 didn't help me as liquibase and some other setup scripts doesn't play nice with just one connection in the pool, so I gave this a go as described above.

https://github.com/robshep/testcontainers-java/commit/5e96a656d66e6241a7d430cdfebdca9a8bc89b50

Worked OK for me, for my initial smoketest.

[rnorth]

@robshep thank you for looking into this and for the patch. If newInstance() is being used to instantiate then it seems a static field is the only option, and it's roughly analagous to there being a singleton instance of ContainerDatabaseDriver with an instance variable, really.

Re cleanup between tests perhaps we could go with an approach similar to using an H2 in memory database - remove the container from the cache when the last connection to it is closed. We could even borrow the DB_CLOSE_ON_EXIT parameter name from H2 so that it's reasonably familiar to H2 users.

Additionally a simple and sensible thing to do would be expose a method to tests that allows the DB to be programmatically cleared in a @ Before block, for example.

What do you think?

[robshep]

I think given that it is an integration testing component, not meant for production, and short lived, it doesn't need to be the cleanest or purest solution, as long as it gets the job done.

I can live with a memory leak in the testing framework, but having an exposed method, as you suggest will allow conscientious maintenance.

I just looked at the relationship between the individual test methods and my hibernate context, as prepared by spring boot.

Spring boot provides a full application context before any methods are run, and you can inject spring components into the test classes.
As such, with everything managed by the framework, you end up with a hibernate entityManager running atop a managed datasource, running atop a managed connection pool.
All of this is re-used over the whole suite, unless a class is marked with @DirtiesContext.

Here are my observations thus, using stacktraces to show where/how it is used,

This one is built at the start of the unit test class, as the spring context is booting
at org.testcontainers.jdbc.ContainerDatabaseDriver.connect(ContainerDatabaseDriver.java:93) at org.apache.tomcat.jdbc.pool.PooledConnection.connectUsingDriver(PooledConnection.java:307) at org.apache.tomcat.jdbc.pool.PooledConnection.connect(PooledConnection.java:200) at org.apache.tomcat.jdbc.pool.ConnectionPool.createConnection(ConnectionPool.java:708) at org.apache.tomcat.jdbc.pool.ConnectionPool.borrowConnection(ConnectionPool.java:642) at org.apache.tomcat.jdbc.pool.ConnectionPool.init(ConnectionPool.java:464) at org.apache.tomcat.jdbc.pool.ConnectionPool.<init>(ConnectionPool.java:141) at org.apache.tomcat.jdbc.pool.DataSourceProxy.pCreatePool(DataSourceProxy.java:115) at org.apache.tomcat.jdbc.pool.DataSourceProxy.createPool(DataSourceProxy.java:102) at org.apache.tomcat.jdbc.pool.DataSourceProxy.getConnection(DataSourceProxy.java:126) at org.hibernate.engine.jdbc.connections.internal.DatasourceConnectionProviderImpl.getConnection(DatasourceConnectionProviderImpl.java:139) at org.hibernate.engine.jdbc.internal.JdbcServicesImpl$ConnectionProviderJdbcConnectionAccess.obtainConnection(JdbcServicesImpl.java:279) at org.hibernate.engine.jdbc.internal.JdbcServicesImpl.configure(JdbcServicesImpl.java:124) at org.hibernate.boot.registry.internal.StandardServiceRegistryImpl.configureService(StandardServiceRegistryImpl.java:111) at org.hibernate.service.internal.AbstractServiceRegistryImpl.initializeService(AbstractServiceRegistryImpl.java:234) at org.hibernate.service.internal.AbstractServiceRegistryImpl.getService(AbstractServiceRegistryImpl.java:206) at org.hibernate.cfg.Configuration.buildTypeRegistrations(Configuration.java:1887) at org.hibernate.cfg.Configuration.buildSessionFactory(Configuration.java:1845) at org.hibernate.jpa.boot.internal.EntityManagerFactoryBuilderImpl$4.perform(EntityManagerFactoryBuilderImpl.java:857) at org.hibernate.jpa.boot.internal.EntityManagerFactoryBuilderImpl$4.perform(EntityManagerFactoryBuilderImpl.java:850) at org.hibernate.boot.registry.classloading.internal.ClassLoaderServiceImpl.withTccl(ClassLoaderServiceImpl.java:425) at org.hibernate.jpa.boot.internal.EntityManagerFactoryBuilderImpl.build(EntityManagerFactoryBuilderImpl.java:849) at org.springframework.orm.jpa.vendor.SpringHibernateJpaPersistenceProvider.createContainerEntityManagerFactory(SpringHibernateJpaPersistenceProvider.java:60) at org.springframework.orm.jpa.LocalContainerEntityManagerFactoryBean.createNativeEntityManagerFactory(LocalContainerEntityManagerFactoryBean.java:343) at org.springframework.orm.jpa.AbstractEntityManagerFactoryBean.afterPropertiesSet(AbstractEntityManagerFactoryBean.java:319) at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:1637) at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1574) at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:545) at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:482) at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:306) at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:230) at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:302) at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:197) at org.springframework.context.support.AbstractApplicationContext.getBean(AbstractApplicationContext.java:1054) at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:829) at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:538) at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:766) at org.springframework.boot.SpringApplication.createAndRefreshContext(SpringApplication.java:361) at org.springframework.boot.SpringApplication.run(SpringApplication.java:307) at org.springframework.boot.test.SpringApplicationContextLoader.loadContext(SpringApplicationContextLoader.java:98) at org.springframework.test.context.cache.DefaultCacheAwareContextLoaderDelegate.loadContextInternal(DefaultCacheAwareContextLoaderDelegate.java:98) at org.springframework.test.context.cache.DefaultCacheAwareContextLoaderDelegate.loadContext(DefaultCacheAwareContextLoaderDelegate.java:116) at org.springframework.test.context.support.DefaultTestContext.getApplicationContext(DefaultTestContext.java:83) at org.springframework.test.context.web.ServletTestExecutionListener.setUpRequestContextIfNecessary(ServletTestExecutionListener.java:183) at org.springframework.test.context.web.ServletTestExecutionListener.prepareTestInstance(ServletTestExecutionListener.java:123) at org.springframework.test.context.TestContextManager.prepareTestInstance(TestContextManager.java:228) at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.createTest(SpringJUnit4ClassRunner.java:230) at org.springframework.test.context.junit4.SpringJUnit4ClassRunner$1.runReflectiveCall(SpringJUnit4ClassRunner.java:289) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.methodBlock(SpringJUnit4ClassRunner.java:291) at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.runChild(SpringJUnit4ClassRunner.java:249) at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.runChild(SpringJUnit4ClassRunner.java:89) at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288) at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.springframework.test.context.junit4.statements.RunBeforeTestClassCallbacks.evaluate(RunBeforeTestClassCallbacks.java:61) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.springframework.test.context.junit4.statements.RunAfterTestClassCallbacks.evaluate(RunAfterTestClassCallbacks.java:70) at org.junit.runners.ParentRunner.run(ParentRunner.java:363) at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.run(SpringJUnit4ClassRunner.java:193)

My "static" alteration defined above seem to allow this to work fine, with no memory leaking.

But then upon each test method, the @Rule causes further containers to be created, but aren't used because the spring framework (from spring-boot's setup magic), the managed connection pool, the managed datasource, and ultimately hibernate is already glued together with the first container.

Here is where the Rule approach kicks in needlessly in this scenario

at org.testcontainers.containers.GenericContainer.tryStart(GenericContainer.java:170) at org.testcontainers.containers.GenericContainer.lambda$start$0(GenericContainer.java:160) at org.rnorth.ducttape.unreliables.Unreliables.retryUntilSuccess(Unreliables.java:76) at org.testcontainers.containers.GenericContainer.start(GenericContainer.java:157) at org.testcontainers.containers.GenericContainer.starting(GenericContainer.java:451) at org.testcontainers.containers.FailureDetectingExternalResource$1.evaluate(FailureDetectingExternalResource.java:28) at org.junit.rules.RunRules.evaluate(RunRules.java:20) at org.springframework.test.context.junit4.statements.SpringRepeat.evaluate(SpringRepeat.java:84) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325) at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.runChild(SpringJUnit4ClassRunner.java:254) at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.runChild(SpringJUnit4ClassRunner.java:89) at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288) at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.springframework.test.context.junit4.statements.RunBeforeTestClassCallbacks.evaluate(RunBeforeTestClassCallbacks.java:61) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.springframework.test.context.junit4.statements.RunAfterTestClassCallbacks.evaluate(RunAfterTestClassCallbacks.java:70) at org.junit.runners.ParentRunner.run(ParentRunner.java:363) at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.run(SpringJUnit4ClassRunner.java:193)

I've found that running without the @Rule works fine! and with minimal setup.

Here is my config in spring boot's YAML format:

datasource: url: jdbc:tc:postgresql://localhost/myapp_test username: rob password: rob driverClassName: org.testcontainers.jdbc.ContainerDatabaseDriver

I've fiddled with ContainerDatabaseDriver and exposed the cache via a static method, and i've inspected it using @Before and @After inspection points: the cache only ever has 1 entry in it.

(Note when using @rule, because it is not setup using the Driver.getConnection(), these containers don't make it into the cache)

As part of the connection wrapper, it seems it cleans up the cache here, but i've yet to see it happening, the main container shutdown seems to occur due to the shutdown listener.

A note on HikariCP with Spring Boot

HikariCP has a setup which tests if a connection can be created in a constructor and throws an exception if it fails. This bumps into the testcontainer watching for the last connection to be closed in order to shut down the container.

new Connection().close() triggers a container shutdown, before anything has got going.

A setting - only available in spring boot 1.4.0 disables this check in Hikari.

spring.datasource.hikari.initialization-fail-fast=false

I'm sorry that's a bit rambling. I'm just trying to give a good picture of how testcontainers might play nice with Spring boot.

I'll try some more experiemntation later on.

Thanks for a great library by the way, i'm looking forward to using it extensively!

R

[robshep]

I've had some easier setup for a SpringBoot application by just creating the container manually, and binding it to a custom DataSource.

https://gist.github.com/robshep/34b13e69e3bcac4caad582d9bb7407f7

I'm confident that I can wangle it to use SpringBoot's DataSourceBuilder to make it a bit more seamless and then allow much more trivial setup for springboot testing.

[rnorth]

Hi Rob

Thanks for this analysis, and sorry for the slightly delayed response while
I'm travelling.

I'll have to look into this further in a couple of days, but it sounds like
the static cache approach is working well for you - as long as we don't
leak containers, some memory accumulation seems acceptable as you say.

Re use of @ Rule, as you've found you'd not want to use it in conjunction
with the tc Jdbc driver. The intention is that you should be able to use
one or the other depending on the situation.

Still, I should probably point out that for situations where it makes
sense, like perhaps Spring integration tests that can share a database, @
ClassRule is often a better option than @ Rule. ClassRule will create one
instance of the container per test class rather than per method, which
eliminates the problem you were seeing with unused containers being
spawned. If you're getting good results out of the patched jdbc driver
though, that sounds good.

Cheers

Rich
On Tue, 21 Jun 2016 at 06:54, robshep notifications@github.com wrote:

I've had some easier setup for a SpringBoot application by just creating
the container manually, and binding it to a custom DataSource.

https://gist.github.com/robshep/34b13e69e3bcac4caad582d9bb7407f7

I'm confident that I can wangle it to use SpringBoot's DataSourceBuilder
to make it a bit more seamless and then allow much more trivial setup for
springboot testing.

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
#159 (comment),
or mute the thread
https://github.com/notifications/unsubscribe/AAIETyg2-GvGE2pofTLFAu83G4blGUb6ks5qNwwUgaJpZM4I2WMv
.

Richard

[hopperd]

Any movement on this issue, it has become a blocker for our development and CI solutions (causing timeouts) as the workaround for setting the connection pool size doesn't play well with liquibase as mentioned in the above comments.

[rnorth]

@hopperd sorry for the long delay on this - recent Docker compatibility updates have been taking up a lot of time :(

I've just pushed into a branch with PR #195. This includes a simple fix, and improvements to the test suite to show the original problem.

I'd hope to release this change in the next few days (following PR review), but in the meantime you could use jitpack to obtain a build based on the fix-jdbc-driver-initialization branch.