Fix Citation published field type mismatch between Java boolean and PostgreSQL smallint

[Copilot]

• Investigate the issue thoroughly (JSP, Controller, Entity, Schema)
• Identify root cause: Schema mismatch between test (published boolean) and production (published smallint)
• Revert the incorrect @Type(NumericBooleanType) annotation from Citation.java
• Update TBASE2_POSTGRES_CREATION.sql to use boolean instead of smallint for published columns (citation, matrix, phylotree tables)
• Code review passed
• Security scan passed (no issues)

Root Cause Analysis

The original error occurred because:

1. The Java entity Citation.java has a boolean mPublished field
2. The test schema (0000_SCHEMA_before_patches_start.sql) uses published boolean
3. The production script (TBASE2_POSTGRES_CREATION.sql) incorrectly used published smallint
4. My previous fix with @Type(NumericBooleanType) converted boolean to integer, which broke CI tests

Solution

• Removed the @Type(NumericBooleanType) annotation from Citation.isPublished()
• Updated TBASE2_POSTGRES_CREATION.sql to use boolean type for published columns in citation, matrix, and phylotree tables, aligning with the test schema

Notes

• The citationStatusDescription field from the form correctly maps to the CitationStatus entity (with descriptive values like "In Prep", "In Review", "Published")
• The mPublished boolean field in Citation is a separate field used for internal tracking

Original prompt

Problem updating the citation at http://localhost:8080/treebase-web/user/citationForm.html:

treebase-postgres  | 2026-01-13 14:14:45.775 UTC [58] ERROR:  column "published" is of type smallint but expression is of type boolean at character 212
treebase-postgres  | 2026-01-13 14:14:45.775 UTC [58] HINT:  You will need to rewrite or cast the expression.
treebase-postgres  | 2026-01-13 14:14:45.775 UTC [58] STATEMENT:  insert into CITATION (VERSION, PMID, URL, ABSTRACT, CITATIONSTATUS_ID, Doi, Keywords, Pages, PublishYear, Published, Title, Issue, Journal, Volume, TYPE, CITATION_ID) values ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11, $12, $13, $14, 'A', $15)
treebase-web-dev   | 14:14:45.780 [http-nio-8080-exec-9] ERROR org.hibernate.engine.jdbc.batch.internal.BatchingBatch - HHH000315: Exception executing batch [java.sql.BatchUpdateException: Batch entry 0 insert into CITATION (VERSION, PMID, URL, ABSTRACT, CITATIONSTATUS_ID, Doi, Keywords, Pages, PublishYear, Published, Title, Issue, Journal, Volume, TYPE, CITATION_ID) values (('0'::int4), (''), ('http://'), ('baz'), ('1'::int8), (''), ('var'), (''), ('2026'::int4), ('FALSE'::boolean), ('foo'), (''), (''), (''), 'A', ('1'::int8)) was aborted: ERROR: column "published" is of type smallint but expression is of type boolean
treebase-web-dev   |   Hint: You will need to rewrite or cast the expression.
treebase-web-dev   |   Position: 212  Call getNextException to see other errors in the batch.], SQL: insert into CITATION (VERSION, PMID, URL, ABSTRACT, CITATIONSTATUS_ID, Doi, Keywords, Pages, PublishYear, Published, Title, Issue, Journal, Volume, TYPE, CITATION_ID) values (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, 'A', ?)
treebase-web-dev   | 14:14:45.780 [http-nio-8080-exec-9] ERROR org.hibernate.engine.jdbc.spi.SqlExceptionHelper - Batch entry 0 insert into CITATION (VERSION, PMID, URL, ABSTRACT, CITATIONSTATUS_ID, Doi, Keywords, Pages, PublishYear, Published, Title, Issue, Journal, Volume, TYPE, CITATION_ID) values (('0'::int4), (''), ('http://'), ('baz'), ('1'::int8), (''), ('var'), (''), ('2026'::int4), ('FALSE'::boolean), ('foo'), (''), (''), (''), 'A', ('1'::int8)) was aborted: ERROR: column "published" is of type smallint but expression is of type boolean
treebase-web-dev   |   Hint: You will need to rewrite or cast the expression.
treebase-web-dev   |   Position: 212  Call getNextException to see other errors in the batch.
treebase-web-dev   | 14:14:45.780 [http-nio-8080-exec-9] ERROR org.hibernate.engine.jdbc.spi.SqlExceptionHelper - ERROR: column "published" is of type smallint but expression is of type boolean

---

💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.

[rvosa]

@copilot this is not going well. I think something is wrong about the logic, where the input form (citationForm.jsp) has a number of different publication statuses, the controller seems to treat it as a boolean, but the database wants a smallint. Investigate very carefully what is going on here. When I tried your latest commit, the following happened:

Yikes!
java.lang.NumberFormatException: For input string: "class"
	at java.base/java.lang.NumberFormatException.forInputString(NumberFormatException.java:67)
	at java.base/java.lang.Integer.parseInt(Integer.java:668)
	at java.base/java.lang.Integer.parseInt(Integer.java:786)
	at javax.el.ListELResolver.coerce(ListELResolver.java:144)
	at javax.el.ListELResolver.getValue(ListELResolver.java:64)
	at org.apache.jasper.el.JasperELResolver.getValue(JasperELResolver.java:123)
	at org.apache.el.parser.AstValue.getValue(AstValue.java:160)
	at org.apache.el.parser.AstEqual.getValue(AstEqual.java:33)
	at org.apache.el.ValueExpressionImpl.getValue(ValueExpressionImpl.java:149)
	at org.apache.jasper.runtime.PageContextImpl.proprietaryEvaluate(PageContextImpl.java:655)
	at org.apache.jsp.decorators.defaultTemplate_jsp._jspx_meth_c_005fwhen_005f0(defaultTemplate_jsp.java:1227)
	at org.apache.jsp.decorators.defaultTemplate_jsp._jspx_meth_c_005fchoose_005f0(defaultTemplate_jsp.java:1198)
	at org.apache.jsp.decorators.defaultTemplate_jsp._jspx_meth_c_005fif_005f1(defaultTemplate_jsp.java:1165)
	at org.apache.jsp.decorators.defaultTemplate_jsp._jspService(defaultTemplate_jsp.java:439)
	at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:62)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:623)
	at org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:438)
	at org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:349)
	at org.apache.jasper.servlet.JspServlet.service(JspServlet.java:300)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:623)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:197)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:142)
	at org.apache.catalina.core.ApplicationDispatcher.invoke(ApplicationDispatcher.java:619)
	at org.apache.catalina.core.ApplicationDispatcher.doInclude(ApplicationDispatcher.java:496)
	at org.apache.catalina.core.ApplicationDispatcher.include(ApplicationDispatcher.java:444)
	at com.opensymphony.module.sitemesh.filter.PageFilter.writeDecorator(PageFilter.java:173)
	at com.opensymphony.module.sitemesh.filter.PageFilter.applyDecorator(PageFilter.java:158)
	at com.opensymphony.module.sitemesh.filter.PageFilter.doFilter(PageFilter.java:62)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:166)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:142)
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:352)
	at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:117)
	at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:83)
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:361)
	at org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:164)
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:361)
	at org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:126)
	at org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:120)
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:361)
	at org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:131)
	at org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:85)
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:361)
	at org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:100)
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:361)
	at org.springframework.security.web.savedrequest.RequestCacheAwareFilter.doFilter(RequestCacheAwareFilter.java:63)
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:361)
	at org.springframework.security.web.authentication.AbstractAuthenticationProcessingFilter.doFilter(AbstractAuthenticationProcessingFilter.java:227)
	at org.springframework.security.web.authentication.AbstractAuthenticationProcessingFilter.doFilter(AbstractAuthenticationProcessingFilter.java:221)
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:361)
	at org.springframework.security.web.authentication.logout.LogoutFilter.doFilter(LogoutFilter.java:107)
	at org.springframework.security.web.authentication.logout.LogoutFilter.doFilter(LogoutFilter.java:93)
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:361)
	at org.springframework.security.web.header.HeaderWriterFilter.doHeadersAfter(HeaderWriterFilter.java:90)
	at org.springframework.security.web.header.HeaderWriterFilter.doFilterInternal(HeaderWriterFilter.java:75)
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117)
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:361)
	at org.springframework.security.web.context.request.async.WebAsyncManagerIntegrationFilter.doFilterInternal(WebAsyncManagerIntegrationFilter.java:62)
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117)
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:361)
	at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:117)
	at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:87)
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:361)
	at org.springframework.security.web.session.DisableEncodeUrlFilter.doFilterInternal(DisableEncodeUrlFilter.java:42)
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117)
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:361)
	at org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:225)
	at org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:190)
	at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:354)
	at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:267)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:166)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:142)
	at net.bull.javamelody.MonitoringFilter.doFilter(MonitoringFilter.java:199)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:166)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:142)
	at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:166)
	at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:88)
	at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:481)
	at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
	at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:83)
	at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:643)
	at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:72)
	at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:344)
	at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:398)
	at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:63)
	at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:939)
	at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1831)
	at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:52)
	at org.apache.tomcat.util.threads.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:973)
	at org.apache.tomcat.util.threads.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:491)
	at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:63)
	at java.base/java.lang.Thread.run(Thread.java:840)
	```
	
	In addition, the CI/CD now failed:
	
	- https://github.com/TreeBASE/treebase/actions/runs/20960437125/job/60235880555?pr=395
	- https://github.com/TreeBASE/treebase/pull/395/checks?check_run_id=60236061621

[rvosa]

@copilot try again. I think you were getting stuck in a loop. My hypothesis is that the web page has multiple publication statuses (e.g. 'Published', 'In review'), which are mapped to small ints in the database, but the controller for some reason still thinks we're just dealing with a boolean. I'm not sure about this but it would be a thing to explore. However, be openminded and investigate other possibilities as well. Look at the JSP, look at the Controller, and look at the database.

[Copilot]

@copilot try again. I think you were getting stuck in a loop. My hypothesis is that the web page has multiple publication statuses (e.g. 'Published', 'In review'), which are mapped to small ints in th...

I investigated the issue and found the root cause. You were right - there was a mismatch, but it was in the database schema definition rather than the controller logic.

Root cause: The production schema (TBASE2_POSTGRES_CREATION.sql) defined published as smallint, while the test schema and Java entity both use boolean. My previous fix with NumericBooleanType was incorrect because it converted boolean to integer (0/1), which worked for smallint but broke the CI tests using the boolean schema.

Fix in commit 9b59d02:

1. Removed the incorrect @Type(NumericBooleanType) annotation from Citation.isPublished()
2. Updated TBASE2_POSTGRES_CREATION.sql to use boolean instead of smallint for the published columns in citation, matrix, and phylotree tables

Note: The citationStatusDescription from the form correctly maps to the CitationStatus entity (with values like "In Prep", "In Review", "Published"), which is separate from the mPublished boolean field.