[BUG] "Setup for flow is not up-to-date" caused by `setup_by_user`

[wxh06]

setup_by_user=True leads to Exception: Setup for flow `TextEmbedding` is not up-to-date. Please run `cocoindex setup` to update the setup.

To Reproduce

1. cd examples/text_embedding

2. cocoindex setup main (or update)

   

   Tables are successfully created.

3. Drop all tables except TextEmbedding__doc_embeddings, which is supposed to be "setup by user" in the following steps, from the database (i.e. cocoindex_setup_metadata and TextEmbedding__cocoindex_tracking).

4. Apply the patch below (add setup_by_user=True)

   diff --git a/examples/text_embedding/main.py b/examples/text_embedding/main.py
   index 2f1acf6..3d107bc 100644
   --- a/examples/text_embedding/main.py
   +++ b/examples/text_embedding/main.py
   @@ -71,6 +71,7 @@ def text_embedding_flow(
                    metric=cocoindex.VectorSimilarityMetric.COSINE_SIMILARITY,
                )
            ],
   +        setup_by_user=True,
        )

5. Run cocoindex setup main again.

   

   CocoIndex tries to recreate the tables that are removed by us previously, while TextEmbedding__doc_embeddings remains unchanged, which is expected, as it's still in the database and we've marked it setup_by_user.

6. Run cocoindex update main.

   

   • ❌ Changes need to be pushed. Continue? [yes/N]: is printed, which is unexpected since we have just run cocoindex setup main.
   • ❌ Exception: Setup for flow `TextEmbedding` is not up-to-date. Please run `cocoindex setup` to update the setup. occured.

Expected behavior

• Not being prompted to execute cocoindex setup again after it has already been executed.
• More detailed error messages indicating why database changes are required (whether the tables need to be updated manually or setup is indeed required).

CocoIndex Version

I've successfully reproduced in versions from 0.3.11 to 0.3.15

[georgeh0]

hi @wxh06, thanks for the detailed bug reporting. Fix #1369 is coming.

[badmonster0]

@wxh06 thanks for working with us and filing the issue! amazing @georgeh0 just fixed it