pdm.lock is missing groups dependencies (and pdm generate wrong requirements.txt)

[jeanmichel-nwsb]

Hi

I'm quite new to using PDM and I may have missed some important points.

I'm using PDM to manage a repo which is about an AWS stack created through CloudFormation and CDK
The repo includes mainly those folders/sections:

• stack/ which is the CDK base code to build the stack in CloudFormation
• sqs2db_lambda/ which is the 1st lambda
• metrics_aggregator/ is a 2nd lambda
• tests/sqs2db/ and tests/metrics_aggregator/
• .github/workflows/ folder for GitHub actions

I used pdm add [-dG groupname] module commands to add my dependcies which then appeared in the pyproject.toml file as below

[project]
name = "my-stack"
version = "0.1.0"
description = "My stack"
dependencies = ["aws-cdk-lib>=2.214.0", "aws-cdk-aws-lambda-python-alpha>=2.214.0a0"]
requires-python = ">=3.12"
readme = "README.md"

[tool.pdm]
distribution = false

[dependency-groups]
test = [
    "pytest>=8.4.1",
    "pytest-clarity>=1.0.1",
    "pytest-cov>=6.2.1",
    "pytest-mock>=3.15.0",
    "aiohttp>=3.12.15",
]
sqs2db = [
    "aws-lambda-powertools>=3.20.0",
    "git+ssh://[email protected]/private-account/private-module.git",
]
aggregator = [
    "aws-lambda-powertools>=3.20.0",
    "git+ssh://[email protected]/private-account/private-module.git",
    "slack-sdk>=3.36.0",
    "boto3>=1.40.34",
]

As you can see:

• the dependencies for building and deploying the stack are in the default dependencies group
• each lambda has its own dependency group
• both lambda depends on aws-lambda-powertools and the same private module, the aggregator has additional dependencies
• there's also a group for tests dependencies

For deploying the lambda code to AWS, I need a requirements.txt file in the folder of each lambda which I create dynamically in the GitHub Actions using:

      - name: create requirements
        run: |
          pdm export -G sqs2db -o sqs2db_lambda/requirements.txt --without-hashes --no-default
          pdm export -G aggregator -o metrics_aggregator/requirements.txt --without-hashes --no-default

The problem is that the dependency on the private module is missing from one of the 2 requirements.txt file. At the very beginning it was in the one for aggregator, but now it's in the one for 'sqs2db`

I have tracked down to the pdm.lock file where the groups entry is wrong for private-module but correct for aws-lambda-powertools

[[package]]
name = "aws-lambda-powertools"
version = "3.20.0"
requires_python = "<4.0.0,>=3.9"
summary = "Powertools for AWS Lambda (Python) is a developer toolkit to implement Serverless best practices and increase developer velocity."
groups = ["aggregator", "sqs2db"]
dependencies = [
    "jmespath<2.0.0,>=1.0.1",
    "typing-extensions<5.0.0,>=4.11.0",
]
files = [
    {file = "aws_lambda_powertools-3.20.0-py3-none-any.whl", hash = "sha256:1641eaf38e59470a28262e34d5d447bf6713223ea8896bf5ce442ffb3418c9c9"},
    {file = "aws_lambda_powertools-3.20.0.tar.gz", hash = "sha256:97cdaa52060972a59d0e769e197d6bffe5a1869c54b6a0c6c69bf31656de5bdd"},
]

[[package]]
name = "private-module"
version = "0.1.3"
requires_python = ">=3.10"
git = "ssh://[email protected]/private-account/private-module.git"
revision = "42ed774ccc9c13b736f71868f20ba888add16ab2"
summary = "A python library for clients processing workflows-metrics"
groups = ["aggregator"]
dependencies = [
    "boto3>=1.40.25",
    "pymongo>=4.14.1",
]

If I manually change the groups entry, the creation of requirements.txt works fine.
But anytime I run either pdm lock or pdm update, the groups entry for the the private module gets wrong again.
The private module is also managed using PDM.

Any idea if I'm doing something wrong ?

Thanks