refactor(devshell): improve devshell experience #1977
+1,437
−1,590
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
These overlay packages will be removed once the extensions are updated to use the new `buildPgrxExtension` function.
The previously used version was failing to build:
/build/source/src/common/get_check_data.c: In function 'pgr_SPI_getText':
/build/source/src/common/get_check_data.c:307:28: error: passing argument 1 of 'DatumGetCString' makes integer from pointer without a cast [-Wint-conversion]
307 | return DatumGetCString(SPI_getvalue(*tuple, *tupdesc, info.colNumber));
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
| |
| char *
In file included from /build/source/include/c_common/postgres_connection.h:29,
from /build/source/src/common/get_check_data.c:27:
/nix/store/nn6vpjqlsxxmynwcrsib9agb3hpr5cqj-postgresql-17.4/include/server/postgres.h:335:23: note: expected 'Datum' {aka 'long unsigned int'} but argument is of type 'char *'
335 | DatumGetCString(Datum X)
| ~~~~~~^
make[2]: *** [src/common/CMakeFiles/common.dir/build.make:149: src/common/CMakeFiles/common.dir/get_check_data.c.o] Error 1
This was discussed upstream here: https://postgr.es/m/0a997455-5aba-4cf2-a354-d26d8bcbfae6%40technowledgy.de Also applied by nixpkgs here: NixOS/nixpkgs#342026
We cannot use nodejs.libv8 as it is too new for plv8 3.1. We will be able to use it with plv8 3.2.
dbmate 2.27 is now using a varchar for the version column in the schema_migrations table. See amacneil/dbmate#641
Updated nixpkgs, rust-overlay, and nix-fast-build to latest versions.
Removed duplicate substituteInPlace calls that were redundantly applied during previous merge operations. Cleaned up the patch application to avoid duplicate substitutions.
Updated postgis.nix to use nixpkgs sfcgal instead of our custom package, eliminating the need to maintain a separate sfcgal build definition.
pg_net requires an older version of curl (pinned from nixpkgs commit a76c4553d7e741e17f289224eda135423de0491d) to resolve build compatibility issues.
Updated cargo-pgrx buildPgrxExtension with rustc wrapper to handle empty postmaster_stub.rs files that cause duplicate symbol errors. (first two filenames are `` and `/build/tmp.*/postmaster_stubs/nix/store/*/bin/postmaster_stub.rs`) pg_jsonschema cargo checks fails on specific versions (0.2.0, 0.3.1, 0.3.3).
Remove deprecated functions that were removed in newer pgrouting versions (_pgr_betweennesscentrality and _pgr_contractionhierarchies).
Improves reproducibility and avoids evaluation-time network access by using a proper flake input instead of fetchTarball for the older nixpkgs needed by pg_net and plv8.
…64-darwin Version 0.19.5 has a dangling pointer issue in src/core.c:177 that causes compilation to fail on aarch64-darwin with newer clang versions. This adds -Wno-error=dangling-assignment to allow compilation to proceed. The flag is restricted to aarch64-darwin only since GCC on Linux doesn't recognize this warning option.
Add -headerpad_max_install_names linker flag to prevent 'install_name_tool: changing install names or rpaths can't be redone' errors on macOS. This allocates sufficient header space for install_name_tool to modify the shared library during post-install processing.
… extensions
Test assertions were inconsistent: expecting {version}.so and {extension_name}-{version}.so
patterns while actual libraries use {lib_name}-{version}.so (e.g. pg_partman_bgw-5.3.1.so,
timescaledb-loader-2.16.1.so). Updated PostgresExtensionTest to accept Optional[str] lib_name
parameter and use consistent {lib_name}-{version}.so assertions throughout.
Added missing timescaledb-loader.so symlink creation and updated the switch script to
maintain both symlinks when switching versions.
Fixes 'Expected timescaledb version 2.16.1, but found timescaledb-loader-2.16.1.so' and
'Expected pg_partman version 5.3.1, but found pg_partman_bgw-5.3.1.so' test failures.
Use overlay instead of passing full nixpkgs-oldstable to avoid exposing entire old nixpkgs attribute set. Now provides specific packages (curl_8_4, v8_oldstable) through overlay.
Keep revision support that was accidentally simplified during nixpkgs update. Restores conditional logic to support building orioledb from specific git revision hashes instead of only tags.
These files were moved to subdirectories and the old .nix files should be removed since they were moved to new directory structure with default.nix files.
Remove nixpkgs-go124 dependency since current nixpkgs has Go 1.25.4 which is sufficient for packer. Remove nix-fast-build input as unused and add follows directives to resolve duplicate dependencies.
Run flake-linter to eliminate duplicate dependencies, and remove unused cargo-pgrx / rustc versions hashes.
Move pgrouting.nix to pgrouting/default.nix and groonga package to pgroonga/groonga.nix. Update imports and fix relative paths for versions.json and patch files.
Replace inline bash script generation with writeShellScriptBin for rustc wrapper that filters empty postmaster_stub.rs arguments. Apply wrapper only for pgrx < 0.12 since issue was fixed upstream in pgrx#1435 and pgrx#1441.
Move nixpkgs-oldstable import to a let binding to avoid importing it three times...
Remove supabase-groonga from global flake packages and import it locally in pgroonga extension instead. Other components access groonga via pgroonga.passthru.groonga since groonga is only used by pgroonga.
The current development shell include a large number of packages which leads to a very large closure size (5 Gb) and slow startup times. To improve the situation we defer installing large packages to devshell commands which only get installed when needed. We introduce the use of devshell to define the commands in a declarative way as well as to improve the overall developer experience. Included changes: - Add organized command menu with categories (check, ami, extension, postgres) - Add watch command using watchexec + nix-fast-build for continuous checking - Expose common tasks as named commands: fmt, check, lint, watch - Add aws-vault to development tools
`writeShellApplication` avoids substitute commands, provides the correct package metadata and make sure that all scripts are shellcheked.
We don't need to maintain our own packer package anymore since we are using a recently updated nixpkgs that includes packer.
31a6d6b to
0649fff
Compare
b48e4c3 to
3df6c4a
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The current development shell include a large number of packages which
leads to a very large closure size (5 Gb) and slow startup times.
To improve the situation we defer installing large packages to devshell
commands which only get installed when needed.
We introduce the use of devshell to define the commands in a
declarative way as well as to improve the overall developer experience.
The closure of the default devshell is now down to 2 Gb (which still could be improved later on).
Included changes: