feat: positional file arguments and multi-file joins#162
Merged
Conversation
Accept files as positional arguments. Each file becomes a table named after its basename (sans extension). Stdin remains table t. Examples: sql-pipe orders.csv 'SELECT * FROM orders WHERE amount > 100' sql-pipe orders.csv customers.csv 'SELECT ... FROM orders o JOIN customers c ...' cat events.csv | sql-pipe users.csv 'SELECT * FROM t JOIN users ON ...' Changes: - format.zig: Add InputFormat.fromExtension() for auto-detection - sqlite.zig: Parameterize createTable/prepareInsertStmt with table name - args.zig: Add FileInput struct, file-vs-query disambiguation, -- separator - loader.zig: Accept table_name and file_path parameters - json.zig/xml.zig: Parameterize loaders with table_name - main.zig: Orchestrate multi-file loading with poll()-based stdin detection - build.zig: Add integration tests 155a-155i - README.md, docs/sql-pipe.1.scd: Document file arguments
9 tasks
Add sample fixture files (CSV, JSON, NDJSON, XML) in tests/fixtures/ and a new 'fixture-test' build step that exercises the binary end-to-end with realistic data across all supported formats. Tests cover: - CSV file arguments (basic queries, type inference, date functions) - Multi-file CSV joins - JSON/NDJSON/XML file arguments with auto-detection - Stdin + file argument mixing - --columns, --validate, --sample with fixture data - JSON output, --header, --output modes Run with: zig build fixture-test
- Use std.Io.File.isTty() for cross-platform stdin detection - Loaders return 0 on empty input to handle /dev/null gracefully - main.zig checks if we have any data and fails if not - Updated AGENTS.md with stdin detection pitfalls Known issue: CI tests with file arguments fail when stdin is /dev/null. Needs further work to distinguish 'no input' from 'header-only input'.
- Loaders now return 0 rows on empty input instead of calling fatal() - Removed empty input check from main.zig - Empty input results in 'no such table' error when querying, which is the expected SQLite behavior - Updated tests 56 and 107 to expect 'no such table' error instead of 'empty input' error - Fixes CI failures where stdin is /dev/null (not a TTY but no data) - All 194 integration tests and unit tests pass
…es, special modes, test coverage Fix all blockers from PR #162 review: - Remove isReadableFile() TOCTOU race (args.zig): let loaders handle open errors - Add DuplicateTableName detection with clear error message (args.zig) - Add empty input file check with explicit error (main.zig) - Fix special modes (--columns, --validate, --sample) to accept file arguments instead of always reading stdin (modes/columns.zig, modes/validate.zig, modes/sample.zig) - Unify file handling: loadCsvInput now takes a reader like all other loaders (loader.zig, main.zig) - Use appendQuotedId consistently in all table_name quoting (sqlite.zig) - Remove dead delimiter field from FileInput struct (args.zig) - Update outdated comments referencing hardcoded table t (main.zig, loader.zig) - Fix dead fixture test 13 (referenced non-existent products.csv) - Fix fixture numbering duplication - Add fixture tests 21-23 for special modes with file arguments - Update existing integration tests for new error messages (build.zig)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Closes #155.
Accept files as positional arguments. Each file becomes a table named after its basename (sans extension). Stdin remains table
t.Examples
Acceptance Criteria
t--separator works for disambiguationImplementation Notes
format.zig: AddedInputFormat.fromExtension()for auto-detection from file extensionsqlite.zig: ParameterizedcreateTable/prepareInsertStmt/createAllTextTablewithtable_nameargs.zig: AddedFileInputstruct, file-vs-query disambiguation,--separator supportloader.zig: GeneralizedloadCsvInputto accepttable_nameandfile_pathjson.zig/xml.zig: Parameterized loaders withtable_namemain.zig: Orchestrates multi-file loading withpoll()-based stdin detection