Skip to content

feat: positional file arguments and multi-file joins#162

Merged
vmvarela merged 7 commits into
masterfrom
issue-155/positional-file-arguments
Jun 4, 2026
Merged

feat: positional file arguments and multi-file joins#162
vmvarela merged 7 commits into
masterfrom
issue-155/positional-file-arguments

Conversation

@vmvarela

@vmvarela vmvarela commented Jun 4, 2026

Copy link
Copy Markdown
Owner

Summary

Closes #155.

Accept files as positional arguments. Each file becomes a table named after its basename (sans extension). Stdin remains table t.

Examples

# Single file — no more cat
sql-pipe orders.csv 'SELECT * FROM orders WHERE amount > 100'

# Multi-file join
sql-pipe orders.csv customers.csv \
  'SELECT c.name, SUM(o.amount) FROM orders o JOIN customers c ON o.cust_id = c.id GROUP BY c.name'

# Stdin still works, still called t
cat data.csv | sql-pipe 'SELECT * FROM t'

# Mix stdin + files
cat events.csv | sql-pipe users.csv 'SELECT * FROM t JOIN users ON t.uid = users.id'

Acceptance Criteria

  • Files passed as positional arguments are loaded as tables
  • Table names are derived from filenames (basename sans extension)
  • Stdin input is still available as table t
  • Multi-file joins work correctly
  • Input format is auto-detected from file extension (.csv, .tsv, .json, .ndjson, .xml)
  • File-vs-query disambiguation works (heuristic: if arg is a readable file, treat as input; query is last non-file argument)
  • -- separator works for disambiguation
  • All existing tests pass
  • New tests cover single-file, multi-file, stdin+file, and edge cases (155a–155i)

Implementation Notes

  • format.zig: Added InputFormat.fromExtension() for auto-detection from file extension
  • sqlite.zig: Parameterized createTable/prepareInsertStmt/createAllTextTable with table_name
  • args.zig: Added FileInput struct, file-vs-query disambiguation, -- separator support
  • loader.zig: Generalized loadCsvInput to accept table_name and file_path
  • json.zig/xml.zig: Parameterized loaders with table_name
  • main.zig: Orchestrates multi-file loading with poll()-based stdin detection
  • Documentation updated in README.md and man page

Accept files as positional arguments. Each file becomes a table named
after its basename (sans extension). Stdin remains table t.

Examples:
  sql-pipe orders.csv 'SELECT * FROM orders WHERE amount > 100'
  sql-pipe orders.csv customers.csv 'SELECT ... FROM orders o JOIN customers c ...'
  cat events.csv | sql-pipe users.csv 'SELECT * FROM t JOIN users ON ...'

Changes:
- format.zig: Add InputFormat.fromExtension() for auto-detection
- sqlite.zig: Parameterize createTable/prepareInsertStmt with table name
- args.zig: Add FileInput struct, file-vs-query disambiguation, -- separator
- loader.zig: Accept table_name and file_path parameters
- json.zig/xml.zig: Parameterize loaders with table_name
- main.zig: Orchestrate multi-file loading with poll()-based stdin detection
- build.zig: Add integration tests 155a-155i
- README.md, docs/sql-pipe.1.scd: Document file arguments
@vmvarela vmvarela added type:feature New functionality priority:high Must be in the next sprint size:l Large — 1 to 2 days labels Jun 4, 2026
vmvarela added 6 commits June 4, 2026 10:47
Add sample fixture files (CSV, JSON, NDJSON, XML) in tests/fixtures/
and a new 'fixture-test' build step that exercises the binary end-to-end
with realistic data across all supported formats.

Tests cover:
- CSV file arguments (basic queries, type inference, date functions)
- Multi-file CSV joins
- JSON/NDJSON/XML file arguments with auto-detection
- Stdin + file argument mixing
- --columns, --validate, --sample with fixture data
- JSON output, --header, --output modes

Run with: zig build fixture-test
- Use std.Io.File.isTty() for cross-platform stdin detection
- Loaders return 0 on empty input to handle /dev/null gracefully
- main.zig checks if we have any data and fails if not
- Updated AGENTS.md with stdin detection pitfalls

Known issue: CI tests with file arguments fail when stdin is /dev/null.
Needs further work to distinguish 'no input' from 'header-only input'.
- Loaders now return 0 rows on empty input instead of calling fatal()
- Removed empty input check from main.zig
- Empty input results in 'no such table' error when querying, which is
  the expected SQLite behavior
- Updated tests 56 and 107 to expect 'no such table' error instead of
  'empty input' error
- Fixes CI failures where stdin is /dev/null (not a TTY but no data)
- All 194 integration tests and unit tests pass
…es, special modes, test coverage

Fix all blockers from PR #162 review:

- Remove isReadableFile() TOCTOU race (args.zig): let loaders handle open errors
- Add DuplicateTableName detection with clear error message (args.zig)
- Add empty input file check with explicit error (main.zig)
- Fix special modes (--columns, --validate, --sample) to accept file arguments
  instead of always reading stdin (modes/columns.zig, modes/validate.zig,
  modes/sample.zig)
- Unify file handling: loadCsvInput now takes a reader like all other loaders
  (loader.zig, main.zig)
- Use appendQuotedId consistently in all table_name quoting (sqlite.zig)
- Remove dead delimiter field from FileInput struct (args.zig)
- Update outdated comments referencing hardcoded table t (main.zig, loader.zig)
- Fix dead fixture test 13 (referenced non-existent products.csv)
- Fix fixture numbering duplication
- Add fixture tests 21-23 for special modes with file arguments
- Update existing integration tests for new error messages (build.zig)
@vmvarela vmvarela merged commit a67bf1c into master Jun 4, 2026
4 checks passed
@vmvarela vmvarela deleted the issue-155/positional-file-arguments branch June 4, 2026 09:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

priority:high Must be in the next sprint size:l Large — 1 to 2 days type:feature New functionality

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Positional file arguments and multi-file joins

1 participant