Skip to content

brandonzylstra/essence

BranchesTags

Repository files navigation

🌿 Essence 🫧 - ActiveRecord Modeling

CI

Essence is a powerful tool for rapid database schema iteration in Rails applications. It provides a clean, YAML-based syntax with intelligent defaults and pattern matching that compiles to Atlas HCL format for seamless database migrations.

🚀 Why Essence?

  • 10x faster schema iteration - Write schemas in clean YAML instead of verbose Rails migrations or GUI tools
  • Flexible data type syntax - Use familiar Rails migration types (:string, :integer) or concise Atlas/SQL types (string(100), integer)
  • Smart defaults - Automatic id, created_at, updated_at columns for every table
  • Pattern-based property inference - league_id: ~ automatically becomes a foreign key to leagues.id
  • Version control friendly - Text-based schemas that diff and merge cleanly
  • Rails integration - Seamless workflow with rake tasks and Atlas backend
  • Template generation - Quick project setup with sensible defaults

🔌 Installation

Note: This gem is not yet published to RubyGems. Install from source:

From GitHub (recommended)

Add this line to your application's Gemfile:

gem 'essence', git: 'https://github.com/brandonzylstra/essence.git'

And then execute:

$ bundle install

Build and install locally

Clone the repository and build the gem:

$ git clone https://github.com/brandonzylstra/essence.git
$ cd essence
$ bundle install
$ gem build essence.gemspec
$ gem install essence-0.1.0.gem

🚀 Quick Start

1. Generate a schema template

# In your Rails project
rake essence:template

This creates db/schema.yaml with sensible defaults:

# Enhanced Essence Schema with Default Columns and Pattern Matching
schema_name: public

defaults:
  "*":
    columns:
      id: ~
      created_at: ~
      updated_at: ~

column_patterns:
  - "^id$": "primary_key"
  - "_id$": "integer -> {table}.id on_delete=cascade not_null"
  - "_at$": "datetime not_null"
  - ".*": "string"

tables:
  users:
    columns:
      email: string(255) not_null unique
      name: string(100) not_null
      league_id: ~
      last_login_at: ~

  leagues:
    columns:
      name: string(255) not_null unique
      description: text

2. Compile to HCL format

rake essence:compile

This generates db/schema.hcl in Atlas format with foreign keys automatically resolved:

schema "public" {
  comment = "Generated by Essence"
}

table "users" {
  schema = schema.public
  column "id" {
    null = false
    type = integer
    auto_increment = true
  }
  column "email" {
    null = false
    type = varchar(255)
  }
  column "name" {
    null = false
    type = varchar(100)
  }
  column "league_id" {
    null = false
    type = integer
  }
  column "last_login_at" {
    null = false
    type = datetime
  }
  column "created_at" {
    null = false
    type = datetime
  }
  column "updated_at" {
    null = false
    type = datetime
  }
  primary_key {
    columns = [column.id]
  }
  foreign_key "FK_USERS_LEAGUE" {
    columns     = [column.league_id]
    ref_columns = [table.leagues.column.id]
    on_delete   = CASCADE
  }
  index "IDX_USERS_EMAIL" {
    columns = [column.email]
    unique  = true
  }
}

3. Preview and apply changes

# See what would change
rake essence:preview

# Generate Rails migration and apply
rake essence:deploy["add tournament tables"]

🎯 Core Features

Smart Defaults

Every table automatically gets:

  • id primary key column
  • created_at and updated_at timestamps
  • Proper indexing and constraints

Pattern-Based Intelligence

Write league_id: ~ and—based on the pattern that name matches—automatically get:

  • Integer column
  • Foreign key to leagues.id
  • Cascading delete
  • Proper indexing

Flexible Data Type Syntax

Choose the syntax that works best for your team:

# Atlas/SQL syntax
tables:
  tournaments:
    columns:
      name: string(255) not_null unique
      league_id: ~   # Becomes foreign key automatically
      start_date: ~  # Becomes datetime not_null

# Rails syntax
  users:
    columns:
      name: :string, limit: 255, null: false, unique: true
      league_id: ~   # Pattern matching works with both syntaxes
      created_at: :datetime, null: false

# Mixed syntax
  posts:
    columns:
      title: string(255) not_null           # Atlas/SQL syntax
      content: :text                        # Rails syntax
      user_id: ~                            # Pattern matching

Full Rails Integration

rake essence:template     # Generate schema template
rake essence:convert      # Convert YAML to HCL
rake essence:preview      # Preview changes
rake essence:apply        # Apply to database
rake essence:deploy[name] # Full workflow

🗃️ Supported Schema Elements

Essence provides comprehensive support for database schema elements, with plans to expand coverage over time.

Status Group Element Explanation
✅ Core Tables Primary schema containers with columns, constraints, and metadata
✅ Core Columns All standard data types with dual syntax support (Atlas/SQL + Rails)
✅ Core Data Types Support for both :string (Rails) and string(100) (Atlas/SQL) syntax
✅ Core Primary Keys Auto-incrementing ID columns with proper constraints
✅ Core Foreign Keys Automatic relationship detection with cascade/set null options
✅ Core Indexes Single and multi-column indexes with unique constraints
⏳ Core Check Constraints Column-level validation rules (planned for Rails 6.1+ support)
⏳ Advanced Views Read-only virtual tables (scenic gem integration)
⏳ Advanced Materialized Views Cached query results (scenic gem integration)
⏳ Advanced Functions/Stored Procedures Database-level business logic (scenic, fx gems)
❓ Advanced Exclusion Constraints PostgreSQL-specific constraints
❓ Advanced Enums Type-safe enumerated values (Rails native with PostgreSQL)
❓ Advanced Partitioned Tables Large table performance optimization (Rails 6.1+ with PostgreSQL)
❓ Advanced Triggers Event-driven database actions (hair_trigger gem integration)
❓ Advanced Full-text Search Indexes Advanced search capabilities (pg_search gem)

Version Compatibility

ActiveRecord Schema Versions:

  • ✅ 8.0 - Fully supported (current)
  • 🚧 7.2 - Possible support
  • 🚧 7.1 - Possible support
  • 🚧 7.0 - Possible support

Priority is given to supporting newer Rails versions as they are released (8.1, 8.2, etc.) before adding backward compatibility.

📋 Available Patterns

Essence automatically recognizes these column patterns (works with both Atlas/SQL and Rails syntax):

Pattern Example Result
*_id user_id: ~ integer -> users.id on_delete=cascade not_null
*_at published_at: ~ datetime not_null
*_on published_on: ~ date not_null
*_date birth_date: ~ date not_null
is_* is_active: ~ boolean default=false not_null
has_* has_premium: ~ boolean default=false not_null
can_* can_edit: ~ boolean default=false not_null
*_flag admin_flag: ~ boolean default=false not_null
*_content body_content: ~ text
*_body email_body: ~ text
*_text description_text: ~ text
*_html content_html: ~ text
*_count view_count: ~ integer default=0 not_null
*_score rating_score: ~ decimal(8,2)
*_amount price_amount: ~ decimal(10,2)
*_price unit_price: ~ decimal(10,2)
*_email contact_email: ~ string(255)
*_url website_url: ~ string(500)
*_code product_code: ~ string(50)
*_slug permalink_slug: ~ string(255) unique
*_status order_status: ~ string(50)
*_state workflow_state: ~ string(50)

🛠️ Advanced Usage

Custom Patterns

Define your own patterns in the schema:

column_patterns:
  - "_uuid$": "uuid not_null unique"
  - "_json$": "json"
  - "encrypted_": "text not_null"

Table-Specific Defaults

Override defaults for specific tables:

defaults:
  "*":
    columns:
      id: ~
      created_at: ~
      updated_at: ~
  "audit_*":
    columns:
      id: uuid primary_key
      created_at: ~
      created_by: string(255) not_null

Explicit Overrides

Override pattern matching with explicit definitions:

tables:
  users:
    columns:
      league_id: string(50) not_null  # Override foreign key pattern
      created_at: timestamp not_null  # Override default

🔄 Workflow Integration

Complete Development Workflow

# 1. Set up (first time)
rake essence:init           # Import existing schema
# OR
rake essence:template       # Start fresh

# 2. Edit schema
vim db/schema.yaml

# 3. Iterate rapidly
rake essence:compile        # YAML → HCL
rake essence:preview        # See changes
rake essence:generate[name] # Create migration
rake essence:apply          # Apply to DB

# 4. Deploy
rake essence:deploy[name]   # All-in-one

CI/CD Integration

# .github/workflows/schema.yml
name: Schema Validation
on: [push, pull_request]
jobs:
  validate:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      - name: Setup Ruby
        uses: ruby/setup-ruby@v1
        with:
          bundler-cache: true
      - name: Validate schema
        run: rake essence:validate

📚 Command Reference

CLI Commands

essence template [path]         # Generate template
essence compile [yaml] [hcl]    # Compile formats
essence version                 # Show version
essence help                    # Show help

Rake Tasks

rake essence:template[path]     # Generate schema template
rake essence:compile[yaml,hcl]  # Compile YAML to HCL
rake essence:preview            # Preview changes
rake essence:generate[name]     # Generate Rails migration
rake essence:apply              # Apply schema to database
rake essence:deploy[name]       # Full workflow
rake essence:seed               # Generate seed data
rake essence:init               # Initialize from existing
rake essence:validate           # Validate schema
rake essence:history            # Show migration history
rake essence:help               # Show all commands

🏗️ Architecture

Essence works by:

  1. YAML Definition - You write clean, pattern-rich schemas
  2. Pattern Resolution - Automatic foreign keys, constraints, indexes
  3. HCL Generation - Compiles to Atlas-compatible HCL format
  4. Atlas Integration - Uses Atlas for migration planning
  5. Rails Compatibility - Generates standard Rails migrations
schema.yaml → [Essence] → schema.hcl → [Atlas] → migration.rb → [Rails] → Database

💻 Examples

Basic Blog Schema

schema_name: public
tables:
  users:  # Atlas/SQL syntax
    columns:
      email: string(255) not_null unique
      username: string(50) not_null unique
      first_name: string(100)
      last_name: string(100)
      is_admin: ~
      last_login_at: ~

  posts:  # Rails syntax
    columns:
      title: :string, limit: 255, null: false
      post_slug: ~                    # Pattern matching works with both
      content_html: :text
      user_id: ~                      # Foreign key via pattern
      published_at: :datetime
      is_published: :boolean, default: false
      view_count: :integer, default: 0

comments: columns: post_id: ~ user_id: ~ comment_body: ~ is_approved: ~


### E-commerce Schema ###
```yaml
schema_name: public
tables:
  customers:
    columns:
      email: string(255) not_null unique
      first_name: string(100) not_null
      last_name: string(100) not_null
      phone: string(20)
      birth_date: ~
      is_active: ~

  products:
    columns:
      name: string(255) not_null
      product_code: ~
      description_text: ~
      unit_price: ~
      cost_amount: ~
      stock_count: ~
      is_active: ~

  orders:
    columns:
      customer_id: ~
      order_status: ~
      total_amount: ~
      tax_amount: ~
      shipping_amount: ~
      order_date: ~
      shipped_at: ~

💡 Tips and Best Practices

1. Always Preview First

rake essence:preview  # See what will change
rake essence:apply    # Apply changes

2. Use Descriptive Names

# Good - descriptive and clear
users:
  columns:
    email: string(255) not_null unique
    contact_email: string(255)

# Avoid - too generic
users:
  columns:
    data1: string
    field2: text

3. Be Explicit About Constraints

# Good - explicit constraints
email: string(255) not_null unique
user_id: integer -> users.id on_delete=cascade not_null

# Avoid - relying only on patterns
email: ~
user_id: ~

4. Group Related Tables

Organize your YAML file with related tables together for better readability:

# Authentication & Users
users:
  # ...
user_sessions:
  # ...

# Content Management
posts:
  # ...
comments:
  # ...

5. Use Pattern Matching Effectively

# Let patterns handle repetitive definitions
published_at: ~     # Becomes: datetime not_null
is_active: ~        # Becomes: boolean default=false not_null
view_count: ~       # Becomes: integer default=0 not_null

# Override patterns when needed
special_id: bigint -> special_table.id on_delete=set_null

🤝 Contributing

  1. Fork it (https://github.com/brandonzylstra/essence/fork)
  2. Create your feature branch (git checkout -b my-new-feature)
  3. Commit your changes (git commit -am 'Add some feature')
  4. Push to the branch (git push origin my-new-feature)
  5. Create a new Pull Request

📄 License

The gem is available as open source under the terms of the MIT License.

🙏 Acknowledgments

  • Atlas for the powerful migration engine
  • Rails for the amazing framework
  • The Ruby community for inspiration and support

About

🧘🏼‍♂️ Relaxed Rails Modeling & Migrations

Topics

ruby rails yaml data gem database modeling hcl ruby-on-rails active-record

Resources

Readme
Activity

Stars

0 stars

Watchers

0 watching

Forks

0 forks
Report repository

Languages