Skip to content

streamline-test-builds.md

Latest commit

 

History

History
1015 lines (792 loc) · 34.3 KB

streamline-test-builds.md

File metadata and controls

1015 lines (792 loc) · 34.3 KB

Streamlining SWFRecomp Test Build Process

Executive Summary

Currently, the SWFRecomp test build process requires duplicating runtime files, build scripts, and headers across each test directory. This document outlines the current issues and proposes solutions to minimize duplication and streamline the process of building and deploying WASM demos.

Key Finding: The SWFModernRuntime repository contains a comprehensive, production-quality runtime library with 1,000+ lines of ActionScript implementation, variable storage, and memory management. This library can potentially replace the simple, duplicated runtimes currently used in test directories.

Current Build Process Issues

Duplication Problems

  1. Runtime files duplicated per test - Each test that has WASM support has its own runtime/ directory with copies of similar files

    • SWFRecomp/tests/trace_swf_4/runtime/
    • SWFRecomp/tests/dyna_string_vars_swf_4/runtime/

  2. Build scripts duplicated - Each test needs its own Makefile and build_wasm.sh with nearly identical content

    • Build scripts are ~70 lines each, mostly identical
    • Any improvements require updating all copies

  3. Header files duplicated - recomp.h, stackvalue.h are copied in each test's runtime directory

    • Makes it difficult to maintain consistency
    • Bug fixes require updates in multiple locations

  4. Inconsistent runtime implementations

    • Some tests (like dyna_string_vars_swf_4) have more advanced runtimes (305 lines) with variable storage
    • Simpler tests (like trace_swf_4) have minimal runtimes (76 lines)
    • Different levels of ActionScript support across tests

  5. Most tests don't have WASM build support yet

    • Only 2 out of ~50 tests have full runtime/wasm/ directories
    • Setting up WASM support for a new test requires creating ~5-7 files

Current Test Structure

SWFRecomp/tests/trace_swf_4/
├── test.swf                     # Source SWF file
├── config.toml                  # SWFRecomp configuration
├── runtime/                     # DUPLICATED runtime files
│   ├── native/
│   │   ├── main.c
│   │   ├── runtime.c
│   │   └── include/
│   │       ├── recomp.h
│   │       └── stackvalue.h
│   └── wasm/
│       ├── main.c
│       ├── runtime.c
│       ├── recomp.h
│       ├── stackvalue.h
│       └── index.html
├── RecompiledScripts/           # Generated by SWFRecomp
├── RecompiledTags/              # Generated by SWFRecomp
├── build/                       # Build outputs
├── Makefile                     # DUPLICATED build script
└── build_wasm.sh                # DUPLICATED build script

What Changes Between Tests

Stays the same across tests:

  • Core runtime implementation (runtime.c structure)
  • Type definitions (recomp.h basics)
  • Build process (copy files, compile, link)
  • HTML template structure

Varies between tests:

  • Number and complexity of ActionScript functions needed
  • Variable storage requirements (simple tests don't need variables)
  • Graphics rendering support (graphics tests need additional runtime)
  • Generated code (SWFRecomp output)

SWFModernRuntime Library Analysis

Overview

The SWFModernRuntime repository (located at SWFModernRuntime/) contains a production-quality runtime library that is significantly more advanced than the simple runtimes currently duplicated in test directories.

What SWFModernRuntime Provides

Core Runtime Components:

  • src/actionmodern/action.c (832 lines) - Complete ActionScript VM implementation
  • src/actionmodern/variables.c (209 lines) - Advanced variable storage with hashmap and array optimization
  • include/actionmodern/ - Clean, well-designed header files
  • include/libswf/recomp.h - Runtime API definitions

Key Features:

  1. Complete ActionScript Implementation

    • All standard operators: add, subtract, multiply, divide, equals, less, and, or, not
    • String operations: length, concatenation, equality
    • Variable management: get/set with proper memory ownership
    • Type conversions: string ↔ float ↔ double
    • Built-in functions: trace(), getTime()

  2. Advanced Variable Storage

    • Hashmap-based storage for dynamic variable names
    • Array-based optimization for constant string IDs (O(1) lookup)
    • Copy-on-Store semantics - Variables own their string data
    • Proper memory management - No leaks, proper cleanup
    • Ownership tracking - Knows which strings need to be freed

  3. Production Quality

    • Comprehensive test suite (1,100+ lines in test_*.c files)
    • Clean architecture with separation of concerns
    • Well-documented code
    • Already compiled to static library: build/libSWFModernRuntime.a

Comparison: Test Runtimes vs SWFModernRuntime

Feature trace_swf_4 Runtime SWFModernRuntime
Lines of code 76 lines 1,041 lines
ActionScript ops 1 (trace) 15+ operations
Variable storage None HashMap + Array
Memory management Stack only Heap + Stack
String handling Basic Advanced (ownership, copying)
Type conversions None Full support
Test coverage None Comprehensive
Build artifact None libSWFModernRuntime.a

Current Usage

Some tests already reference SWFModernRuntime via CMake:

# From SWFRecomp/tests/trace_swf_4/CMakeLists.txt
set(RUNTIME_INCLUDES
    ${CMAKE_SOURCE_DIR}/../SWFModernRuntime/include
    ${CMAKE_SOURCE_DIR}/../SWFModernRuntime/include/actionmodern
    ${CMAKE_SOURCE_DIR}/../SWFModernRuntime/include/libswf
)

target_link_libraries(${PROJECT_NAME} PRIVATE
    RecompiledTags
    RecompiledScripts
    ${PROJECT_SOURCE_DIR}/SWFModernRuntime.lib
)

However, the WASM test runtimes don't use SWFModernRuntime - they use simplified, duplicated implementations instead.

Why This Matters for WASM Builds

Current situation:

  • WASM tests use minimal, hand-written runtimes
  • Missing advanced features (variables, complex operations)
  • Each test reimplements basic functionality

Opportunity:

  • SWFModernRuntime can be compiled to WASM with Emscripten
  • Provides a complete, tested runtime for all tests
  • Eliminates the need for test-specific runtime implementations
  • Enables more complex SWF files to work in WASM

Potential Integration Approaches

Option A: Use SWFModernRuntime as-is

  • Compile libSWFModernRuntime.a with Emscripten
  • Link all WASM builds against it
  • Minimal wrapper code needed per test

Option B: Extract needed components

  • Create lightweight WASM runtime based on SWFModernRuntime
  • Cherry-pick only the ActionScript functions needed
  • Smaller WASM file sizes

Option C: Tiered approach

  • Simple tests: minimal runtime (trace only)
  • Standard tests: SWFModernRuntime core (variables, math)
  • Graphics tests: SWFModernRuntime + rendering backend

Headers Available in SWFModernRuntime

SWFModernRuntime/include/
├── actionmodern/
│   ├── action.h           # ActionScript VM API
│   ├── stackvalue.h       # Stack value type definitions
│   └── variables.h        # Variable storage API
├── libswf/
│   ├── recomp.h          # Main runtime header
│   ├── swf.h             # SWF file structures
│   └── tag.h             # Tag definitions
├── common.h              # Common types (u8, u16, u32, etc.)
└── utils.h               # Utility functions

These headers are more complete and better designed than the duplicated headers in test runtimes.

Recommendation: Leverage SWFModernRuntime

Instead of creating a new shared runtime from scratch, we should:

  1. Use SWFModernRuntime as the foundation for all WASM builds
  2. Create thin wrappers for test-specific initialization
  3. Compile once - all tests share the same runtime library
  4. Benefit from ongoing improvements to SWFModernRuntime upstream

This approach provides:

  • ✅ Production-quality, tested runtime
  • ✅ Full ActionScript support out of the box
  • ✅ Proper variable storage and memory management
  • ✅ Easy maintenance (changes in one place)
  • ✅ Compatibility with native builds (already using it)

Recommended Solutions

Option 1: Use SWFModernRuntime Directly (Recommended)

Leverage the existing, production-quality SWFModernRuntime library for all test builds.

Proposed Structure

Use SWFModernRuntime as-is with minimal wrappers:

SWFModernRuntime/                     # EXISTING: Production runtime
├── src/actionmodern/                 # Complete ActionScript VM
│   ├── action.c                      # All ActionScript operations
│   └── variables.c                   # Variable storage
├── src/utils.c                       # Utility functions
├── include/                          # Runtime headers
│   ├── actionmodern/
│   │   ├── action.h
│   │   ├── stackvalue.h
│   │   └── variables.h
│   ├── libswf/
│   │   └── recomp.h
│   ├── common.h
│   └── utils.h
└── build/
    └── libSWFModernRuntime.a         # Pre-compiled library (optional)

SWFRecomp/
├── wasm_wrappers/                    # NEW: Minimal WASM glue code
│   ├── main.c                        # Entry point with Emscripten exports
│   ├── index_template.html           # HTML template with {{TEST_NAME}} placeholder
│   └── README.md                     # Documentation
├── scripts/
│   ├── build_test.sh                 # Unified build script (native + wasm)
│   ├── deploy_example.sh             # Deployment script
│   └── build_all_examples.sh         # Batch build script
└── tests/
    ├── trace_swf_4/
    │   ├── test.swf                  # Test-specific files only
    │   ├── config.toml               # SWFRecomp configuration
    │   ├── RecompiledScripts/        # Generated by SWFRecomp
    │   ├── RecompiledTags/           # Generated by SWFRecomp
    │   └── build/
    │       ├── native/               # Native build output
    │       └── wasm/                 # WASM build output
    └── [other tests]/

SWFRecompDocs/
└── docs/
    └── examples/
        ├── trace_swf_4/              # Deployed WASM (matches test name)
        │   ├── trace_swf_4.wasm      # WASM binary (matches test name)
        │   ├── trace_swf_4.js        # JS loader (matches test name)
        │   └── index.html            # Browser interface
        └── [other tests]/

Key changes:

  • Use SWFModernRuntime source files directly (compile with each test)
  • Minimal WASM wrappers in SWFRecomp/wasm_wrappers/ (~50 lines)
  • Tests contain only test-specific files (SWF + config)
  • Build scripts compile SWFModernRuntime sources + generated code together
  • Naming convention: Test name used throughout (directory, WASM files, deployment)

Benefits

  • ✅ Leverage existing, tested runtime - SWFModernRuntime has 1,000+ lines of production code
  • ✅ Full ActionScript support - All operators, variables, type conversions work out of the box
  • ✅ Consistency across builds - Native and WASM use the same runtime
  • ✅ Easy maintenance - Runtime improvements happen in one place (SWFModernRuntime)
  • ✅ Minimal WASM wrapper - Only need ~100 lines of glue code for Emscripten
  • ✅ Tests only contain test-specific files - SWF and config.toml
  • ✅ Reduced disk usage - Eliminate ~700 lines of duplicated code per test
  • ✅ More capable WASM builds - Complex SWFs with variables and math operations work

Implementation Steps

  1. Build SWFModernRuntime with Emscripten

    • Compile SWFModernRuntime/src/actionmodern/*.c to WASM
    • Create libSWFModernRuntime.wasm.a library
    • Test that all ActionScript functions work in WASM

  2. Create WASM wrapper in SWFRecomp

    • Create SWFRecomp/wasm_wrappers/main.c with Emscripten exports
    • Create frame execution wrapper that calls generated code
    • Create HTML template with test name placeholder

  3. Create unified build script

    • build_wasm.sh that uses SWFModernRuntime
    • Compiles SWFModernRuntime source + test-specific generated code
    • Links everything into single WASM file

  4. Test with existing tests

    • Build trace_swf_4 with new system
    • Build dyna_string_vars_swf_4 with new system
    • Verify output matches current behavior

  5. Migrate tests

    • Remove per-test runtime/ directories
    • Update documentation
    • Enable WASM for all ~50 tests

Option 2: Makefile-Based Build System with Templates

Use a top-level Makefile system with includes and templates.

Proposed Structure

SWFRecomp/
├── build_templates/
│   ├── common.mk                    # Common Makefile rules
│   ├── native.mk                    # Native build rules
│   ├── wasm.mk                      # WASM build rules
│   └── runtime_files.mk             # Runtime file lists
└── tests/
    ├── trace_swf_4/
    │   └── Makefile                 # Simple: includes templates
    └── [other tests]/
        └── Makefile

Note: Uses SWFModernRuntime directly (no local runtime/ directory needed)

Example Test Makefile

TEST_NAME = trace_swf_4
SWF_FILE = test.swf
BUILD_WASM = yes
BUILD_NATIVE = yes

include ../../build_templates/common.mk

Benefits

  • ✅ Standard GNU Make approach - familiar to most developers
  • ✅ Fine-grained control per test
  • ✅ Tests can override specific settings
  • ✅ Works well with existing CMake for some tests

Drawbacks

  • ❌ Still requires a Makefile in each test directory
  • ❌ More complex than Option 1 for simple cases
  • ❌ Make syntax can be difficult to maintain

Option 3: Python/Shell Script-Based Build System

Create a unified build tool that can build any test.

Proposed Structure

SWFRecomp/
├── build_tool.py                    # Main build orchestrator
├── build_config.yaml                # Optional: test-specific overrides
└── tests/
    ├── trace_swf_4/
    │   ├── test.swf
    │   └── config.toml              # SWFRecomp config
    └── [other tests]/

Note: Uses SWFModernRuntime directly (no local runtime/ directory needed)

Usage Examples

# Build single test for WASM
./build_tool.py --test trace_swf_4 --target wasm

# Build all tests for WASM
./build_tool.py --all --target wasm

# Build and deploy to docs examples
./build_tool.py --test trace_swf_4 --target wasm --deploy ../SWFRecompDocs/docs/examples/

Benefits

  • ✅ Most flexible - can handle complex logic
  • ✅ Can auto-generate index.html with test name
  • ✅ Can batch-build multiple tests
  • ✅ Can integrate deployment to docs
  • ✅ Easiest to extend with new features
  • ✅ Better error messages and validation

Drawbacks

  • ❌ New dependency (Python 3)
  • ❌ More initial development work
  • ❌ May be overkill for simple use cases

Implementation Details

Unified Build Script

Key Design Decision: Separate Native and WASM Build Paths

The build system maintains completely separate build paths:

tests/test_name/
└── build/
    ├── native/          # Native build (gcc/clang)
    │   ├── *.o          # Object files
    │   └── test_name    # Executable
    └── wasm/            # WASM build (emcc)
        ├── *.wasm       # WebAssembly binary
        ├── *.js         # Emscripten glue code
        └── index.html   # Browser interface

Why separate paths:

  • Different compilers (gcc vs emcc)
  • Different output formats (executable vs WASM)
  • Different runtime needs (native entry vs Emscripten exports)
  • No cross-contamination between builds
  • Can maintain both simultaneously

Native build approach options:

Option A - Link against pre-built library (faster, current CMake approach):

  • Uses libSWFModernRuntime.a from SWFModernRuntime/build/
  • Faster rebuilds (runtime already compiled)
  • Matches current CMake setup

Option B - Compile from source (simpler, unified with WASM):

  • Compiles SWFModernRuntime sources each time
  • Slower but no library dependency
  • Easier for first-time setup

The script below uses Option B for simplicity, but can be modified for Option A.

Create SWFRecomp/scripts/build_test.sh:

#!/bin/bash
# Usage: ./scripts/build_test.sh <test_name> [native|wasm]
# Example: ./scripts/build_test.sh trace_swf_4 wasm

set -e

TEST_NAME=$1
TARGET=${2:-wasm}              # Default: wasm

SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
SWFRECOMP_ROOT="$(cd "${SCRIPT_DIR}/.." && pwd)"
TEST_DIR="${SWFRECOMP_ROOT}/tests/${TEST_NAME}"
BUILD_DIR="${TEST_DIR}/build/${TARGET}"

# Validate inputs
if [ -z "$TEST_NAME" ]; then
    echo "Error: Test name required"
    echo "Usage: $0 <test_name> [native|wasm]"
    exit 1
fi

if [ ! -d "$TEST_DIR" ]; then
    echo "Error: Test directory not found: $TEST_DIR"
    exit 1
fi

# Run SWFRecomp if needed
if [ ! -d "${TEST_DIR}/RecompiledScripts" ]; then
    echo "Running SWFRecomp..."
    cd "${TEST_DIR}"
    "${SWFRECOMP_ROOT}/build/SWFRecomp" config.toml
fi

# Setup build directory
echo "Setting up build directory..."
mkdir -p "${BUILD_DIR}"

# Setup paths to SWFModernRuntime (sibling directory)
SWFMODERN_ROOT="${SWFRECOMP_ROOT}/../SWFModernRuntime"
SWFMODERN_SRC="${SWFMODERN_ROOT}/src"
SWFMODERN_INC="${SWFMODERN_ROOT}/include"

# Verify SWFModernRuntime exists
if [ ! -d "$SWFMODERN_ROOT" ]; then
    echo "Error: SWFModernRuntime not found at: $SWFMODERN_ROOT"
    echo "Expected directory structure:"
    echo "  /path/to/projects/SWFRecomp/"
    echo "  /path/to/projects/SWFModernRuntime/"
    exit 1
fi

# Copy WASM wrapper files
if [ "$TARGET" == "wasm" ]; then
    cp "${SWFRECOMP_ROOT}/wasm_wrappers/main.c" "${BUILD_DIR}/"
    cp "${SWFRECOMP_ROOT}/wasm_wrappers/index_template.html" "${BUILD_DIR}/index.html"

    # Customize HTML with test name
    sed -i "s/{{TEST_NAME}}/${TEST_NAME}/g" "${BUILD_DIR}/index.html"
fi

# Copy SWFModernRuntime source files (for both native and WASM)
# Note: For native, could alternatively link against libSWFModernRuntime.a
cp "${SWFMODERN_SRC}/actionmodern/action.c" "${BUILD_DIR}/"
cp "${SWFMODERN_SRC}/actionmodern/variables.c" "${BUILD_DIR}/"
cp "${SWFMODERN_SRC}/utils.c" "${BUILD_DIR}/"

# Copy hashmap library (required for variable storage)
cp "${SWFMODERN_ROOT}/lib/c-hashmap/map.c" "${BUILD_DIR}/"

# Copy generated files from SWFRecomp
cp "${TEST_DIR}/RecompiledScripts"/*.c "${BUILD_DIR}/" 2>/dev/null || true
cp "${TEST_DIR}/RecompiledScripts"/*.h "${BUILD_DIR}/" 2>/dev/null || true
cp "${TEST_DIR}/RecompiledTags"/*.c "${BUILD_DIR}/" 2>/dev/null || true
cp "${TEST_DIR}/RecompiledTags"/*.h "${BUILD_DIR}/" 2>/dev/null || true

# Build
if [ "$TARGET" == "wasm" ]; then
    echo "Building WASM..."

    # Check if emcc is available
    if ! command -v emcc &> /dev/null; then
        echo "Error: Emscripten (emcc) not found!"
        echo "Run: source ~/tools/emsdk/emsdk_env.sh"
        exit 1
    fi

    cd "${BUILD_DIR}"
    emcc \
        *.c \
        -I. \
        -I"${SWFMODERN_INC}" \
        -I"${SWFMODERN_INC}/actionmodern" \
        -I"${SWFMODERN_INC}/libswf" \
        -I"${SWFMODERN_ROOT}/lib/c-hashmap" \
        -o "${TEST_NAME}.js" \
        -s WASM=1 \
        -s EXPORTED_FUNCTIONS='["_main","_runSWF"]' \
        -s EXPORTED_RUNTIME_METHODS='["ccall","cwrap"]' \
        -s ALLOW_MEMORY_GROWTH=1 \
        -s INITIAL_MEMORY=16MB \
        -O2

    echo ""
    echo "✅ WASM build complete!"
    echo "Output: ${BUILD_DIR}/${TEST_NAME}.wasm"
    echo ""
    echo "To test:"
    echo "  cd ${BUILD_DIR}"
    echo "  python3 -m http.server 8000"
    echo "  Open http://localhost:8000/index.html"

else
    echo "Building native..."
    cd "${BUILD_DIR}"

    # Option B: Compile SWFModernRuntime from source (simpler, shown here)
    gcc \
        *.c \
        -I. \
        -I"${SWFMODERN_INC}" \
        -I"${SWFMODERN_INC}/actionmodern" \
        -I"${SWFMODERN_INC}/libswf" \
        -I"${SWFMODERN_ROOT}/lib/c-hashmap" \
        -Wall \
        -Wno-unused-variable \
        -std=c17 \
        -o "${TEST_NAME}"

    # Option A: Link against pre-built library (faster, commented out)
    # gcc \
    #     tagMain.c constants.c draws.c \
    #     script_0.c script_defs.c \
    #     main.c \
    #     -I. \
    #     -I"${SWFMODERN_INC}" \
    #     -I"${SWFMODERN_INC}/actionmodern" \
    #     -I"${SWFMODERN_INC}/libswf" \
    #     -L"${SWFMODERN_ROOT}/build" \
    #     -lSWFModernRuntime \
    #     -Wall -Wno-unused-variable -std=c17 \
    #     -o "${TEST_NAME}"

    echo ""
    echo "✅ Native build complete!"
    echo "Output: ${BUILD_DIR}/${TEST_NAME}"
    echo ""
    echo "To run:"
    echo "  ${BUILD_DIR}/${TEST_NAME}"
fi

Deployment Script

Create SWFRecomp/scripts/deploy_example.sh:

#!/bin/bash
# Deploy a test's WASM build to docs examples
# Usage: ./scripts/deploy_example.sh <test_name> [docs_dir]

set -e

TEST_NAME=$1
DOCS_DIR=${2:-../SWFRecompDocs/docs/examples}

SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
SWFRECOMP_ROOT="$(cd "${SCRIPT_DIR}/.." && pwd)"
BUILD_DIR="${SWFRECOMP_ROOT}/tests/${TEST_NAME}/build/wasm"
DEPLOY_DIR="${DOCS_DIR}/${TEST_NAME}"

# Validate inputs
if [ -z "$TEST_NAME" ]; then
    echo "Error: Test name required"
    echo "Usage: $0 <test_name> [docs_dir]"
    exit 1
fi

if [ ! -d "$BUILD_DIR" ]; then
    echo "Error: Build directory not found: $BUILD_DIR"
    echo "Run ./scripts/build_test.sh ${TEST_NAME} wasm first"
    exit 1
fi

# Create deployment directory
mkdir -p "${DEPLOY_DIR}"

# Copy WASM artifacts
echo "Deploying ${TEST_NAME} to ${DEPLOY_DIR}..."
cp "${BUILD_DIR}"/*.wasm "${DEPLOY_DIR}/" 2>/dev/null || true
cp "${BUILD_DIR}"/*.js "${DEPLOY_DIR}/" 2>/dev/null || true
cp "${BUILD_DIR}"/index.html "${DEPLOY_DIR}/" 2>/dev/null || true

echo "✅ Deployed ${TEST_NAME} to ${DEPLOY_DIR}"
echo ""
echo "Files deployed:"
ls -lh "${DEPLOY_DIR}"

Batch Build Script

Create SWFRecomp/scripts/build_all_examples.sh:

#!/bin/bash
# Build all tests and deploy to docs
# Usage: ./scripts/build_all_examples.sh [docs_dir]

set -e

SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
DOCS_DIR=${1:-../SWFRecompDocs/docs/examples}

# List of tests to build (can be auto-discovered or manually maintained)
TESTS=(
    "trace_swf_4"
    "dyna_string_vars_swf_4"
    # Add more tests here
)

echo "Building ${#TESTS[@]} tests for WASM deployment..."
echo ""

for test_name in "${TESTS[@]}"; do
    echo "========================================="
    echo "Building: $test_name"
    echo "========================================="

    # Build
    "${SCRIPT_DIR}/build_test.sh" "$test_name" wasm

    # Deploy
    "${SCRIPT_DIR}/deploy_example.sh" "$test_name" "$DOCS_DIR"

    echo ""
done

echo "✅ All tests built and deployed!"
echo "Documentation examples location: $DOCS_DIR"

Implementation Roadmap

Step 1: Create WASM Wrappers Directory

  1. Create directory: SWFRecomp/wasm_wrappers/
  2. Create main.c with Emscripten exports (minimal ~50 lines)
  3. Create index_template.html with {{TEST_NAME}} placeholder
  4. Create README.md explaining the wrapper approach

Estimated effort: 1-2 hours

Step 2: Create Build Scripts

  1. Create directory: SWFRecomp/scripts/
  2. Create build_test.sh with the script above
  3. Create deploy_example.sh with the deployment script
  4. Create build_all_examples.sh with the batch script
  5. Make them executable: chmod +x scripts/*.sh
  6. Test with trace_swf_4: 
    ./scripts/build_test.sh trace_swf_4 wasm
  7. Verify output: trace_swf_4.wasm, trace_swf_4.js, index.html

Estimated effort: 2-3 hours

Step 3: Test and Verify

  1. Build trace_swf_4 with new system
  2. Compare output with existing build
  3. Test in browser - verify functionality matches
  4. Check that file names match test name (trace_swf_4.wasm, etc.)

Estimated effort: 1 hour

Step 4: Rename Deployed Examples

  1. Rename docs/examples/trace-swf-test/ → docs/examples/trace_swf_4/
  2. Rename trace_swf.wasm → trace_swf_4.wasm
  3. Rename trace_swf.js → trace_swf_4.js
  4. Update index.html to reference new file names
  5. Update any links in documentation

Estimated effort: 30 minutes

Step 5: Clean Up Old Test Files

  1. Once verified, remove trace_swf_4/runtime/ directory
  2. Remove trace_swf_4/Makefile and trace_swf_4/build_wasm.sh
  3. Repeat for dyna_string_vars_swf_4
  4. Update test READMEs with new build process

Estimated effort: 30 minutes

Step 6: Deploy and Test

  1. Use deploy_example.sh to deploy trace_swf_4
  2. Test deployment in SWFRecompDocs/docs/examples/trace_swf_4/
  3. Test batch build with build_all_examples.sh
  4. Verify all examples work in browser

Estimated effort: 30 minutes

Step 7: Enable WASM for More Tests

With the new system in place, enabling WASM for additional tests is trivial:

# Build any test
./scripts/build_test.sh string_add_swf_4 wasm
./scripts/deploy_example.sh string_add_swf_4

# Or batch build
./scripts/build_all_examples.sh

Estimated effort: 5-10 minutes per test

Step 8: Optional - Python Build Tool

If the shell scripts become too complex, migrate to Python:

  1. Create SWFRecomp/build_tool.py
  2. Add features:
    • Better error messages and validation
    • Progress reporting for batch builds
    • Incremental builds (only rebuild if source changed)
    • Integration with SWFRecompDocs to update example listings
    • Automated testing of WASM builds

Estimated effort: 8-12 hours

Benefits Summary

Before (Current System)

  • ❌ ~500 lines of duplicated runtime code per test
  • ❌ ~70 lines of duplicated build scripts per test
  • ❌ Manual setup required for each new WASM-enabled test
  • ❌ Inconsistent runtime implementations
  • ❌ Limited ActionScript support (only trace in simple tests)
  • ❌ Only 2/~50 tests have WASM support
  • ❌ Manual deployment to docs examples

After (Proposed System with SWFModernRuntime)

  • ✅ Production-quality runtime - 1,000+ lines of tested ActionScript VM code
  • ✅ Full ActionScript support - All operators, variables, type conversions
  • ✅ Single runtime source - SWFModernRuntime repository
  • ✅ Zero runtime code in test directories - Only test-specific files (SWF, config)
  • ✅ Minimal WASM wrapper - ~100 lines of Emscripten glue code (shared)
  • ✅ One command to build: ./scripts/build_test.sh <name> wasm
  • ✅ One command to deploy: ./scripts/deploy_example.sh <name>
  • ✅ Easy to enable WASM for all ~50 tests
  • ✅ Automated batch builds and deployment
  • ✅ Consistency with native builds - Both use SWFModernRuntime

Metrics

Metric Current Proposed with SWFModernRuntime Improvement
Runtime quality Basic (76 lines) Production (1,041 lines) 13x more capable
ActionScript ops 1-3 functions 15+ functions Full coverage
Lines of code per test ~650 ~0 100% reduction
Time to add WASM to test ~30 min ~30 sec 60x faster
Build script maintenance 50+ files 3 files 95% reduction
Deployment process Manual Automated -
Variable storage Limited/none HashMap + Array Production-ready
Memory management Stack only Heap + Stack Proper cleanup

Next Steps

  1. Review and approve this proposal
  2. Choose implementation approach (recommended: Option 1 with shell scripts)
  3. Allocate time for implementation (estimated 6-8 hours total for basic system)
  4. Begin with Step 1 (create WASM wrappers directory)
  5. Test thoroughly before removing old build system
  6. Document new process in README and reference docs
  7. Migrate all tests incrementally
  8. Deploy examples to SWFRecompDocs
  9. Add automated testing for WASM builds (optional but recommended)

Build Path Separation Summary

The proposed system fully supports separate native and WASM build paths:

Directory Isolation

tests/trace_swf_4/
├── test.swf                  # Source (shared)
├── config.toml               # Config (shared)
├── RecompiledScripts/        # Generated code (shared)
├── RecompiledTags/           # Generated code (shared)
└── build/
    ├── native/               # SEPARATE: Native build
    │   ├── trace_swf_4       # Linux/Mac/Windows executable
    │   └── *.o               # Native object files
    └── wasm/                 # SEPARATE: WASM build
        ├── trace_swf_4.wasm  # WebAssembly binary
        ├── trace_swf_4.js    # JavaScript loader
        └── index.html        # Browser interface

Command Separation

# Build native version only
./scripts/build_test.sh trace_swf_4 native

# Build WASM version only
./scripts/build_test.sh trace_swf_4 wasm

# Build both (separate commands)
./scripts/build_test.sh trace_swf_4 native
./scripts/build_test.sh trace_swf_4 wasm

Compiler/Toolchain Separation

Aspect Native Build WASM Build
Compiler gcc / clang emcc (Emscripten)
Output Executable binary .wasm + .js
Runtime SWFModernRuntime (source or .a) SWFModernRuntime (compiled to WASM)
Entry point Standard main() Emscripten-exported functions
Build dir build/native/ build/wasm/
Deployment Local execution Deploy to docs/examples/

No Cross-Contamination

  • Native and WASM builds are completely independent
  • Different source files can be included (e.g., WASM-specific wrappers)
  • Different compiler flags and optimizations
  • Can delete one build directory without affecting the other
  • Can maintain both builds simultaneously

Integration with Existing Systems

Native builds can use:

  • Existing CMake setup (for complex tests)
  • New shell script (for simple tests)
  • Pre-built libSWFModernRuntime.a library
  • Or compile SWFModernRuntime from source

WASM builds use:

  • New shell script only (Emscripten-based)
  • Always compile SWFModernRuntime from source (simpler for cross-platform WASM)

Build Flow Diagram

                    test.swf
                       ↓
                 [SWFRecomp]
                       ↓
            ┌──────────┴──────────┐
            ↓                     ↓
    RecompiledScripts      RecompiledTags
            │                     │
            └──────────┬──────────┘
                       ↓
          ┌────────────┴────────────┐
          ↓                         ↓
    [Native Build]            [WASM Build]
    build/native/             build/wasm/
          ↓                         ↓
   ┌──────────────┐         ┌──────────────┐
   │ gcc + .a lib │         │ emcc sources │
   │      or      │         │   compiled   │
   │ gcc sources  │         │   directly   │
   └──────┬───────┘         └──────┬───────┘
          ↓                         ↓
    trace_swf_4              trace_swf_4.wasm
   (executable)              + trace_swf_4.js
          ↓                  + index.html
          ↓                         ↓
   [Run locally]            [Deploy to docs]
    ./trace_swf_4           docs/examples/

Both paths share the same generated code but produce completely different outputs.

Questions - All Resolved ✅

  1. SWFModernRuntime integration approach: ✅ RESOLVED

    • Decision: Compile SWFModernRuntime source files directly with each test (simpler, chosen approach)
    • Alternative: Build libSWFModernRuntime.wasm.a once and link against it (faster rebuilds, more complex)

  2. Naming convention: ✅ RESOLVED & IMPLEMENTED

    • Decision: Use test_name.wasm format (e.g., trace_swf_4.wasm)
    • Completed: Renamed trace_swf.wasm → trace_swf_4.wasm
    • Completed: Renamed deployment directory trace-swf-test/ → trace_swf_4/

  3. HTML templates: ✅ RESOLVED

    • Decision: Generic template with {{TEST_NAME}} placeholder substitution

  4. CMake integration: ✅ RESOLVED

    • Decision: Keep CMake for native builds, use shell scripts for WASM
    • Native builds can use either approach (CMake or scripts)

  5. Hashmap dependency: ✅ RESOLVED & IMPLEMENTED

    • Location: SWFModernRuntime/lib/c-hashmap/ (contains map.c and map.h)
    • Implemented: Build script now copies map.c to build directory
    • Implemented: Include path added: -I"${SWFMODERN_ROOT}/lib/c-hashmap"
    • See lines 522 and 548 in build script

  6. Testing: ✅ RESOLVED - YES

    • Decision: Add automated tests to verify WASM builds work correctly
    • Plan: Reuse SWFModernRuntime's test suite for WASM validation where applicable
    • Implementation: Add to Step 8 (optional Python build tool with testing features)

  7. Documentation deployment: ✅ RESOLVED - YES

    • Decision: Build script should auto-update docs example index page
    • Plan: Auto-generate example listing with screenshots
    • Implementation: Add to Step 8 (optional Python build tool with doc generation)

Appendix: File Size Analysis

Current duplication in tests with WASM support:

SWFRecomp/tests/trace_swf_4/runtime/
├── native/
│   ├── main.c              (~30 lines)
│   ├── runtime.c           (~80 lines)
│   └── include/
│       ├── recomp.h        (~150 lines)
│       └── stackvalue.h    (~50 lines)
└── wasm/
    ├── main.c              (~30 lines)
    ├── runtime.c           (~80 lines)
    ├── recomp.h            (~150 lines)
    ├── stackvalue.h        (~50 lines)
    └── index.html          (~80 lines)

Total: ~700 lines duplicated per test with WASM support

With shared runtime, each test would only contain:

SWFRecomp/tests/trace_swf_4/
├── test.swf               (80 bytes)
└── config.toml            (4 lines)

Total: ~4 lines per test

Code reduction: ~99.4%