Merge pull request #930 from lmorg/develop

# v7.1

This release focuses mainly on bugfixes and quality-of-life with the exception of three **experimental** new major additions:
* `foreach` now supports running processes in parallel
* `fanout` is a new builtin that allows sending stdout to the stdin of many processes
* `md` is a new datatype added. Currently only supports rendering markdown tables but more features will follow in future releases

## Breaking Changes

None

## v7.1.xxxx

### Features

* `fanout`: new builtin (EXPERIMENTAL)
* REPL: default prompt modernized
* ANSI titlebar: Windows support added ([PR](#931))
* autocomplete: add support for substring matches ([discussion](#934))
* autocomplete: remove `stat` syscall from recursive directory lookups to improve performance
* `foreach`: parallel support added (EXPERIMENTAL) ([discussion](#933))
* macro vars: history support added (history stored in cache.db)
* integrations: `basti` autocomplete
* integrations: Charm Bracelet's `freeze`
* tooling: Makefile added for simplified development on POSIX systems
* `runtime --about`: `GOMAXPROCS` field added
* xml: improved interop between CSV and XML
* core: data types can now register what their layout type is. Allowing for `format` to better deduce how to reformat the data. eg next few refinements below
* csv: support for JSON and YAML maps
* yaml: convert tables to maps
* json: convert tables to maps
* `select`: support for maps
* `config`: new option: **syntax-completion-enabled**
* `config`: new option: **history-write-enabled**
* md: new datatype, markdown (EXPERIMENTAL)
* murex: new flag added: `-ignore-whatsnew`
* core: hint text added for `cd -` ([PR](#951))
* core: errors reformatted to make them slightly easier to read
* `test`: default test report now uses the full width of the terminal, plus other minor readability improvements
* docgen: new functions added
* docgen: support for metadata
* Update dependencies

### Bug Fixes

* `datetime`: stdout datatype set to `str`
* `fid-kill`: autocomplete fix ([issue](#949))
* exec: don't error on wait if PID is already closed ([issue](#946))
* spellcheck: fixed bug causing users to be re-prompted when `aspell` isn't installed
* REPL: fixed regression bug causing hint text not to display application hints
* `murex-package`: working directory preserved when installing packages
* readline: panic fixed in command line preview
* cache.db: fixed potential race condition when checking if cache.db is enabled
* REPL: fixed potential race condition when instantiating `readline`
* tests: `env` test skips on Windows, uses full path on POSIX
* tests: skip tilde parser test if username contains non-var chars (eg `@`)
* readline: null check when using tab grid
* readline: improved SGR detection
* readline: fixed wider character rendering bug when using tab grid
* readline: fixed potential race condition with read/write access to unicode slices
* readline: fixed potential race condition with terminal width caching
* readline: fixed potential race condition with asynchronous updates to tab completion
* readline: fixed potential race condition with hint text height reporting
* readline: fixed potential race condition with tab completion height reporting
* readline: fixed potential race condition with previews
* readline: fixed potential race condition with hint text cache

## Special Thanks

Thank yous for this release goes to [TheBearodactyl](https://github.com/TheBearodactyl), [JeffMelton](https://github.com/JeffMelton), [Th0rgal](https://github.com/Th0rgal), [lokalius](https://github.com/lokalius), [tiymat](https://github.com/tiymat) for your code, testing and feedback.

Also thank you to everyone in the [discussions group](https://github.com/lmorg/murex/discussions) and all who raise bug reports.

You rock!

Author: lmorg

.github/workflows/archlinux-aur-package.yaml

@@ -7,7 +7,7 @@ on:
   workflow_dispatch:
 
 env:
-  GO_VERSION: "1.24"
+  GO_VERSION: "1.25"
   PKGBUILD:   ".github/workflows/aur/PKGBUILD"
   SRCINFO:    ".github/workflows/aur/SRCINFO"
 

.github/workflows/clean-build.yaml

@@ -5,17 +5,17 @@ on:
     paths:
       - "**/*.go"
     branches:
-      - "!v*"
+      - "develop"
 
   workflow_dispatch:
 
 jobs:
-  Clean-Build:
+  Clean-Build-Bash:
 
     runs-on: ubuntu-latest
     strategy:
       matrix:
-        go-version: [ "1.21", "1.22", "1.23", "1.24" ]
+        go-version: [ "1.21", "1.22", "1.23", "1.24", "1.25" ]
 
     steps:
       - name: Setup Go ${{ matrix.go-version }}
@@ -28,9 +28,37 @@ jobs:
           go version
 
       - name: Build Murex
+        shell: bash
         run: |
           GOBIN="$(pwd)" go install -v github.com/lmorg/murex@${{ github.ref_name }}
 
       - name: Display Murex version
         run: |
           ./murex --version
+
+  Clean-Build-Powershell:
+
+    runs-on: ubuntu-latest
+    strategy:
+      matrix:
+        go-version: [ "1.21", "1.22", "1.23", "1.24", "1.25" ]
+
+    steps:
+      - name: Setup Go ${{ matrix.go-version }}
+        uses: actions/setup-go@v4
+        with:
+          go-version: ${{ matrix.go-version }}
+
+      - name: Display Go version
+        run: |
+          go version
+
+      - name: Build Murex
+        shell: pwsh
+        run: |
+          $env:GOBIN="$(pwd)"; & go install -v github.com/lmorg/murex@${{ github.ref_name }}
+
+      - name: Display Murex version
+        run: |
+          ./murex --version
+

.github/workflows/code-coverage.yaml

@@ -13,7 +13,7 @@ jobs:
     runs-on: ubuntu-latest
     strategy:
       matrix:
-        go-version: [ "1.24" ]
+        go-version: [ "1.25" ]
     env:
       MUREX_TEST_NO_EXEC_DEPS: true
     steps:

.github/workflows/deploy-dev-docs.yaml

@@ -3,6 +3,8 @@ name: Deploy Docs
 on:
   push:
     branches:
+      - develop
+      - website
       - master
     paths:
       - "**/*.md"
@@ -14,6 +16,7 @@ on:
       - "**/*.jpeg"
       - "**/*.gif"
       - "**/*.svg"
+      - "package.json"
 
   workflow_dispatch:
 
@@ -22,7 +25,7 @@ jobs:
     runs-on: ubuntu-latest
     strategy:
       matrix:
-        go-version: [ "1.24" ]
+        go-version: [ "1.25" ]
 
     steps:
       - name: Checkout
@@ -49,6 +52,8 @@ jobs:
         run: |
           ./docgen -panic -warning -config gen/docgen.yaml
           mv -v *.md *.svg docs/
+          mkdir -p gen/vuepress/public/images
+          cp -vR images/* gen/vuepress/public/images
           mv -v gen/vuepress docs/.vuepress
           
           echo Lower case documents...
@@ -70,6 +75,7 @@ jobs:
       - name: Build Docs
         env:
           NODE_OPTIONS: --max_old_space_size=8192
+          NODE_ENV: DEV
         run: |-
           pnpm run docs:build
           > docs/.vuepress/dist/.nojekyll
@@ -79,13 +85,17 @@ jobs:
           cp -v docs/.vuepress/dist/install.html      docs/.vuepress/dist/INSTALL.html
           cp -v docs/.vuepress/dist/download.html     docs/.vuepress/dist/DOWNLOAD.html
           cp -v docs/.vuepress/dist/contributing.html docs/.vuepress/dist/CONTRIBUTING.html
-  
+
       - name: Deploy Docs
         env:
           AWS_ACCESS_KEY_ID:     ${{ secrets.S3_ACCESS }}
           AWS_SECRET_ACCESS_KEY: ${{ secrets.S3_SECRET }}
           AWS_DEFAULT_REGION:    eu-west-1
           AWS_DEFAULT_OUTPUT:    json
+          CLOUDFRONT_ID:         ${{ github.ref == 'refs/heads/master' && secrets.CLOUDFRONT_DISTRIBUTION_ID_MASTER || secrets.CLOUDFRONT_DISTRIBUTION_ID_DEV }}
+          S3_PREFIX:             ${{ github.ref == 'refs/heads/master' && 'master' || 'develop' }}
         run: |
-          aws s3 sync docs/.vuepress/dist s3://murex.rocks/master --exclude "*.md" --no-progress
-          aws cloudfront create-invalidation --distribution-id "${{ secrets.CLOUDFRONT_DISTRIBUTION_ID_MASTER }}" --paths "/*"
+          mv -v images/vhs* images/banner* docs/.vuepress/dist/
+          aws s3 sync docs/.vuepress/dist s3://murex.rocks/${S3_PREFIX} --exclude "*.md" --no-progress --delete
+          aws cloudfront create-invalidation --distribution-id "${CLOUDFRONT_ID}" --paths "/*"
+    

.github/workflows/deploy-docs.yaml

@@ -8,7 +8,7 @@ on:
   workflow_dispatch:
 
 env:
-  GO_VERSION: "1.24"
+  GO_VERSION: "1.25"
 
 jobs:
   Build-Downloads:

.github/workflows/murex-downloads.yaml

@@ -13,7 +13,7 @@ jobs:
     runs-on: ubuntu-latest
     strategy:
       matrix:
-        go-version: [ '1.21', '1.22', '1.23', "1.24" ]
+        go-version: [ '1.21', '1.22', '1.23', "1.24", "1.25" ]
     env:
       MUREX_BUILD_FLAGS: --no-colour
       MUREX_TEST_NO_EXEC_DEPS: true
@@ -30,21 +30,17 @@ jobs:
         run: |
           go version
 
-      - name: Go Test
+      - name: Build
         run: |
-          go test ./... -tags $(cat builtins/optional/standard-opts.txt) -count 1 -race
+          make build
 
-      - name: Build native target
+      - name: Test
         run: |
-          go build -tags $(cat builtins/optional/standard-opts.txt) -v github.com/lmorg/murex
-
-      - name: Behavioural tests
-        run: |
-          ./murex -c 'g: behavioural/*.mx -> foreach: f { source $f }; test: run *'
+          make test
 
       - name: Build cross-compiled targets
         run: |
-          ./murex ./test/build_all_platforms.mx --no-colour
+          ./bin/murex ./test/build_all_platforms.mx --no-colour
 
       - name: Build web assembly target
         run: |

.github/workflows/murex-tests.yaml

@@ -32,6 +32,7 @@ wasm_exec.js
 *.prof
 *.pprof
 *local.*
+/local/*
 
 debug.log
 

.gitignore

@@ -6,16 +6,14 @@
 <div id="toc">
 
 - [Installing From A Package Manager](#installing-from-a-package-manager)
-  - [ArchLinux](#archlinux)
-  - [FreeBSD Ports](#freebsd-ports)
-  - [Homebrew](#homebrew)
-  - [MacPorts](#macports)
 - [Pre-Compiled Binaries (HTTPS download)](#pre-compiled-binaries-https-download)
 - [Compiling From Source](#compiling-from-source)
   - [Prerequisites](#prerequisites)
   - [Compiling](#compiling)
+    - [Bash: Linux, MacOS, UNIX](#bash-linux-macos-unix)
+    - [Powershell: Windows](#powershell-windows)
+    - [Makefile: Linux, MacOS, UNIX](#makefile-linux-macos-unix)
 - [External Dependencies (Optional)](#external-dependencies-optional)
-- [Recommended Terminal Typeface](#recommended-terminal-typeface)
 
 </div>
 
@@ -38,6 +36,7 @@ There is a more detailed breakdown of known compatibility issues in the
 
 [![Packaging status](https://repology.org/badge/vertical-allrepos/murex.svg?exclude_unsupported=1)](https://repology.org/project/murex/versions)
 
+
 ### ArchLinux
 
 From AUR: [https://aur.archlinux.org/packages/murex](https://aur.archlinux.org/packages/murex)
@@ -65,6 +64,7 @@ brew install murex
 port install murex
 ```
 
+
 ## Pre-Compiled Binaries (HTTPS download)
 
 [![Version](version.svg)](DOWNLOAD.md)
@@ -89,7 +89,7 @@ binaries.
 
 You will need `go` (Golang) compiler, and `git` installed.
 
-> Go 1.20 or higher is required.
+> Go 1.21 or higher is required.
 
 These should be easy to install on most operating systems however Windows is a
 lot more tricky with regards to `gcc`. Please check with your operating systems
@@ -104,32 +104,38 @@ package manager first but see further reading below if you get stuck.
 
 Installation from source is as simple as the following one liner:
 
+#### Bash: Linux, MacOS, UNIX
+
 ```bash
 GOBIN="$(pwd)" go install -v github.com/lmorg/murex@latest
 ```
 
-However you can change the `GOBIN` value to point to any writable location you
-wish.
+#### Powershell: Windows
+
+```powershell
+$env:GOBIN="$(pwd)"; & go install -v github.com/lmorg/murex@latest
+```
+
+#### Makefile: Linux, MacOS, UNIX
+
+We also provide a `Makefile`.
+
+This is convenient if you already have the source downloaded or want to taylor
+the install. For example reintroduce a deprecated builtin, add optional
+builtins, or utilize a different backend library for networking and/or sqlite3.
+
+For options available in the Makefile, run `make help`.
 
 ## External Dependencies (Optional)
 
 Some of Murex's extended features will have additional external dependencies.
 
 * `aspell`: This is used for spellchecking. Murex will automatically enable or
   disable spellchecking based on whether `aspell` can be found in your `$PATH`.
-  [http://aspell.net](http://aspell.net)
+  ([GNU Aspell](http://aspell.net))
 
 * `git`: This is used by Murex's package manager, `murex-package`.
-  [How to install git](https://github.com/git-guides/install-git)
-
-## Recommended Terminal Typeface
-
-This is obviously just a subjective matter and everyone will have their own
-personal preference. However if I was asked what my preference was then that
-would be [Hasklig](https://github.com/i-tu/Hasklig). It's a clean typeface
-based off Source Code Pro but with a few added ligatures - albeit subtle ones
-designed to make Haskell more readable. Those ligatures also suite Murex
-pretty well. So the overall experience is a clean and readable terminal.
+  ([How to install git](https://github.com/git-guides/install-git))
 
 ## See Also