+tree-sitter (380/548)

[mfts]

Formulae: https://formulae.brew.sh/formula/tree-sitter

[king-11]

I am trying to work on this one

[king-11]

@jhheider @mfts if you could help I ran the following operations and am seeing this error:

gh repo clone teaxyz/pantry
cd pantry
export TEA_PANTRY_PATH=$(pwd)
tea pkg init github.com/tree-sitter/tree-sitter
tea pkg build

Before build I edited package.yml as:

# this is the source tarball. Swap the version out for {{version}} or we won’t
# be able to upgrade the package when it updates!
distributable:
  url: https://github.com/tree-sitter/tree-sitter/archive/v{{version}}.tar.gz
  # this obscure option dictates how many components are removed from the
  # extracted tarball. It only works with tarballs. You almost always want
  # `1`. If after extraction the src-directory is funky try `0` or `2`.
  strip-components: 1

# if there’s a github then we can parse the versions
versions:
  # if the repo has a github then we can parse versions
  # this should be one of these (remove the rest or the YAML won’t parse)
  github: tree-sitter/tree-sitter  # reads github tags but only tags of releases (this is usually what you want)

dependencies:
  # to figure out dependencies you can read the project’s README
  # however it is often easier to read the [brew formula](https://formulae.brew.sh/formula/)
  #
  # see tea.xyz/+/ for the packages tea provides
  # if you need a package that isn’t there, you may need to build it!
  # we can help you: https://discord.tea.xyz
  #
  # see semver.org for version specifications
  node: '*'
  emscripten: '*'

build:
  dependencies:
    node: '*'
    emscripten: '*'
    rust: '*'
    # ^^ often packages need dependencies to build, it is very common for
    # packages to need a C compiler and make, the above provide those
  working-directory: build
  script: |
    cargo build -r
  # it’s extremely common for packages to require the above

  # script also supports a list of commands, which is useful for complex builds
  # or if you want to run a command in a different directory

provides:
  - {{prefix}}/target/release/tree-sitter
  # ^^ specify the binaries or tea magic won’t work

The error I am getting is:

error: Uncaught Error: Not in pantry: \ngithub.com/tree-sitter/tree-sitter

https://github.com/teaxyz/pantry#contributing
      throw new TeaError('not-found: pantry: package.yml', {project}, )
            ^
    at https://deno.land/x/libtea@v0.6.2/src/hooks/usePantry.ts:49:13
    at project (https://deno.land/x/libtea@v0.6.2/src/hooks/usePantry.ts:50:5)
    at Object.getDeps (file:///home/king-11/.tea/tea.xyz/brewkit/v0.44.1/lib/usePantry.ts:30:21)
    at file:///home/king-11/.tea/tea.xyz/brewkit/v0.44.1/libexec/sort.ts:24:29
    at ascend (https://deno.land/x/libtea@v0.6.2/src/plumbing/hydrate.ts:62:31)
    at go (https://deno.land/x/libtea@v0.6.2/src/plumbing/hydrate.ts:93:11)
    at hydrate (https://deno.land/x/libtea@v0.6.2/src/plumbing/hydrate.ts:102:13)
    at file:///home/king-11/.tea/tea.xyz/brewkit/v0.44.1/libexec/sort.ts:23:19

[jhheider]

Error: Not in pantry: \ngithub.com/tree-sitter/tree-sitter looks strange. I wonder where that \n came from.

But other things that will need to be fixed:

• node > nodejs.org
• emscripten > not packaged; will need to be packaged if needed
• rust > rust-lang.org
• all runtime dependencies are also build dependencies, so node and emscripten are duplicates there
• files must be created under {{prefix}} by the build script, or the build will be considered a failure; for crates, this is usually: cargo install --root "{{prefix}}" --path .
• crates should generally be namespaced under crates.io, i.e. crates.io/tree-sitter
• provides: should be an array of executables (usually bin/foo, and sbin/bar). here is the code that audits those lines. it'll likely be bin/tree-sitter.

All that said, I'm concerned that there's a newline in your package name somewhere (either the branch name, the project path, or being mentioned somewhere else. If I run the last two of your commands, I get the following (darwin+aarch64):

$ pkg init github.com/tree-sitter/tree-sitter; pkg build
branch 'new/github.com/tree-sitter/tree-sitter' set up to track 'origin/main'.
Switched to a new branch 'new/github.com/tree-sitter/tree-sitter'
  
  created  ./projects/github.com/tree-sitter/tree-sitter/package.yml 
  
  Now type  pkg edit  to open the yaml in your  $EDITOR 
  
  the package will need renaming before we can merge it.
  we typically name packages after their homepage.
  
  if you aren’t sure about the name you can submit and we’ll assist.
error: Uncaught Error: pantry entry invalid. please report this bug!

    https://github.com/teaxyz/pantry/issues/new

----------------------------------------------------->> attachment begin
github.com/tree-sitter/tree-sitter: duplicated mapping key in "/Users/jacob/tea/pantry/projects/github.com/tree-sitter/tree-sitter/package.yml" at line 23, column 3:
      url: git+https://github.com/user ... 
      ^
<<------------------------------------------------------- attachment end
          .catch(cause => { throw new TeaError('parser: pantry: package.yml', {cause, project, filename}) }))

which is expected, since that's an unedited package.yml.

Actually,

$ pwd | hexdump
0000000 552f 6573 7372 6a2f 6361 626f 742f 6165
0000010 702f 6e61 7274 0a79                    
0000018

it looks like pwd might include a new line. That might be the culprit. export TEA_PANTRY_PATH=$(pwd) does work for me, but that's the best guess I have at the moment. :/

[jhheider]

I updated my brewkit to be sure, but it all still works the same. So, my top possible culprits are:

• does linux pwd work differently?
• does your shell treat newlines in envvars differently than mine (OSX built-in zsh)
• some other problem that's getting a newline in the middle of a string

Since most of brewkit is shell scripts, you can get more info out with bash -x "$(tea --prefix)"/tea.xyz/brewkit/v0/bin/pkg-build to have it show all the commands and values it's using.

[king-11]

++ tr : '\n'
+ for x in $(echo "$TEA_PANTRY_PATH" | tr ':' '\n')
+ test -d /home/king-11/pantry/.git
++ GIT_WORK_TREE=/home/king-11/pantry
++ peek.sh
+ PKGS='\ncrates.io/tree-sitter '

Seems like the package name only has issues where can I correct that?

[king-11]

So I tried exporting TEA_PANTRY_PATH manually without pwd still the error is same:

➜ bash -x "$(tea --prefix)"/tea.xyz/brewkit/v0/bin/pkg-build
+ set -e
+ pwd
+ grep -q ' '
+ test -n '' -a -n '' -o -n ''
+++ dirname /home/king-11/.tea/tea.xyz/brewkit/v0/bin/pkg-build
++ cd /home/king-11/.tea/tea.xyz/brewkit/v0/bin/..
++ pwd
+ d=/home/king-11/.tea/tea.xyz/brewkit/v0
+ PATH='/home/king-11/.tea/tea.xyz/brewkit/v0/libexec:/home/king-11/.local/share/zinit/polaris/bin:/home/king-11/.local/bin:/home/king-11/.local/share/pnpm:/run/user/1000/fnm_multishells/16926_1691475666114/bin:/home/king-11/.local/share/fnm:/home/king-11/.pyenv/shims:/home/linuxbrew/.linuxbrew/bin:/home/linuxbrew/.linuxbrew/sbin:/usr/lib64/ccache:/home/king-11/.cargo/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/usr/lib/wsl/lib:/mnt/c/windows/system32:/mnt/c/windows:/mnt/c/windows/System32/Wbem:/mnt/c/windows/System32/WindowsPowerShell/v1.0/:/mnt/c/windows/System32/OpenSSH/:/mnt/c/Program Files/usbipd-win/:/Docker/host/bin:/mnt/c/Users/lakshyasingh/AppData/Local/Microsoft/WindowsApps:/mnt/c/Users/lakshyasingh/AppData/Local/Programs/Microsoft VS Code/bin:/var/lib/snapd/snap/bin'
+ test -z ''
+ test -z /home/king-11/pantry
++ echo /home/king-11/pantry
++ tr : '\n'
+ for x in $(echo "$TEA_PANTRY_PATH" | tr ':' '\n')
+ test -d /home/king-11/pantry/.git
++ GIT_WORK_TREE=/home/king-11/pantry
++ peek.sh
+ PKGS='\ncrates.io/tree-sitter '
+ test -n ''
+ available.ts
+ test -z ''
++ tea gh auth token
+ GITHUB_TOKEN=<GH_TOKEN>
+ test -z <GH_TOKEN>
+ export GITHUB_TOKEN
++ sort.ts '\ncrates.io/tree-sitter' --delimiter ' '
error: Uncaught Error: Not in pantry: \ncrates.io/tree-sitter

https://github.com/teaxyz/pantry#contributing
      throw new TeaError('not-found: pantry: package.yml', {project}, )
            ^
    at https://deno.land/x/libtea@v0.6.2/src/hooks/usePantry.ts:49:13
    at project (https://deno.land/x/libtea@v0.6.2/src/hooks/usePantry.ts:50:5)
    at Object.getDeps (file:///home/king-11/.tea/tea.xyz/brewkit/v0/lib/usePantry.ts:30:21)
    at file:///home/king-11/.tea/tea.xyz/brewkit/v0/libexec/sort.ts:24:29
    at ascend (https://deno.land/x/libtea@v0.6.2/src/plumbing/hydrate.ts:62:31)
    at go (https://deno.land/x/libtea@v0.6.2/src/plumbing/hydrate.ts:93:11)
    at hydrate (https://deno.land/x/libtea@v0.6.2/src/plumbing/hydrate.ts:102:13)
    at file:///home/king-11/.tea/tea.xyz/brewkit/v0/libexec/sort.ts:23:19
+ PKGS=

[jhheider]

Ok, we'll we can definitely see the error showing up out of peek.sh. This is the call:

PKGS="$(GIT_WORK_TREE="$x" peek.sh) $PKGS"

The relevant code in peek.sh is:

OUTPUT=""

for CHANGED_FILE in $CHANGED_FILES; do
  PROJECT=$(echo "$CHANGED_FILE" | sed -n 's#projects/\(.*\)/package\.yml$#\1#p')
  if test -z "$PROJECT"
  then
    true # noop
  elif test "$1" = "--print-paths"; then
    OUTPUT="$OUTPUT\n$CHANGED_FILE"
  else
    OUTPUT="$OUTPUT\n$PROJECT"
  fi
done

# shellcheck disable=SC2046
echo $(echo $OUTPUT | sort | uniq)

So, your /bin/sh (dash I assume, under a ubuntu/debian derivative?) is taking:

echo $(echo "\ncrates.io/tree-sitter " | sort | uniq)

and returnting \ncrates.io/tree-sitter . That's not great. Quick test: if you replace the end of ~/.tea/tea.xyz/brewkit/v0/libexec/peek.sh to the following, does it work?

  elif test "$1" = "--print-paths"; then
    OUTPUT="$OUTPUT $CHANGED_FILE"
  else
    OUTPUT="$OUTPUT $PROJECT"
  fi
done

# shellcheck disable=SC2046
echo $(echo $OUTPUT | tr ' ' '\n' | sort | uniq | tr '\n' ' ')

[king-11]

that does it

[jhheider]

Great. I think it might behoove us to just change all of brewkit to use #!/usr/bin/env -S tea bash for sanity's sake. This is at least the second dash idiosyncrasy that's shown up.

[king-11]

Right when I execute pkg build it gets stuck:

pkg build                                                                    
  
  building crates.io/tree-sitter
  
  fetching /home/king-11/pantry/srcs/crates.io∕tree-sitter-0.20.8.tar.gz
/home/king-11/pantry/srcs/crates.io∕tree-sitter-0.20.8.tar.gz
  
  extracting /home/king-11/pantry/srcs/crates.io∕tree-sitter-0.20.8
  
  staging for build

Should it be taking this long? package.yml is as below now:

distributable:
  url: https://github.com/tree-sitter/tree-sitter/archive/refs/tags/v{{version}}.tar.gz
  strip-components: 1
 
provides:
  - bin/tree-sitter

platforms:
  - linux

versions:
  github: tree-sitter/tree-sitter
  strip: /v/

build:
  dependencies:
    rust-lang.org: ^1.65
    rust-lang.org/cargo: '*'
  script:
    cargo install --locked --path . --root {{prefix}}

[king-11]

Didn't really get what you meant by dash there: the - redirection of pipe or the shell that am using? Very much not familiar with that 😅