Problem with benchmarks (pandas)

[gsakkas]

Hello,

I am trying to run the benchmarks automatically, but I have a problem running pandas-12, pandas-31 and pandas-36. It seems that dependency_setup.sh fails to install the package and I can't execute the tests. Is there anything else that needs to be done differently for the other bugs?

Thank you,
George

[JohnnyPeng18]

Hello,

You should follow the instructions in the original BugsInPy repo for installation.

[gsakkas]

Yes, I tried that too but it fails even with BugsInPy scripts for me. It works fine with all the other benchmarks, so I was wondering if you did anything extra to make these specific ones to work.

[JohnnyPeng18]

I am not sure what your problem is. We did not do anything extra to run the benchmark. I suggest you use the script provided by PyTER at https://github.com/kupl/PyTER/blob/main/INSTALL.md if you still cannot address the problem.

[gsakkas]

I used your code initially, but it seems that you have pushed an older version of TypeBugs as well. PyTER has made several updates on their benchmarks 3 months ago that I don't see at your repo. For example the pandas-22198 test.

I also had to make several adjustments in the requirements.txt of several projects just to be able to run the benchmarks with the correct programs (not the buggy ones) that pass all their tests. That's why I asked if you made any more updates locally that you haven't pushed maybe here. Because, I wasn't able to run the benchmarks as is.

I can open a PR if you want with all the changes.

[JohnnyPeng18]

Hi,

Thanks very much for your efforts! Please kindly create a PR, and I will merge the changes accordingly.

[gsakkas]

Hello,

Ok, I will open a PR soon with all the changes. I am close to make everything work. However, I have the opposite problem now. For example, pandas-36 in bugsinpy doesn't seem to fail the tests. Your missing-145.py, missing-176.py and missing.py still pass the given test and don't crash with a TypeError like all the other benchmarks. However, the patch.txt for that one shows actually more changed lines. Any ideas for this?

[chenyangyc]

@gsakkas

Hello, I encountered a similar issue as you: I had to adjust the requirements of many projects to successfully complete the setup process.

Additionally, the dependency setup script in typebugs couldn't run directly.

I'm curious—did you eventually manage to resolve these issues and successfully run the benchmark?