Added TimeWarp: A cool new web agent benchmark on multiple temporal versions of web environments#385
Conversation
Updated README to clarify environment setup instructions.
|
Fixed the previous bugs |
|
reformatted code, fixed the remaining bugs, should be good now |
|
Hi @farhanishmam, Thank you for contributing an interesting benchmark to the BrowserGym ecosystem, we really appreciate it! 🙂 Would it be possible to maintain TimeWrap as a namespace package, similar to WorkArena (see https://github.com/ServiceNow/WorkArena for inspiration)? This way, you could own both the TimeWrap repository on your GitHub and the Importantly, this would allow you to push updates to |
|
Hi @amanjaiswal73892. Thanks a bunch for the suggestion! I just did that. |
There was a problem hiding this comment.
We can revert this change as timewarp is no longer in /browsergym.
browsergym/pyproject.toml
Outdated
| "browsergym-webarenalite==0.14.3", | ||
| "browsergym-webarena-verified==0.14.3" | ||
| "browsergym-webarena-verified==0.14.3", | ||
| "browsergym-timewarp==0.14.3", |
There was a problem hiding this comment.
This needs to be updated to the actual version of timewarp package, once it is in pypi.
There was a problem hiding this comment.
Adding some references below to help with pypi publishing (for your GitHub repo):
- workarena TOML
- Setting up PYPI
- CI/CD Tests workflow Optional CI/CD tests workflow (after they are migrated to the your repo)
tests/timewarp/__init__.py
Outdated
There was a problem hiding this comment.
I suppose tests/timewarp can be moved to the original repo here https://github.com/sparklabutah/timewarp.
|
Thanks for the review @amanjaiswal73892! Updated the code. |
|
I think BrowserGym should be good now. Should I also do a PR to AgentLab? |
|
The checks have passed. Is there anything I need to do on my end? |
browsergym/pyproject.toml
Outdated
| "browsergym-webarenalite==0.14.3", | ||
| "browsergym-webarena-verified==0.14.3" | ||
| "browsergym-webarena-verified==0.14.3", | ||
| "browsergym-timewarp==0.1.0", |
There was a problem hiding this comment.
It would be great if we could remove browsergym-timewarp==0.1.0 from this toml and add to the dev/reqs for CI/CD here: https://github.com/ServiceNow/BrowserGym/blob/main/dev/requirements.txt. The idea is to to keep the bgym meta-package lightweight and contributors can plugin by installing their benchmarks from pypi.
amanjaiswal73892
left a comment
There was a problem hiding this comment.
Requested minor changes before merging
Hi, please see the comment above. Once done we can merge :) |
|
Thanks a bunch @amanjaiswal73892! Done with the changes. |
Hi, I'm adding our new project TimeWarp. We introduce an environment with three websites: Wiki, News, and Shop, each with 6 temporal versions.