local chatglm api #3

Jerry-Kon · 2023-09-07T02:57:43Z

add feature about local chatglm api
add the relevant test program

kerthcet · 2023-09-07T04:35:49Z

apis/tests/chatglm_test.py

@@ -0,0 +1,13 @@
+import sys
+sys.path.append('/home/jerry_kon/my_chatglm/ChatLLM')


Remove this.

kerthcet · 2023-09-07T04:36:08Z

apis/tests/chatglm_test.py

@@ -0,0 +1,13 @@
+import sys
+sys.path.append('/home/jerry_kon/my_chatglm/ChatLLM')
+from apis.api import ChatLLM


Format this with python lint tools.

kerthcet · 2023-09-07T04:37:22Z

apis/tests/chatglm_test.py

+import sys
+sys.path.append('/home/jerry_kon/my_chatglm/ChatLLM')
+from apis.api import ChatLLM
+chat = ChatLLM(


for unit test, this is not available because we have no model in this project. I'm still figuring out the proper approach to solve this, so currently, just remove this test.

kerthcet · 2023-09-07T04:37:42Z

apis/tests/llama_test.py

@@ -0,0 +1,13 @@
+import sys
+sys.path.append('/home/jerry_kon/my_chatglm/ChatLLM')


Remove this.

kerthcet · 2023-09-07T04:38:33Z

llms/chatglm.py

@@ -0,0 +1,22 @@
+from transformers import AutoTokenizer, AutoModel
+
+#if not history:


Remove this, it's ugly.

kerthcet · 2023-09-07T04:38:44Z

llms/chatglm.py

+        self.tokenizer = AutoTokenizer.from_pretrained(model_name_or_path, trust_remote_code=True)
+        self.model = AutoModel.from_pretrained(model_name_or_path, trust_remote_code=True).half().cuda()
+        self.model = self.model.eval()
+    def completion(self,system_prompt, user_prompt):


Format these codes plz.

kerthcet · 2023-09-08T03:27:02Z

llms/chatglm.py

@@ -0,0 +1,20 @@
+from transformers import AutoTokenizer, AutoModel
+
+class ChatglmChat:


This should inherit from APIChat

kerthcet · 2023-09-08T03:27:47Z

llms/chatglm.py

+        self.model = (
+            AutoModel.from_pretrained(model_name_or_path, trust_remote_code=True)
+            .half()
+            .cuda()
+        )


Suggested change

self.model = (

AutoModel.from_pretrained(model_name_or_path, trust_remote_code=True)

.half()

.cuda()

)

self.model = AutoModel.from_pretrained(model_name_or_path, trust_remote_code=True)

.half()

.cuda()

kerthcet · 2023-09-08T03:44:44Z

apis/tests/llama_test.py

@@ -0,0 +1,11 @@
+from apis.api import ChatLLM


format this

kerthcet · 2023-09-08T03:46:17Z

apis/tests/llama_test.py

@@ -0,0 +1,11 @@
+from apis.api import ChatLLM
+chat = ChatLLM(


Please refer to python test about how to write a Test. Basically, it should be

Class TestXXX(): pass def test_xxx(): pass

Please remove this to the top-level ChatLLM/integration_tests/ which means you should create a new folder.

Also add a test for chatGLM

kerthcet · 2023-09-08T03:51:34Z

llms/chatglm.py

+        )
+        self.model = self.model.eval()
+
+    def completion(self, system_prompt, user_prompt):


Add a warning here if we do provide the system_prompt.

kerthcet reviewed Sep 7, 2023

View reviewed changes

kerthcet reviewed Sep 8, 2023

View reviewed changes

Jerry-Kon force-pushed the main branch from b13f283 to d463e1a Compare September 8, 2023 12:26

updata pr

5dab5e8

Jerry-Kon force-pushed the main branch from d463e1a to 5dab5e8 Compare September 8, 2023 12:40

kerthcet merged commit 3d4f31b into InftyAI:main Sep 8, 2023

		@@ -0,0 +1,13 @@
		import sys
		sys.path.append('/home/jerry_kon/my_chatglm/ChatLLM')

		@@ -0,0 +1,22 @@
		from transformers import AutoTokenizer, AutoModel

		#if not history:

		@@ -0,0 +1,20 @@
		from transformers import AutoTokenizer, AutoModel

		class ChatglmChat:

local chatglm api #3

local chatglm api #3

Uh oh!

Conversation

Jerry-Kon commented Sep 7, 2023

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants