feat: implement NotImplementedError for token counting in LLMs and reintroduce disabled token count method

Signed-off-by: -LAN- <laipz8200@outlook.com>
release/0.15.6-alpha-1 0.15.6-alpha.1
-LAN- 11 months ago
parent 94c08462a1
commit fa6fa730b5
No known key found for this signature in database
GPG Key ID: 6BA0D108DED011FF

@ -553,8 +553,7 @@ if you are not sure about the structure.
:param tools: tools for tool calling
:return:
"""
# Disable the token count in LLMs for profermance testing.
return 0
raise NotImplementedError
def enforce_stop_tokens(self, text: str, stop: list[str]) -> str:
"""Cut off the text as soon as any stop words occur."""
@ -915,3 +914,7 @@ if you are not sure about the structure.
filtered_model_parameters[parameter_name] = parameter_value
return filtered_model_parameters
def _get_num_tokens_by_gpt2(self, text: str) -> int:
# Disable the token count in LLMs for profermance testing.
return 0

Loading…
Cancel
Save