调用huggingface_hub将内容向量化

环境:
OS:Windows 10
Python:3.10.11
pycharm:2023.2

 

前提条件:需要安装如下的包

 

pip install transformers --timeout=100
pip install sentence_transformers --timeout=100

 

注意numpy需要安装如下版本
pip install numpy==1.26.4

 

 

1.pycharm里面确定安装了huggingface_hub
如下:

 

2.修改constants.py文件
D:\python-3.10.11\Lib\site-packages\huggingface_hub\constants.py

##ENDPOINT = os.getenv("HF_ENDPOINT") or (_HF_DEFAULT_STAGING_ENDPOINT if _staging_mode else _HF_DEFAULT_ENDPOINT) ##注释掉该行
ENDPOINT ="https://hf-mirror.com" ##新增该行

 

3.运行例子
例子来自官网
https://hf-mirror.com/sentence-transformers/all-MiniLM-L6-v2

#!/usr/bin/env python
#coding=utf-8

from sentence_transformers import SentenceTransformer

sentences = ["This is an example sentence", "Each sentence is converted"]

##model = SentenceTransformer(model_name_or_path='all-MiniLM-L6-v2', local_files_only=False)
model = SentenceTransformer('sentence-transformers/all-MiniLM-L6-v2')
embeddings = model.encode(sentences)
print(embeddings)

 

运行输出: