（五）t5x依赖包

一、相关的要求

# 必需的包
airio @ git+https://github.com/google/airio#egg=airio
absl-py
cached_property
clu @ git+https://github.com/google/CommonLoopUtils#egg=clu
flax @ git+https://github.com/google/flax#egg=flax
fiddle >= 0.2.5
gin-config
jax >= 0.4.16
jaxlib >= 0.4.16
jestimator @ git+https://github.com/google-research/jestimator#egg=jestimator
numpy
optax @ git+https://github.com/deepmind/optax#egg=optax
orbax-checkpoint >= 0.5
seqio @ git+https://github.com/google/seqio#egg=seqio
tensorflow-cpu
tensorstore >= 0.1.20

# gcp 额外的包
gevent
google-api-python-client
google-compute-engine
google-cloud-storage
oauth2client

# test 额外的包
pytest
t5

# tpu 额外的包
jax[tpu] >= 0.4.16

# gpu 额外的包
ipdb==0.13.9
fasttext==0.9.2
pysimdjson==5.0.2
pytablewriter==0.64.2
gdown==4.5.3
best-download==0.0.9
lm_dataformat==0.0.20
dllogger @ git+https://github.com/NVIDIA/dllogger#egg=dllogger
tfds-nightly
t5==0.9.4

二、安装后发现gpu用不了

import tensorflow as tf
import jax
print("Num GPU Avail: ", len(tf.config.list_physical_devices("GPU")))
print("JAX", jax.local_device_count())

三、重新安装GPU

 由于nvidia-smi是12.2，安装的tensorflow是2.16.1

版本不对

应该安装

pip install tensorflow==2.14.1
pip install tensorflow-text==2.14.0

四、最终的

absl-py==2.1.0
accelerate==0.28.0
airio==0.0.1
array_record==0.5.1
asttokens==2.2.1
astunparse==1.6.3
attrs==23.2.0
Babel==2.15.0
backcall==0.2.0
beautifulsoup4==4.12.3
best-download==0.0.9
bitsandbytes==0.43.0
cached-property==1.5.2
cachetools==5.3.3
certifi==2024.2.2
chardet==5.2.0
charset-normalizer==3.3.2
chex==0.1.86
click==8.1.7
cloudpickle==3.0.0
clu==0.0.12
cmake==3.28.4
colorama==0.4.6
comm==0.1.3
contextlib2==21.6.0
DataProperty==1.0.1
debugpy==1.6.7
decorator==5.1.1
DLLogger==1.0.0
dm-tree==0.1.8
docstring_parser==0.16
editdistance==0.8.1
etils==1.7.0
executing==1.2.0
fasttext==0.9.2
fiddle==0.3.0
filelock==3.13.1
flatbuffers==24.3.25
flax==0.8.5
fsspec==2024.3.1
future==1.0.0
gast==0.5.4
gdown==4.5.3
gin-config==0.5.0
google-auth==2.29.0
google-auth-oauthlib==1.0.0
google-pasta==0.2.0
gplearn==0.4.2
grain-nightly==0.0.7
graphviz==0.20.3
grpcio==1.64.0
h5py==3.11.0
huggingface-hub==0.21.4
idna==3.6
immutabledict==4.2.0
importlib_resources==6.4.0
ipdb==0.13.9
ipykernel==6.22.0
ipython==8.12.0
jax==0.4.28
jaxlib==0.4.28
jaxtyping==0.2.29
jedi==0.18.2
jestimator==0.3.3
Jinja2==3.1.3
joblib==1.2.0
jsonlines==4.0.0
jupyter_client==8.2.0
jupyter_core==5.3.0
keras==2.14.0
libclang==18.1.1
libcst==1.4.0
lit==18.1.2
lm-dataformat==0.0.20
lxml==5.2.2
Markdown==3.6
markdown-it-py==3.0.0
MarkupSafe==2.1.5
matplotlib-inline==0.1.6
mbstrdecoder==1.1.3
mdurl==0.1.2
mesh-tensorflow==0.1.21
ml-collections==0.1.1
ml-dtypes==0.2.0
more-itertools==10.2.0
mpmath==1.3.0
msgpack==1.0.8
namex==0.0.8
nest-asyncio==1.5.6
networkx==3.2.1
nltk==3.8.1
numpy==1.25.2
nvidia-cublas-cu11==11.10.3.66
nvidia-cuda-cupti-cu11==11.7.101
nvidia-cuda-nvrtc-cu11==11.7.99
nvidia-cuda-runtime-cu11==11.8.89
nvidia-cudnn-cu11==8.5.0.96
nvidia-cufft-cu11==10.9.0.58
nvidia-curand-cu11==10.2.10.91
nvidia-cusolver-cu11==11.4.0.1
nvidia-cusparse-cu11==11.7.4.91
nvidia-nccl-cu11==2.14.3
nvidia-nvtx-cu11==11.7.91
oauthlib==3.2.2
opt-einsum==3.3.0
optax==0.2.3.dev0
optree==0.11.0
orbax-checkpoint==0.5.14
packaging==23.1
pandas==1.5.2
parso==0.8.3
pathvalidate==2.5.2
pexpect==4.9.0
pickleshare==0.7.5
pillow==10.2.0
pip==23.3.1
platformdirs==3.2.0
portalocker==2.8.2
promise==2.3
prompt-toolkit==3.0.38
protobuf==3.20.3
psutil==5.9.4
ptyprocess==0.7.0
pure-eval==0.2.2
pyasn1==0.6.0
pyasn1_modules==0.4.0
pybind11==2.12.0
pyglove==0.4.4
Pygments==2.18.0
pysimdjson==5.0.2
PySocks==1.7.1
pytablewriter==0.64.2
python-dateutil==2.8.2
pytz==2024.1
PyYAML==6.0.1
pyzmq==25.0.2
regex==2023.12.25
rehash==1.0.1
requests==2.31.0
requests-oauthlib==2.0.0
rich==13.7.1
rouge-score==0.1.2
rsa==4.9
sacrebleu==2.4.2
safetensors==0.4.2
scikit-learn==1.2.0
scipy==1.9.3
sentencepiece==0.2.0
seqio==0.0.18
seqio-nightly==0.0.18.dev20240528
setuptools==68.2.2
six==1.16.0
sklearn==0.0.post1
soupsieve==2.5
stack-data==0.6.2
sympy==1.12
t5==0.9.4
t5x==0.0.0
tabledata==1.3.3
tabulate==0.9.0
tcolorpy==0.1.6
tensorboard==2.14.1
tensorboard-data-server==0.7.2
tensorflow==2.14.1
tensorflow-cpu==2.16.1
tensorflow-datasets==4.9.4
tensorflow-estimator==2.14.0
tensorflow-hub==0.16.1
tensorflow-io-gcs-filesystem==0.37.0
tensorflow-metadata==1.15.0
tensorflow-text==2.14.0
tensorstore==0.1.59
termcolor==2.4.0
tf-keras==2.15.0
tfds-nightly==4.9.2.dev202308090034
threadpoolctl==3.1.0
tokenizers==0.15.2
toml==0.10.2
toolz==0.12.1
torch==1.13.0+cu117.with.pypi.cudnn
torchaudio==2.0.1
torchvision==0.14.0+cu117
tornado==6.2
tqdm==4.64.1
traitlets==5.9.0
transformers==4.39.1
triton==2.0.0
typeguard==2.13.3
typepy==1.3.2
typing_extensions==4.4.0
ujson==5.10.0
urllib3==2.2.1
wcwidth==0.2.6
Werkzeug==3.0.3
wheel==0.41.2
wrapt==1.14.1
zipp==3.19.0
zstandard==0.22.0