Whl自助搜索下载器
本文转载自以下链接：https://github.com/Light-City/AutoDownloadWhl
源码地址：
本次脚本我将它命名为：Whl自助搜索下载器
实现了如下功能：
- 支持检索
- 支持查找
- 支持下载
- 单、多文件下载
2.如何使用及功能演示
2.1 如何使用
运行脚本需要：
- requests、lxml、os库
- curl或axel命令
win系统的直接下载win版本的按照网上配置即可！
linux系统与mac系统可用yum 或 apt 或 brew等来进行安装！
复制源码地址并下载脚本，运行脚本即可：
python AutoDownloadWhl.py
2.2 功能演示
- 所有包信息展示
- 适应平台whl包
- 多包下载
3.代码详解
3.1 封装
封装包地址与下载地址：
class AutoDownWhl():
def __init__(self):
# 包地址
self.url = 'https://www.lfd.uci.edu/~gohlke/pythonlibs/'
# 下载地址
self.base_url = 'https://download.lfd.uci.edu/pythonlibs/r5uhg2lo/'
# 模拟浏览器
self.headers = {
'User-Agent': 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/72.0.3626.96 Safari/537.36'
}
3.2 获取选择器与包名
解析源码与获取包名：
# 获取选择器与包名
def getLib(self):
raw_text = requests.get(self.url, headers=self.headers).content
seletor = etree.HTML(raw_text)
lib_names = seletor.xpath('//ul[@class="pylibs"]//li//strong//text()')
return seletor, lib_names
3.3 输出所有包名
一行15个包名打印：
# 输出所有包名
def print_AllLib(self, lib_names):
print('\n------------------共查找到' + str(len(lib_names)) + '个包------------------\n')
for i in range(len(lib_names)):
if i and i % 15 == 0:
print('\n')
print(lib_names[i], end=' ')
3.4 搜索与下载
只摘取了核心代码，下载采用调用系统命令curl与axel，请在自己的系统上安装这两个其中之一下载！
def searchDown(self, downloadMethod='axel'):
# 此部分代码只摘取核心部分
if int(lib_sequence) <= max:
detail_name = detail_libs[int(lib_sequence) - 1].strip().replace('‑', '-')
download_url = self.base_url + detail_name
print('\n\n输入包名正确,下载地址：' + download_url + '\n')
print("---------------------------------------开始下载---------------------------------------\n")
if downloadMethod == 'curl':
cmd = 'curl -O %s' % download_url
else:
cmd = 'axel %s' % download_url
while True:
cmd_res = os.system(cmd)
if cmd_res == 0:
print('下载成功!\n')
break
else:
print('下载失败!\n')
is_try = input('是否重试下载(y|n)： ')
if is_try == 'y' or is_try == 'Y':
continue
else:
break
else:
print("\n \033[1;31m 输入序号不符合要求,请重新输入！ \033[0m \n")
3.5 调用
采用axel下载：
dw = AutoDownWhl()
dw.searchDown(downloadMethod='axel')
实际运行效果显示：
因为下载使用的windows系统版本的axel命令无法访问https，
(使用axel命令下载报错，错误原因是Unable to connect to server download.lfd.uci.edu:80。使用浏览器直接访问download.lfd.uci.edu报错，但是通过查看包的下载地址，地址开头是https，使用https://www.lfd.uci.edu/就能访问该网站，而http://www.lfd.uci.edu/却无法访问。)
所以使用curl命令，修改最后一行：dw.searchDown(downloadMethod='curl') (前提是需要把curl命令添加到系统环境变量里)
执行python AutoDownloadWhl.py
首先会列出当前搜索到的所有whl包，接下来输入下载包的名字(英文逗号分割)，然后会按照顺序，挨个查找下载包，根据python版本的不同，输入需要下载的包序号，会出现该包的下载地址。
然后curl就开始下载包了，下载的包会保存到当前目录。
或者只获取包的下载地址，使用迅雷下载
完整的下载演示如下所示：
C:\Users\sandu\Anaconda3\envs\Network\python.exe D:/scrapyseleniumtest/AutoDownloadWhl.py
---------------------------------------欢迎使用Whl自助搜索下载器---------------------------------------
------------------共查找到476个包------------------
AD3 Aggdraw Aiohttp Akima APSW Arctic Aspell-python Assimulo Astropy Atom AutoPy Babel Backports.lzma Basemap Bazaar
Bcolz Bigfloat Bintrees Bio_formats Biopython Bitarray Blaze Blist Blosc Blz Boost.Python Bottleneck Brotli brotlipy Bsddb3
Bsdiff4 BTrees Cairocffi Cantera Carray Cartopy Casuarius cChardet Cdecimal CellCognition CellProfiler Centrosome ceODBC Cf-units Cffi
Cftime CGAL-bindings Cgkit Chaco Chebyfit Cheetah Chompack CLD Cmapfile cmarkgfm COBRA Coverage Crc16 Crcmod Cupy
Curses CVXcanon CVXOPT CVXPY cx_Freeze cx_Oracle Cyassimp Cyordereddict Cyrasterize Cython Cytoolz Czifile Daal4py Datrie Debug-information-files
Delny Dipy Discretize Dnacurve Dulwich DyND ECOS Enable Enaml Entropy ETS FabIO Fann2 Fast-histogram Fastcache
Fastparquet Fastrlock fastText fastTSNE Faulthandler Fcsfiles Fdint Ffnet Fiona Fisher Fisx FLANN Fmkr Fpzip FreeImageDLL
FreeSASA Friture GDAL Genshi Gensim Gevent Glumpy GMPY GPy GR Grako Greenlet gRPCio GuiQwt Gvar
H5py HDDM Heatmap Hmmlearn HoloPy HTSeq HyperSpy Ilastik Imagecodecs ImageD11 Iminuit Imread Imreg Indexed_gzip Intbitset
IOCBio Iris Javabridge JCC JPype Jsonlib Jupyter Kapteyn Kivy Kiwisolver Kwant La Lazy_object_proxy Lazyflow Lfdfiles
LibLAS LIBLINEAR Libpython libSBML LIBSVM Libtfr Libxml-python Lightning Line_profiler Llist Llvmlite LLVMPy Logbook Louvain-igraph lp_solve
Lru_dict Lsqfit Lucam Lxml Lz4 Mahotas Marisa-trie MarkupSafe Mathutils Matplotlib Mayavi Menpo Mercurial MeshPy Milk
Minepy Mistune MKL-service MKL_fft MKL_random Mlpy mmLib MMTK Mod_wsgi ModernGL Molmass Mpi4py Mplcairo Msgpack Multidict
MultiNEAT Multiprocess mxBase MySQL-python Mysqlclient Natgrid NDimage NetCDF4 Netifaces Netpbmfile NIPY NiTime NLopt nMOLDYN Noise
Numba Numcodecs Numexpr NumPy Numpy-quaternion numpy-stl ObsPy ODE Oiffile Open3D OpenBabel OpenCV OpenEXR OpenImageIO OpenPIV
openTSNE Orange Osqp Oursql Pandas Param Peewee Pendulum Persistent Pgmagick Pillow Pillow-SIMD Pip Planar Pocketsphinx
Polygon Polylearn Polymode PostgresAdapter Protobuf Psf Psutil Psycopg PuLP Py-earth Py-fcm Py-lmdb Py2exe Py_gd PyAlembic
PyAMF PyAMG PyArrow Pyasn PyAudio PyBluez PyBox2D Pycairo Pycares PyCifRW PyCLD2 PyCluster PyCogent PyCorrFit Pycosat
PyCryptoSat PyCUDA PycURL Pycvodes PyDbg PyDDE PyEDA PyEMD PyEphem Pyexiv2 PyFFTW PyFITS pyFLTK PyFlux pyFM
PyFMI Pygame Pygit2 PyGraphviz PyGreSQL PyGTK PyHDF PyHook Pyhull PyICU PyISAPIe PyJNIus Pykinsol PyKSVD Pyldap
PyLibDeconv PyLibTiff PyLZMA Pymatgen PyMC PyMca PyMCMC PyMCubes PyMedia PyMetis PyMinuit PyMix PyMOL Pymongo Pymssql
Pymunk Pymutt PyMVPA PyODBC Pyodeint PyOpenCL PyOpenGL Pypmc Pyproj PyQPBO PyQt4 PyQwt Pyropes Pyrsistent pyRXP
PySFML PySide pySLALIB PySparse Pyspharm Pysqlite PyStemmer PyStruct Pyswisseph PyTables Python-cjson Python-geohash Python-igraph Python-ldap Python-Levenshtein
Python-lzf Python-LZO Python-snappy Python-sundials PythonMagick PythonNET Pytiff PyTorch PyTST PyUSB-FTDI PyViennaCL PyVRML97 PyWavelets PyWCS PyWin32
PyWinHook Pywinpty PyX PyXML PyYAML PyZMQ Pyzopfli Qdafile Qimage2ndarray Qt_graph_helpers QuantLib QuickFIX QuTiP Rapidjson Rasterio
Re2 Recordclass Regex ReportLab Rpy2 Rtmidi-python Rtree Ruamel.yaml Salientdetect Sasl Scandir ScientificPython Scikit-cycling Scikit-fmm Scikit-image
Scikit-learn Scikit-misc Scikit-umfpack Scikits.ann Scikits.audiolab Scikits.delaunay Scikits.hydroclimpy Scikits.odes Scikits.samplerate Scikits.scattpy Scikits.timeseries Scikits.vectorplot Scimath SciPy Scipy-cluster
SCS Sdtfile SendKeys Seqlearn Setproctitle SfePy Shapely SilverCity Sima Simpleaudio SimpleITK Simplejson SimpleParse Slycot Sounddevice
spaCy Sparsesvd Spectrum Spglib SQLAlchemy Statsmodels STEPS Stratify Swiglpk Sympy TA-Lib TatSu TensorFlow Thrift ThriftPy
Tifffile Tinyarray Tomopy Tornado Traits Transformations Triangle Trollius TWAINmodule Twisted typed_ast Uciwebauth UDUNITS Ujson Umysql
Veusz VideoCapture Vidsrc VIGRA ViTables VPython VTK Winrandom Wordcloud Wrapt Wrf_python wxPython x86cpu Xgboost Xxhash
XYlib-py Yappi Yarl Yt Zfec Zipline Zodbpickle Zope.interface Zs Zstd Misc
请输入下载包的名字(英文逗号分割): SQLAlchemy
---------查找SQLAlchemy成功,正在输出包的详细信息---------
(1)SQLAlchemy‑1.2.18‑cp27‑cp27m‑win32.whl
(2)SQLAlchemy‑1.2.18‑cp27‑cp27m‑win_amd64.whl
(3)SQLAlchemy‑1.2.18‑cp35‑cp35m‑win32.whl
(4)SQLAlchemy‑1.2.18‑cp35‑cp35m‑win_amd64.whl
(5)SQLAlchemy‑1.2.18‑cp36‑cp36m‑win32.whl
(6)SQLAlchemy‑1.2.18‑cp36‑cp36m‑win_amd64.whl
(7)SQLAlchemy‑1.2.18‑cp37‑cp37m‑win32.whl
(8)SQLAlchemy‑1.2.18‑cp37‑cp37m‑win_amd64.whl
请输入下载的包名序号： 8
输入包名正确,下载地址：https://download.lfd.uci.edu/pythonlibs/r5uhg2lo/SQLAlchemy-1.2.18-cp37-cp37m-win_amd64.whl
---------------------------------------开始下载---------------------------------------
curl -O https://download.lfd.uci.edu/pythonlibs/r5uhg2lo/SQLAlchemy-1.2.18-cp37-cp37m-win_amd64.whl
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 1087k 100 1087k 0 0 39631 0 0:00:28 0:00:28 --:--:-- 45299
下载成功!
是否继续下载其他库(y|n)： n
---------------------------------------感谢使用---------------------------------------
修改后使用curl命令下载的代码地址：
https://files.cnblogs.com/files/sanduzxcvbnm/AutoDownloadWhl-master.7z
