【Python连接数据库】Python连接Teradata数据库-ODBC方式(pyodbc包和teradata包)
1.安装Python
Python下载地址:Python-3.6.8.tar.xz
(1)前置安装包查看
rpm -qa |grep -i zlib rpm -qa |grep -i bzip2 rpm -qa |grep -i ncurses rpm -qa |grep -i readline rpm -qa |grep -i openssl rpm -qa |grep -i xz rpm -qa |grep -i sqlite rpm -qa |grep -i gdbm rpm -qa |grep -i tk rpm -qa |grep -i gcc #非常重要,而且单独安装非常麻烦需要1-2h
(2)安装Python
将安装软件上传到/software目录,执行如下语句安装
mkdir -p /usr/local/python3 cd /software xz -d Python-3.6.8.tar.xz tar -xvf Python-3.6.8.tar cd Python-3.6.8 ./configure --prefix=/usr/local/python3 make make install
(3)更新默认Python为新版本
ln -s /usr/local/python3/bin/python3.6 /usr/bin/python3 ln -s /usr/local/python3/bin/pip3.6 /usr/bin/pip3 mv /usr/bin/python /usr/bin/python_bak_2.6.9 mv /usr/bin/pip /usr/bin/pip_bak_2.6.9 ln -s /usr/local/python3/bin/python3.6 /usr/bin/python ln -s /usr/local/python3/bin/pip3.6 /usr/bin/pip
(4)验证安装
pip --version
python -v
2.使用ODBC+pyodbc连接TD数据库
TeraGSS和tdicu(ODBC依赖包在TTU中):TeradataToolsAndUtilitiesBase__linux_indep.16.20.10.00.tar
Teradata ODBC驱动下载地址:tdodbc1620__linux_indep.16.20.00.65-1.tar 其它版本下载地址: Windows AIX
2.1检查是否安装TeraGSS和tdicu
rpm -qa | grep -i GSS rpm -qa | grep -i tdicu
如果没有安装,则使用TTU安装上述组件(TeradataToolsAndUtilitiesBase__linux_indep.16.20.10.00.tar)。
特别说明:
除使用上述16版本TTU安装方式外,也可以使用15版本的ODBC驱动包中含有的TERAGSS和tdicu独立安装包进行安装(16版本ODBC驱动包不含独立安装包)。15版本的ODBC驱动下载地址为tdodbc__linux_indep.15.00.00.08-1.tar.gz
使用rpm -ivh 安装相应的rpm包即可。
rpm -ivh TeraGSS_linux_x64-15.00.06.05-1.noarch.rpm rpm -ivh tdicu-15.00.00.00-1.noarch.rpm
2.2 安装Teradata的ODBC驱动
将安装软件tdodbc1620__linux_indep.16.20.00.65-1.tar上传到/software目录,执行如下语句安装(如果odbc已经安装完毕则跳过如下安装步骤,或者卸载当前版本后安装新版本)
rpm -qa | grep -i odbc #验证是否安装tdodbc cd /software tar -xvf tdodbc1620__linux_indep.16.20.00.65-1.tar cd tdodbc1620/ rpm -ivh tdodbc1620-16.20.00.65-1.noarch.rpm --nodeps rpm -qa | grep -i tdodbc tdodbc-16.20.00.65-1 rpm -e tdodbc-16.20.00.65-1 #卸载tdodbc
ODBC安装完毕后,会在teradata安装目录/opt/teradata/client/16.20下出现两个目录odbc_32和odbc_64,分别为32位和64位驱动。
64位驱动路径(每个版本不同可查看odbc.ini中Driver):/opt/teradata/client/16.20/odbc_64/lib/tdataodbc_sb64.so 32位驱动路径(每个版本不同可查看odbc.ini中Driver):/opt/teradata/client/16.20/odbc_32/lib/tdataodbc_sb32.so 64位odbc.ini模板位置:/opt/teradata/client/16.20/odbc_64/odbc.ini 64位odbcinst.ini模板位置: /opt/teradata/client/16.20/odbc_64/odbcinst.ini
32位odbc.ini模板位置:/opt/teradata/client/16.20/odbc_32/odbc.ini
32位odbcinst.ini模板位置: /opt/teradata/client/16.20/odbc_32/odbcinst.ini
2.3 安装pyodbc
Python连接ODBC模块下载地址:pyodbc-4.0.26.tar.gz 其它版本下载地址:pyodbc 4.0.26
(1)将安装软件上传到/software目录,执行如下语句安装
cd /software tar -zxvf pyodbc-4.0.26.tar.gz cd pyodbc-4.0.26
(2)编辑安装配置
vi setup.py
# Other posix-like: Linux, Solaris, etc. # Python functions take a lot of 'char *' that really should be const. gcc complains about this *a lot* settings['extra_compile_args'].append('-Wno-write-strings') 改变为(红色字体16.20,根据odbc的版本不同而有所改动)(尽量在原有基础上去改动,不要两行一起复制粘贴,否则会报格式问题,注意Python是使用tab键控制层次) # Other posix-like: Linux, Solaris, etc. # Python functions take a lot of 'char *' that really should be const. gcc complains about this *a lot* settings['extra_compile_args']=['-Wno-write-strings','-I/opt/teradata/client/16.20/odbc_64/include','-DSQL_CP_ONE_PER_HENV=2'] settings['extra_link_args']=['-L/opt/teradata/client/16.20/odbc_64/lib']
(3)安装pyodbc
python setup.py build python setup.py install
(4)验证安装
#查看如下目录中有pyodbc目录则安装成功
[root@localhost] ll /usr/local/python3/lib/python3.6/site-packages
[root@localhost]$ python >>> import pyodbc >>> pyodbc.apilevel '2.0'
2.4 配置ODBC并使用pyodbc连接数据库
2.4.1 直接使用DRIVER的连接方式(不使用DSN,仅配置odbcinst.ini文件即可,无需配置odbc.ini)
connection=pyodbc.connect('DRIVER={Teradata Database ODBC Driver 16.20};DBCNAME=192.168.253.131;UID=dbc;PWD=dbc;QUIETMODE=YES;')
配置步骤如下:
(1)查看并拷贝odbcinst.ini模板到当前用户根目录,并改名称为隐藏文件.odbcinst.ini
updatedb #创建或更新slocate命令所必需的数据库文件,执行时间可能会很长 locate odbcinst.ini #查看odbcinst模板位置 cp /opt/teradata/client/16.20/odbc_64/odbcinst.ini ~/.odbcinst.ini
(2)odbcinst.ini配置样例
如下ODBC注册配置文件odbcinst.ini中可以看到,可以使用的Driver名称为Teradata Database ODBC Driver 16.20(可以根据实际情况自行改动此名称,连接数据库时DRIVER同步更改即可)。
[ODBC Drivers] Teradata Database ODBC Driver 16.20=Installed [Teradata Database ODBC Driver 16.20] Description=Teradata Database ODBC Driver 16.20 Driver=/opt/teradata/client/16.20/odbc_64/lib/tdataodbc_sb64.so # Note: Currently, Data Direct Driver Manager does not support Connection Pooling feature. CPTimeout=60
(3)连接数据库样例(test_pyodbc.py)
# import pyodbc module import pyodbc # disable connection pooling pyodbc.pooling = False # create connection connection=pyodbc.connect('DRIVER={Teradata Database ODBC Driver 16.20};DBCNAME=192.168.253.131;UID=dbc;PWD=dbc;QUIETMODE=YES;') # enable auto commit connection.autocommit = True; # python 3.x connect Teradata set charset(如果不设置此字符集,数据库连接无问题,但是执行SQL会无法执行) connection.setdecoding(pyodbc.SQL_CHAR, encoding='utf-8') connection.setdecoding(pyodbc.SQL_WCHAR, encoding='utf-8') #connection.setdecoding(pyodbc.SQL_WMETADATA, encoding='utf-32le') connection.setencoding(encoding='utf-8') # print driver and database info print ('-ODBC version =',connection.getinfo(10)) print ('-DBMS name =',connection.getinfo(17)) print ('-DBMS version =',connection.getinfo(18)) print ('-Driver name =',connection.getinfo(6)) print ('-Driver version =',connection.getinfo(7)) print ('-Driver ODBC version =',connection.getinfo(77)) #disconnect connection.close()
(4)执行结果
2.4.2 使用DSN数据源配置的连接方式(仅配置odbc.ini文件即可,无需配置odbcinst.ini)
connection=pyodbc.connect('DSN=testdsn;UID=dbc;PWD=dbc;QUIETMODE=YES;') #DSN中没有配置了连接数据库用户名和口令
或
connection=pyodbc.connect('DSN=testdsn;QUIETMODE=YES;') #DSN中配置了连接数据库用户名和口令
配置步骤如下:
(1)查看并拷贝odbc.ini模板到当前用户根目录,并改名称为隐藏文件.odbc.ini
updatedb #创建或更新slocate命令所必需的数据库文件,执行时间可能会很长 locate odbc.ini #查看odbc模板位置 cp /opt/teradata/client/16.20/odbc_64/odbc.ini ~/.odbc.ini
(2)odbc.ini配置样例
如下ODBC的DSN文件odbc.ini中可以看到,可以使用的DSN名称为testdsn(可以根据实际情况自行改动此名称,连接数据库时DSN值同步更改即可)。
[ODBC] # For Data Direct to load its error messages # Data Direct Driver Manager looks for the messages here: # "/opt/teradata/client/16.20/locale/xx_xx/LC_MESSAGES/" InstallDir=/opt/teradata/client/16.20/odbc_64 Trace=0 TraceDll=/opt/teradata/client/16.20/odbc_64/lib/odbctrac.so TraceFile=/usr/joe/odbcusr/trace.log TraceAutoStop=0 [ODBC Data Sources] testdsn=tdata.so [testdsn] # This key is not necessary and is only to give a description of the data source. Description=Teradata Database ODBC Driver 16.20 # Driver: The location where the ODBC driver is installed to. Driver=/opt/teradata/client/16.20/odbc_64/lib/tdataodbc_sb64.so # Required: These values can also be specified in the connection string. DBCName=192.168.253.131 UID= PWD= CharacterSet=UTF8 # Optional AccountString= DatasourceDNSEntries= DateTimeFormat=AAA DefaultDatabase= DontUseHelpDatabase=0 DontUseTitles=1 EnableExtendedStmtInfo=1 EnableReadAhead=1 IgnoreODBCSearchPattern=0 LogErrorEvents=0 LoginTimeout=20 MaxRespSize=65536 MaxSingleLOBBytes=4000 MaxTotalLOBBytesPerRow=65536 MechanismName= NoScan=0 PrintOption=N retryOnEINTR=1 ReturnGeneratedKeys=N SessionMode=System Default SplOption=Y TABLEQUALIFIER=0 TCPNoDelay=1 TdmstPortNumber=1025 UPTMode=Not set USE2XAPPCUSTOMCATALOGMODE=0 UseDataEncryption=0 UseDateDataForTimeStampParams=0 USEINTEGRATEDSECURITY=0 UseSequentialRetrievalOnly=0 UseXViews=0 EnableUDFUpload=0 UDFUploadPath=
(3)连接数据库样例(test_pyodbc_dsn.py)
# import pyodbc module import pyodbc # disable connection pooling pyodbc.pooling = False # create connection #connection=pyodbc.connect('DSN=testdsn;UID=dbc;PWD=dbc;QUIETMODE=YES;') #DSN中未设定UID and PWD connection=pyodbc.connect('DSN=testdsn;QUIETMODE=YES;') #DSN中设定了UID and PWD # enable auto commit connection.autocommit = True; # python 3.x connect Teradata set charset(如果不设置此字符集,数据库连接无问题,但是执行SQL会无法执行) connection.setdecoding(pyodbc.SQL_CHAR, encoding='utf-8') connection.setdecoding(pyodbc.SQL_WCHAR, encoding='utf-8') #connection.setdecoding(pyodbc.SQL_WMETADATA, encoding='utf-32le') connection.setencoding(encoding='utf-8') # print driver and database info print ('-ODBC version =',connection.getinfo(10)) print ('-DBMS name =',connection.getinfo(17)) print ('-DBMS version =',connection.getinfo(18)) print ('-Driver name =',connection.getinfo(6)) print ('-Driver version =',connection.getinfo(7)) print ('-Driver ODBC version =',connection.getinfo(77)) #disconnect connection.close()
(4)执行结果
2.5 使用pyodbc执行SQL
2.5.1操作流程
获取连接对象==>创建游标==>执行SQL==>获取结果==>关闭游标==>关闭数据库连接
# import pyodbc module import pyodbc # disable connection pooling pyodbc.pooling = False # create connection #connection=pyodbc.connect('DRIVER={Teradata Database ODBC Driver 16.20};DBCNAME=192.168.253.131;UID=dbc;PWD=dbc;QUIETMODE=YES;') #使用DRIVER方式连接 #connection=pyodbc.connect('DSN=testdsn;UID=dbc;PWD=dbc;QUIETMODE=YES;') #使用DSN方式连接,DSN中未设定UID and PWD connection=pyodbc.connect('DSN=testdsn;QUIETMODE=YES;') #使用DSN方式连接,DSN中设定了UID and PWD # enable auto commit connection.autocommit = True; # python 3.x connect Teradata set charset(如果不设置此字符集,数据库连接无问题,但是执行SQL会无法执行) connection.setdecoding(pyodbc.SQL_CHAR, encoding='utf-8') connection.setdecoding(pyodbc.SQL_WCHAR, encoding='utf-8') #connection.setdecoding(pyodbc.SQL_WMETADATA, encoding='utf-32le') connection.setencoding(encoding='utf-8') # create cursor cursor = connection.cursor() # fetch result cursor.execute("select * from dbc.dbcinfov") rows=cursor.fetchall() for row in rows: print(row) # close cursor cursor.close() #disconnect connection.close()
2.5.2常用操作函数
(1)连接对象
close() 关闭数据库连接
commit() 提交事务
rollback() 回滚事务
cursor() 获取游标对象,操作数据库,如执行DML操作,调用存储过程等
(2)游标对象
close() 关闭当前游标
execute("SQL") 执行数据库操作,SQL语句或者数据库命令,其中SQL为执行语句
fetchone() 获取查询结果集中下一条记录
fetchmany(n) 获取结果集指定数量n条记录
fetchall() 获取结果集所有记录
2.5.3 经典实例
在数据库testbase中,创建数据库表books,插入数据后,查询插入数据。
# import pyodbc module import sys import pyodbc print(sys.getdefaultencoding()) # disable connection pooling pyodbc.pooling = False # create connection #connection=pyodbc.connect('DRIVER={Teradata Database ODBC Driver 16.20};DBCNAME=192.168.253.131;UID=dbc;PWD=dbc;QUIETMODE=YES;') #使用DRIVER方式连接 #connection=pyodbc.connect('DSN=testdsn;UID=dbc;PWD=dbc;QUIETMODE=YES;') #使用DSN方式连接,DSN中未设定UID and PWD connection=pyodbc.connect('DSN=testdsn;QUIETMODE=YES;') #使用DSN方式连接,DSN中设定了UID and PWD # enable auto commit connection.autocommit = True; # python 3.x connect Teradata set charset(如果不设置此字符集,数据库连接无问题,但是执行SQL会无法执行) connection.setdecoding(pyodbc.SQL_CHAR, encoding='utf-8') connection.setdecoding(pyodbc.SQL_WCHAR, encoding='utf-8') #connection.setdecoding(pyodbc.SQL_WMETADATA, encoding='utf-32le') connection.setencoding(encoding='utf-8') # print driver and database info print ('-ODBC version =',connection.getinfo(10)) print ('-DBMS name =',connection.getinfo(17)) print ('-DBMS version =',connection.getinfo(18)) print ('-Driver name =',connection.getinfo(6)) print ('-Driver version =',connection.getinfo(7)) print ('-Driver ODBC version =',connection.getinfo(77)) # create cursor cursor = connection.cursor() # execute SQL statement (create table books) cursor.execute("DROP TABLE testbase.books") cursor.execute("CREATE MULTISET TABLE testbase.books (book_name VARCHAR(50) NOT NULL, category VARCHAR(50) NOT NULL, price INTEGER,publish_time date NOT NULL)") try: #execute SQL, instert records cursor.execute("insert into testbase.books(book_name,category,price,publish_time) values('Python3.6.8','Python','79.80','2018-05-20')") # commit data connection.commit() except: #rollback when error connection.rollback() print("insert error data rollback") cursor.execute("select book_name,category from testbase.books") # fetch result set rows for row in cursor: print(row) # close cursor cursor.close() #disconnect connection.close()
3.使用ODBC+Teradata Python Module连接TD数据库(已停止更新,不建议使用)
teradata python模块包括两个子模块,它们实现了Python Database API规范v2.0,一个使用rest(teradata.tdrest),另一个使用odbc(teradata.tdbc)。尽管这些模块可直接被访问,但建议使用基本的UdaExec模块,因为它提供了所有额外的支持DevOps的功能。
(1)下载安装Teradata Python Module
下载地址:teradata-15.10.0.21.tar.gz 其它版本
将下载的安装包teradata-15.10.0.21.tar.gz拷贝到/software目录下。
#安装 cd /software tar -zxvf teradata-15.10.0.21.tar.gz cd teradata-15.10.0.21 setup.py install
#验证 ll /usr/local/python3/lib/python3.6/site-packages/ 查看是否有teradata-15.10.0.21-py3.6.egg目录,有则说明安装成功。
(2)使用Teradata Python Module连接数据库
test_tdmodule.py脚本如下(需配置好odbcinst.ini,配置方法详见上文):
#!/usr/bin/python # encoding=utf-8 import teradata udaExec = teradata.UdaExec (appName="HelloWorld", version="1.0",logConsole=False) session = udaExec.connect(method="odbc", system="192.168.253.131",username="dbc", password="dbc"); for row in session.execute("SEL * FROM DBC.dbcinfov"): print(row)
执行结果如下:
参考文档(Python安装)
参考文档(pyodbc包连接Teradata数据库):
参考文档(teradata包连接Teradata数据库):