第十三章：Python の 网络编程进阶(二)

本課主題

  • SQLAlchemy - Core
  • SQLAlchemy - ORM
  • Paramiko 介紹和操作
  • 上下文操作应用
  • 初探堡垒机

 

SQLAlchemy - Core

连接 URL

通过 create_engine 方法创建 MySQL 数据库的连接，create_engine("url") 接受一个 URL 连接:

 >>> MySQL-Python: mysql+mysqldb://<user>:<password>@<host>[:<port>]/<dbname>
 >>> pymysql: mysql+pymysql://<username>:<password>@<host>/<dbname>[?<options>]
 >>> MySQL-Connector: mysql+mysqlconnector://<user>:<password>@<host>[:<port>]/<dbname>
 >>> cx_Oracle: oracle+cx_oracle://user:pass@host:port/dbname[?key=value&key=value...]
 >>> PostgreSQL: postgresql+pg8000://<user>:<password>@<host>/<db>

Details: http://docs.sqlalchemy.org/en/latest/dialects/index.html
create_engine("url")

用 SQLAlchemy 来创建数据库表

from datetime import datetime

# ------------------------------------------------------------
# Import the datatype
# ------------------------------------------------------------
from sqlalchemy import (
    MetaData, Table, Column, Integer, Numeric, String, Float, Boolean,
    DateTime, ForeignKey, create_engine
)
metadata = MetaData()

# ------------------------------------------------------------
# Table Objects
# ------------------------------------------------------------
# 创建 cookies 的数据库表
cookies = Table('cookies', metadata,
    Column('cookie_id', Integer(), primary_key=True),
    Column('cookie_name', String(50), index=True), #创建索引
    Column('cookie_recipe_url', String(255)),
    Column('cookie_sku', String(55)),
    Column('quantity', Integer()),
    Column('unit_cost', Numeric(12,2))

    # Index('ix_cookies_cookie_name', 'cookie_name') #也可以调用这个方法来创建索引
)

# 创建 users 的数据库表
user = Table('users', metadata,
    Column('user_id', Integer(), primary_key=True),
    Column('username', String(15), nullable=False, unique=True),
    Column('email_address', String(255), nullable=False),
    Column('phone', String(20), nullable=False),
    Column('password', String(25), nullable=False),
    Column('created_on', DateTime(), default=datetime.now),
    Column('updated_on', DateTime(), default=datetime.now, onupdate=datetime.now)

    # PrimaryKeyConstraint('user_id', name='user_pk'), #也可以调用这个方法来创建主键
    # UniqueConstraint('username', name='uix_username'), #也可以调用这个方法来创建唯一
    # CheckConstraint('unit_cost >= 0.00', name='unit_cost_positive')
)

# 创建 orders 的数据库表
orders = Table('orders', metadata,
    Column('order_id', Integer(), primary_key=True),
    Column('user_id', ForeignKey('users.user_id')),
    Column('shipped', Boolean(), default=False)
)

# 创建 line_items 的数据库表
line_items = Table('line_items', metadata,
    Column('line_items_id', Integer(), primary_key=True),
    Column('order_id', ForeignKey('orders.order_id')),
    Column('cookie_id', ForeignKey('cookies.cookie_id')),
    Column('quantity', Integer()),
    Column('extended_cost', Numeric(12,2))
)

# ------------------------------------------------------------
# 连接url
# ------------------------------------------------------------
engine = create_engine("mysql+pymysql://root@localhost/demo_db", pool_recycle=3600)

# ------------------------------------------------------------
# 调用 create_all() 方法来创建所有数据库表
# 默认该方法不会对已经存在的数据库表重新创建。
# ------------------------------------------------------------
metadata.create_all(engine)
metadata.create_all(engine)

   

 

SQLAlchemy - ORM

这是一个叫 Object Relational Mapping 框架，可以让我们通过类和对象来操作数据库，具体功能包括创建表，定义数据类型，新增或者查询，一般 MySQL 能做的功能，都可以在 SQLALchemy 里实现，我也是用上一章的那个数据模型去介紹如何用 SQLALchemy 的API去操作数据库。

一对多例子：

从人的角度看：每个人只可以选择一种颜色（一对一）；从颜色的角度看：但是一种颜色可以有多个人选择（一对多）

多对多例子：

从组的角度看：每个组可以有不同的服务器（一对多）；从服务器的角度看：每个服务器也可以属于不同的组（一对多）

表操作

  1. 创建连接
     
    engine = create_engine('mysql+pymysql://myuser:mypass@192.168.80.128:3306/s13', max_overflow = 5)
    创建 MySQL连接
  2. 创建表
     
    Base.metadata.create_all(engine)
    Base.metadata.create_all( )
    #!/usr/bin/env python
# -*- coding:utf-8 -*-

from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy import Column, Integer, String, ForeignKey, UniqueConstraint, Index
from sqlalchemy.orm import sessionmaker, relationship
from sqlalchemy import create_engine

engine = create_engine('mysql+pymysql://myuser:mypass@192.168.80.128:3306/s13', max_overflow = 5) # 创建连接
Base = declarative_base()


# Many-to-many
class Servers_to_Groups(Base):
    __tablename__ = 'rel_servers_groups'
    nid = Column(Integer, nullable=False, primary_key=True)
    server_id = Column(Integer, ForeignKey('servers.id'))
    group_id = Column(Integer, ForeignKey('groups.id'))

    # 在 sqlalchemy 支持创建 relationship，方便查询
    groups = relationship('Groups', backref = 'lkp_servers') # 在groups表的內部, sqlalchemy 會多創建一個隱性字段名叫 lkp_servers
    servers = relationship('Servers', backref = 'lkp_groups') # 在servers表的內部, sqlalchemy 會多創建一個隱性字段名叫 lkp_groups


class Groups(Base):
    __tablename__ = 'groups'
    id = Column(Integer, nullable=False, primary_key=True)
    name = Column(String(50), unique=True, nullable=False)
    port = Column(Integer, default=22)


class Servers(Base):
    __tablename__ = 'servers'
    id = Column(Integer, nullable=False, primary_key=True)
    name = Column(String(64), unique=True, nullable=False)
    sqlalchemy创建表(多对多)
    #!/usr/bin/env python
# -*- coding:utf-8 -*-

from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy import Column, Integer, String, ForeignKey, UniqueConstraint, Index
from sqlalchemy.orm import sessionmaker, relationship
from sqlalchemy import create_engine

engine = create_engine('mysql+pymysql://myuser:mypass@192.168.80.128:3306/s13', max_overflow = 5) # 创建连接
Base = declarative_base()


class Users(Base):
    __tablename__ = 'users'
    id = Column(Integer, nullable=False, primary_key=True)
    name = Column(String(32))
    extra = Column(String(16))

    __table_args__ = (
        UniqueConstraint('id','name', name = 'unix_id_name'),
        Index('ix_id_name','name','extra')
    )


# One-to-many
class Favor(Base):
    __tablename__ = 'favor'
    nid = Column(Integer, primary_key=True)
    caption = Column(String(32), default='red', unique=True)

    #当打印这个类时,会使用以下方法
    def __repr__(self):
        return "%s-%s" %(self.nid, self.caption)


class Person(Base):
    __tablename__ = 'person'
    nid = Column(Integer, primary_key=True)
    name = Column(String(32), index=True, nullable=True)
    favor_id = Column(Integer, ForeignKey('favor.nid'))

    favor = relationship('favor', backref='lkp_person')  # 在favor表的內部,sqlalchemy會多創建一個隱性字段名叫lkp_person
    sqlalchemy创建表(一对多)
  3. 删除表 DROP TABLE
     
    Base.metadata.drop_all(engine)
    Base.metadata.drop_all( )
  4. 定义自动增量 AUTO INCREMEN／主键 PRIMARY KEY
     
    sid = Column(Integer, primary_key=True, autoincrement=True)
    autoincrement,primary_key
  5. 定义外键 FOREIGN KEY
     
    person_sid = Column(Integer, ForeignKey("dm_person.sid"),nullable=False)
    ForeignKey( )
  6. 定义关系 relationship
     
    class Favor(Base):
    __tablename__ = 'favor'
    nid = Column(Integer, primary_key=True)
    caption = Column(String(32), default='red', unique=True)

    #当打印这个类时,会使用以下方法
    def __repr__(self):
        return "%s-%s" %(self.nid, self.caption)


class Person(Base):
    __tablename__ = 'person'
    nid = Column(Integer, primary_key=True)
    name = Column(String(32), index=True, nullable=True)
    favor_id = Column(Integer, ForeignKey('favor.nid'))

    favor = relationship('favor', backref='lkp_person')
    relationship( )
  7. 定义约束 CONSTRAINT
     
    class Users(Base):
    __tablename__ = 'users'
    id = Column(Integer, nullable=False, primary_key=True)
    name = Column(String(32))
    extra = Column(String(16))

    __table_args__ = (
        UniqueConstraint('id','name', name = 'unix_id_name'),
        Index('ix_id_name','name','extra')
    )
    sqlalchemy中的约束
  8. ALTER TABLE
  9. WHERE
     
    # WHERE CLAUSE
record = session.query(Users).filter_by(name='janice').all()
record = session.query(Users).filter(Users.id > 1, Users.name == 'janice').all()
record = session.query(Users).filter(Users.id.between(1,3)).all()
record = session.query(Users).filter(Users.id.in_([1,3,4])).all()
record = session.query(Users).filter(~Users.id.in_([1,3,4])).all()
record = session.query(Users).filter(Users.id.in_(session.query(Users.id).filter_by(name='janice'))).all()
    session.query( ).filter( )
  10. ORDER BY
     
    record = session.query(Users).order_by(Users.name.desc()).all()
record = session.query(Users).order_by(Users.name.desc(), Users.id.asc()).all()

for res in record:
    print(res.name, res.extra)

"""
ken manager
janice engineer
alex director
"""
    session.query( ).order_by(User.name.desc())
  11. GROUP BY
     
    record = session.query(
    func.max(Users.id),
    func.min(Users.id),
    func.sum(Users.id)
).group_by(Users.name).having(func.min(Users.id) > 2).all()

for res in record:
    print(res)
    session.query().group_by()
  12. UNION/ UNION ALL
     
    q1 = session.query(Users.name).filter(Users.id > 2)
q2 = session.query(Favor.nid).filter(Favor.nid > 2)

# record = q1.union(q2)
record = q1.union_all(q2)

for res in record.all():
    print(res)
    q1.union()/union_all(q2)
  13. LIKE
     
    # record = session.query(Users).filter(Users.name.like('j%')).all()
record = session.query(Users).filter(~Users.name.like('j%')).all()

for res in record:
    print(res.name, res.extra)
    session.query().filter(User.name.like())
  14. LIMIT
     
    record = session.query(Users)[1:2]
    session.query()[1:2]
  15. INSERT
     
    session.add_all([
    Favor(caption='red'),
    Favor(caption='orange'),
    Favor(caption='yellow'),
    Favor(caption='green'),
    Favor(caption='blue'),
    Favor(caption='purple')
])

session.add_all([
    Person(name='janice',favor_id=4),
    Person(name='alex',favor_id=1),
    Person(name='kennith',favor_id=6),
    Person(name='peter',favor_id=3),
    Person(name='jennifer',favor_id=2),
    Person(name='winnie',favor_id=5)
])
    session.add()/add_all([])
  16. DELETE
     
    session.query(Users).filter(Users.id > 2).delete()
    session.query().delete()
  17. UPDATE
     
    session.query(Users).filter(Users.id > 2).update({"name":"updated_kennith"})
session.query(Users).filter(Users.id > 2).update({Users.name: Users.name + "_100"}, synchronize_session=False)
session.query(Users).filter(Users.id > 2).update({"id": Users.id + 100}, synchronize_session="evaluate")
    session.query().update()
  18. SELECT
     
    record = session.query(Users) #可以查看原生 SQL 語句
record = session.query(Users).all() # 返回的是Users object对象
record = session.query(Users.name, Users.extra).all() # 返回的是Users.name 和 Users.extra 的内容，因为Users.name, Users.extra作为参数传入了 query()
record = session.query(Users).filter_by(name='alex').all() # 返回的是一个可迭代的对象 e.g.for res in record; print(res.name)
record = session.query(Users).filter_by(name='alex').first() # 返回的是Users.name 和 Users.extra 的內容
    session.query().first()/all()
  19. INNER JOIN / LEFT OUTER JOIN
     
    record = session.query(Favor, Person).filter(Person.favor_id==Favor.nid).all()
for res in record:
    print(res[0], res[1].name)

record = session.query(Person.name,Favor.caption).join(Favor).all()
record = session.query(Person).join(Favor, isouter=True)
print(record)
    session.query().join()/.join(isouter=True)
  20. SUM/ MIN/ MAX
     
    record = session.query(
    func.max(Users.id),
    func.min(Users.id),
    func.sum(Users.id)
).group_by(Users.name).having(func.min(Users.id) > 2).all()

for res in record:
    print(res)

"""
(3, 3, Decimal('3'))
"""
    SUM/ MIN/ MAX

 

数据分析例子

现在我会用我上一章的那个数据模型来回答以下问题，这是一个记录了每个用户购买了那些货品的一张数据表，还有记录了购买了多少件数和每件货品的价格是多少。


第一步是在数据库上创建以上的表，分别是产品，人和销售表。产品表记录了一些产品名称和产品类型; 人表记录了购买人的名称和年龄；销售表完整的记录了谁买了什么、买了多少产品和产品的价格是多少、等等...现在会使用 Python 的 SQLAlchemy 去完成所有的工作,而不是用原生 SQL 去做。

  1. 第一步：创建产品，人和销售表的表
     
    from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy import Column, Integer, String, ForeignKey, UniqueConstraint, Index
from sqlalchemy.orm import sessionmaker, relationship
from sqlalchemy import create_engine

# 创建连接
engine = create_engine('mysql+pymysql://myuser:mypass@172.16.201.134:3306/s13?',
                       max_overflow = 5,
                       pool_recycle=3600)

Base = declarative_base()


class Person(Base):
    __tablename__ = 'dm_person'
    sid = Column(Integer, primary_key=True, autoincrement=True)
    name = Column(String(10), unique=True, nullable=False)
    age = Column(Integer, nullable=False)


class Product(Base):
    __tablename__ = 'dm_product'
    sid = Column(Integer, primary_key=True, autoincrement=True)
    product_name = Column(String(50), unique=True, nullable=False)
    product_category = Column(String(50), nullable=False)


class Sales(Base):
    __tablename__ = 'fct_sales'
    sid = Column(Integer, primary_key=True, autoincrement=True)
    person_sid = Column(Integer, ForeignKey("dm_person.sid"),nullable=False)
    product_sid = Column(Integer, ForeignKey("dm_product.sid"),nullable=False)
    unit_price = Column(Integer, nullable=True)
    qty = Column(Integer, nullable=True)
    
    person = relationship("Person","lkp_sales")
    product = relationship("Product","lkp_sales")
    创建表ORM模型
  2. 第二步：创建一个 session 来进行数据操作
     
    Session = sessionmaker(bind=engine)
session = Session()
# blablabla...
session.commit()
    Session = sessionmaker(bind=engine)
  3. 第三步：插入一些测试数据，此时,我会调用刚才学的 session.add_all( )方法
     
    def data_init():

    # 创建连接
    Session = sessionmaker(bind=engine)
    session = Session()

    # 初始化插入数据 - Person Table
    session.add_all([
        Person(name='janice', age=22),
        Person(name='alex', age=33),
        Person(name='ken', age=30),
        Person(name='peter', age=28),
        Person(name='david', age=23),
        Person(name='ziv', age=25),
        Person(name='ronald', age=21),
        Person(name='kenny', age=36)
    ])

    # 初始化插入数据 - Product Table
    session.add_all([
        Product(product_name='iPhone 6S', product_category='Electronic'),
        Product(product_name='iPhone 7', product_category='Electronic'),
        Product(product_name='XiaoMi 5', product_category='Electronic'),
        Product(product_name='Samsung Note 7', product_category='Electronic'),
        Product(product_name='Programming in Python', product_category='Books'),
        Product(product_name='Python In Action', product_category='Books'),
        Product(product_name='Shakespeare', product_category='Books'),
        Product(product_name='Coconut Water', product_category='Foods and Drinks'),
        Product(product_name='Coffe', product_category='Foods and Drinks'),
        Product(product_name='Bike', product_category='Automobile'),
        Product(product_name='Tesla Model X', product_category='Automobile')
    ])

    # SQL commit
    session.commit()


data_init()
    session.add( )/ add_all( )函数例子

数据准备好了，现在可以尝试回答问题啦！

  1. 问题一：我想知道这商店有什么产品，可以调用 session.query( ) 去查询 
    print("问题一：我想知道这商店有什么产品?")
Session = sessionmaker(bind=engine)
session = Session()

record = session.query(Product.product_name, Product.product_category).all()

print("\n答: 这商店有以下产品:")
for row in enumerate(record,1):
    print('{}. {}'.format(row[0],row[1][0]))

# record = session.query(Product).all()
# for r in enumerate(record, 1):
#     print('{}. {}'.format(r[0],r[1].product_name))

session.commit()

"""
问题一：我想知道这商店有什么产品?

答: 这商店有以下产品:
1. iPhone 6S
2. iPhone 7
3. XiaoMi 5
4. Samsung Note 7
5. Programming in Python
6. Python In Action
7. Shakespeare
8. Coconut Water
9. Coffe
10. Bike
11. Tesla Model X
"""
    session.query( )函数例子
  2. 问题二：我想知道这商店有什么"电子产品"可以卖 (提示：Electronic) 
     
    print("问题二：我想知道这商店有什么'电子产品'可以卖")
Session = sessionmaker(bind=engine)
session = Session()

# record = session.query(Product).filter_by(product_category = 'Electronic').all()
record = session.query(Product).filter(Product.product_category == 'Electronic').all()

print("\n答: 这商店有以下电子产品:")
for row in enumerate(record,1):
    print('{}. {}'.format(row[0],row[1].product_name))

session.commit()


"""
问题二：我想知道这商店有什么'电子产品'可以卖

答: 这商店有以下电子产品:
1. iPhone 6S
2. iPhone 7
3. XiaoMi 5
4. Samsung Note 7
"""
    session.query( ).filter( )/ filter_by( )函数例子
  3. 问题三：我想知道 janice 买了什么东西
     
    print("问题三：我想知道 janice 买了什么东西")
Session = sessionmaker(bind=engine)
session = Session()

#method1
record = session.query(Product.product_name).filter(Sales.person_sid==Person.sid, Sales.product_sid==Product.sid, Person.name == 'janice').all()

print("\n答: janice 买了以下东西:")
for row in enumerate(record,1):
    print('{}. {}'.format(row[0],row[1].product_name))

session.commit()

"""
问题三：我想知道 janice 买了什么东西

答: janice 买了以下东西:
1. Shakespeare
2. Coffe
3. Samsung Note 7
4. Tesla Model X
5. iPhone 7
6. XiaoMi 5
7. Coconut Water
8. Python In Action
"""
    表关联和filter( )函数例子
  4. 问题四：我想知道 janice 总共花费了多少钱
     
    print("问题四：我想知道 janice 总共花费了多少钱")
Session = sessionmaker(bind=engine)
session = Session()

from sqlalchemy.sql import func

record = session.query(
    func.sum(Sales.unit_price * Sales.qty)
).filter(Sales.person_sid==Person.sid,
         Sales.product_sid==Product.sid,
         Person.name == 'janice').group_by(Person.name).all()

print("\n答: janice 总共花费了 ${}".format(record[0][0]))

session.commit()    

"""
问题四：我想知道 janice 总共花费了多少钱
答: janice 总共花费了 $880595

"""
    表关联, sum(), group_by()和filter( )函数例子
  5. 问题五：我想知道每个用户总共花费了多少钱，以花费最多的排序
     
    print("问题五：我想知道每个用户总共花费了多少钱，以花费最多的排序")
Session = sessionmaker(bind=engine)
session = Session()

from sqlalchemy.sql import func

record = session.query(
    Person.name,
    func.sum(Sales.unit_price * Sales.qty)
).filter(Sales.person_sid==Person.sid,
         Sales.product_sid==Product.sid).group_by(Person.name).order_by(func.sum(Sales.unit_price * Sales.qty).desc()).all()

for row in enumerate(record,1):
    print('第{}名: {}，总共花费了 ${}元'.format(row[0],row[1][0],row[1][1]))

session.commit()

"""
问题五：我想知道每个用户总共花费了多少钱，以花费最多的排序

第1名: janice，总共花费了 $880595元
第2名: peter，总共花费了 $820101元
第3名: alex，总共花费了 $97684元
第4名: ken，总共花费了 $97126元
"""
    表关联, sum(), group_by(), order_by()和filter( )函数例子
  6. 问题六：我想知道谁买了 Tesla 又买了一个小米手机
     
    print("问题六：我想知道谁买了 Tesla 又买了一个小米手机")
Session = sessionmaker(bind=engine)
session = Session()

from sqlalchemy import and_, or_, distinct

record = session.query(distinct(Person.name,)).\
    filter(Sales.person_sid==Person.sid,Sales.product_sid==Product.sid).\
    filter(Product.product_name.in_(['Tesla Model X','XiaoMi 5'])).all()

print("买了 Tesla 又买了一个小米手机是:")
ret = [row[0] for row in record]
print(ret)

"""
问题六：我想知道谁买了 Tesla 又买了一个小米手机
买了 Tesla 又买了一个小米手机是:
['janice', 'peter', 'ken']
"""
    表关联,in_(), distinct()

 

Paramiko 介紹和操作

paramiko模块，基于SSH用于连接远程服务器并执行相关操作，先给大家有一个大概的概念，Paramiko 可支持以下两种登入方法和创建三种不同的对象来完成工作:

  1. 创建 SSHClient e.g. paramiko.SSHClient( ) - 用于连接远程服务器并执行基本命令
     
    class SSHClient (ClosingContextManager):
    """
    A high-level representation of a session with an SSH server.  This class
    wraps `.Transport`, `.Channel`, and `.SFTPClient` to take care of most
    aspects of authenticating and opening channels.  A typical use case is::

        client = SSHClient()
        client.load_system_host_keys()
        client.connect('ssh.example.com')
        stdin, stdout, stderr = client.exec_command('ls -l')

    You may pass in explicit overrides for authentication and server host key
    checking.  The default mechanism is to try to use local key files or an
    SSH agent (if one is running).

    Instances of this class may be used as context managers.

    .. versionadded:: 1.6
    """

    def __init__(self):
        """
        Create a new SSHClient.
        """
        self._system_host_keys = HostKeys()
        self._host_keys = HostKeys()
        self._host_keys_filename = None
        self._log_channel = None
        self._policy = RejectPolicy()
        self._transport = None
        self._agent = None

    def load_system_host_keys(self, filename=None):
        """
        Load host keys from a system (read-only) file.  Host keys read with
        this method will not be saved back by `save_host_keys`.

        This method can be called multiple times.  Each new set of host keys
        will be merged with the existing set (new replacing old if there are
        conflicts).

        If ``filename`` is left as ``None``, an attempt will be made to read
        keys from the user's local "known hosts" file, as used by OpenSSH,
        and no exception will be raised if the file can't be read.  This is
        probably only useful on posix.

        :param str filename: the filename to read, or ``None``

        :raises IOError:
            if a filename was provided and the file could not be read
        """
        if filename is None:
            # try the user's .ssh key file, and mask exceptions
            filename = os.path.expanduser('~/.ssh/known_hosts')
            try:
                self._system_host_keys.load(filename)
            except IOError:
                pass
            return
        self._system_host_keys.load(filename)

    def load_host_keys(self, filename):
        """
        Load host keys from a local host-key file.  Host keys read with this
        method will be checked after keys loaded via `load_system_host_keys`,
        but will be saved back by `save_host_keys` (so they can be modified).
        The missing host key policy `.AutoAddPolicy` adds keys to this set and
        saves them, when connecting to a previously-unknown server.

        This method can be called multiple times.  Each new set of host keys
        will be merged with the existing set (new replacing old if there are
        conflicts).  When automatically saving, the last hostname is used.

        :param str filename: the filename to read

        :raises IOError: if the filename could not be read
        """
        self._host_keys_filename = filename
        self._host_keys.load(filename)

    def save_host_keys(self, filename):
        """
        Save the host keys back to a file.  Only the host keys loaded with
        `load_host_keys` (plus any added directly) will be saved -- not any
        host keys loaded with `load_system_host_keys`.

        :param str filename: the filename to save to

        :raises IOError: if the file could not be written
        """

        # update local host keys from file (in case other SSH clients
        # have written to the known_hosts file meanwhile.
        if self._host_keys_filename is not None:
            self.load_host_keys(self._host_keys_filename)

        with open(filename, 'w') as f:
            for hostname, keys in self._host_keys.items():
                for keytype, key in keys.items():
                    f.write('%s %s %s\n' % (hostname, keytype, key.get_base64()))

    def get_host_keys(self):
        """
        Get the local `.HostKeys` object.  This can be used to examine the
        local host keys or change them.

        :return: the local host keys as a `.HostKeys` object.
        """
        return self._host_keys

    def set_log_channel(self, name):
        """
        Set the channel for logging.  The default is ``"paramiko.transport"``
        but it can be set to anything you want.

        :param str name: new channel name for logging
        """
        self._log_channel = name

    def set_missing_host_key_policy(self, policy):
        """
        Set policy to use when connecting to servers without a known host key.

        Specifically:

        * A **policy** is an instance of a "policy class", namely some subclass
          of `.MissingHostKeyPolicy` such as `.RejectPolicy` (the default),
          `.AutoAddPolicy`, `.WarningPolicy`, or a user-created subclass.

          .. note::
            This method takes class **instances**, not **classes** themselves.
            Thus it must be called as e.g.
            ``.set_missing_host_key_policy(WarningPolicy())`` and *not*
            ``.set_missing_host_key_policy(WarningPolicy)``.

        * A host key is **known** when it appears in the client object's cached
          host keys structures (those manipulated by `load_system_host_keys`
          and/or `load_host_keys`).

        :param .MissingHostKeyPolicy policy:
            the policy to use when receiving a host key from a
            previously-unknown server
        """
        self._policy = policy

    def _families_and_addresses(self, hostname, port):
        """
        Yield pairs of address families and addresses to try for connecting.

        :param str hostname: the server to connect to
        :param int port: the server port to connect to
        :returns: Yields an iterable of ``(family, address)`` tuples
        """
        guess = True
        addrinfos = socket.getaddrinfo(hostname, port, socket.AF_UNSPEC, socket.SOCK_STREAM)
        for (family, socktype, proto, canonname, sockaddr) in addrinfos:
            if socktype == socket.SOCK_STREAM:
                yield family, sockaddr
                guess = False

        # some OS like AIX don't indicate SOCK_STREAM support, so just guess. :(
        # We only do this if we did not get a single result marked as socktype == SOCK_STREAM.
        if guess:
            for family, _, _, _, sockaddr in addrinfos:
                yield family, sockaddr

    def connect(
        self,
        hostname,
        port=SSH_PORT,
        username=None,
        password=None,
        pkey=None,
        key_filename=None,
        timeout=None,
        allow_agent=True,
        look_for_keys=True,
        compress=False,
        sock=None,
        gss_auth=False,
        gss_kex=False,
        gss_deleg_creds=True,
        gss_host=None,
        banner_timeout=None
    ):
        """
        Connect to an SSH server and authenticate to it.  The server's host key
        is checked against the system host keys (see `load_system_host_keys`)
        and any local host keys (`load_host_keys`).  If the server's hostname
        is not found in either set of host keys, the missing host key policy
        is used (see `set_missing_host_key_policy`).  The default policy is
        to reject the key and raise an `.SSHException`.

        Authentication is attempted in the following order of priority:

            - The ``pkey`` or ``key_filename`` passed in (if any)
            - Any key we can find through an SSH agent
            - Any "id_rsa", "id_dsa" or "id_ecdsa" key discoverable in
              ``~/.ssh/``
            - Plain username/password auth, if a password was given

        If a private key requires a password to unlock it, and a password is
        passed in, that password will be used to attempt to unlock the key.

        :param str hostname: the server to connect to
        :param int port: the server port to connect to
        :param str username:
            the username to authenticate as (defaults to the current local
            username)
        :param str password:
            a password to use for authentication or for unlocking a private key
        :param .PKey pkey: an optional private key to use for authentication
        :param str key_filename:
            the filename, or list of filenames, of optional private key(s) to
            try for authentication
        :param float timeout:
            an optional timeout (in seconds) for the TCP connect
        :param bool allow_agent:
            set to False to disable connecting to the SSH agent
        :param bool look_for_keys:
            set to False to disable searching for discoverable private key
            files in ``~/.ssh/``
        :param bool compress: set to True to turn on compression
        :param socket sock:
            an open socket or socket-like object (such as a `.Channel`) to use
            for communication to the target host
        :param bool gss_auth:
            ``True`` if you want to use GSS-API authentication
        :param bool gss_kex:
            Perform GSS-API Key Exchange and user authentication
        :param bool gss_deleg_creds: Delegate GSS-API client credentials or not
        :param str gss_host:
            The targets name in the kerberos database. default: hostname
        :param float banner_timeout: an optional timeout (in seconds) to wait
            for the SSH banner to be presented.

        :raises BadHostKeyException: if the server's host key could not be
            verified
        :raises AuthenticationException: if authentication failed
        :raises SSHException: if there was any other error connecting or
            establishing an SSH session
        :raises socket.error: if a socket error occurred while connecting

        .. versionchanged:: 1.15
            Added the ``banner_timeout``, ``gss_auth``, ``gss_kex``,
            ``gss_deleg_creds`` and ``gss_host`` arguments.
        """
        if not sock:
            errors = {}
            # Try multiple possible address families (e.g. IPv4 vs IPv6)
            to_try = list(self._families_and_addresses(hostname, port))
            for af, addr in to_try:
                try:
                    sock = socket.socket(af, socket.SOCK_STREAM)
                    if timeout is not None:
                        try:
                            sock.settimeout(timeout)
                        except:
                            pass
                    retry_on_signal(lambda: sock.connect(addr))
                    # Break out of the loop on success
                    break
                except socket.error as e:
                    # Raise anything that isn't a straight up connection error
                    # (such as a resolution error)
                    if e.errno not in (ECONNREFUSED, EHOSTUNREACH):
                        raise
                    # Capture anything else so we know how the run looks once
                    # iteration is complete. Retain info about which attempt
                    # this was.
                    errors[addr] = e

            # Make sure we explode usefully if no address family attempts
            # succeeded. We've no way of knowing which error is the "right"
            # one, so we construct a hybrid exception containing all the real
            # ones, of a subclass that client code should still be watching for
            # (socket.error)
            if len(errors) == len(to_try):
                raise NoValidConnectionsError(errors)

        t = self._transport = Transport(sock, gss_kex=gss_kex, gss_deleg_creds=gss_deleg_creds)
        t.use_compression(compress=compress)
        if gss_kex and gss_host is None:
            t.set_gss_host(hostname)
        elif gss_kex and gss_host is not None:
            t.set_gss_host(gss_host)
        else:
            pass
        if self._log_channel is not None:
            t.set_log_channel(self._log_channel)
        if banner_timeout is not None:
            t.banner_timeout = banner_timeout
        t.start_client()
        ResourceManager.register(self, t)

        server_key = t.get_remote_server_key()
        keytype = server_key.get_name()

        if port == SSH_PORT:
            server_hostkey_name = hostname
        else:
            server_hostkey_name = "[%s]:%d" % (hostname, port)

        # If GSS-API Key Exchange is performed we are not required to check the
        # host key, because the host is authenticated via GSS-API / SSPI as
        # well as our client.
        if not self._transport.use_gss_kex:
            our_server_key = self._system_host_keys.get(server_hostkey_name,
                                                         {}).get(keytype, None)
            if our_server_key is None:
                our_server_key = self._host_keys.get(server_hostkey_name,
                                                     {}).get(keytype, None)
            if our_server_key is None:
                # will raise exception if the key is rejected; let that fall out
                self._policy.missing_host_key(self, server_hostkey_name,
                                              server_key)
                # if the callback returns, assume the key is ok
                our_server_key = server_key

            if server_key != our_server_key:
                raise BadHostKeyException(hostname, server_key, our_server_key)

        if username is None:
            username = getpass.getuser()

        if key_filename is None:
            key_filenames = []
        elif isinstance(key_filename, string_types):
            key_filenames = [key_filename]
        else:
            key_filenames = key_filename
        if gss_host is None:
            gss_host = hostname
        self._auth(username, password, pkey, key_filenames, allow_agent,
                   look_for_keys, gss_auth, gss_kex, gss_deleg_creds, gss_host)

    def close(self):
        """
        Close this SSHClient and its underlying `.Transport`.

        .. warning::
            Failure to do this may, in some situations, cause your Python
            interpreter to hang at shutdown (often due to race conditions).
            It's good practice to `close` your client objects anytime you're
            done using them, instead of relying on garbage collection.
        """
        if self._transport is None:
            return
        self._transport.close()
        self._transport = None

        if self._agent is not None:
            self._agent.close()
            self._agent = None

    def exec_command(self, command, bufsize=-1, timeout=None, get_pty=False):
        """
        Execute a command on the SSH server.  A new `.Channel` is opened and
        the requested command is executed.  The command's input and output
        streams are returned as Python ``file``-like objects representing
        stdin, stdout, and stderr.

        :param str command: the command to execute
        :param int bufsize:
            interpreted the same way as by the built-in ``file()`` function in
            Python
        :param int timeout:
            set command's channel timeout. See `Channel.settimeout`.settimeout
        :return:
            the stdin, stdout, and stderr of the executing command, as a
            3-tuple

        :raises SSHException: if the server fails to execute the command
        """
        chan = self._transport.open_session(timeout=timeout)
        if get_pty:
            chan.get_pty()
        chan.settimeout(timeout)
        chan.exec_command(command)
        stdin = chan.makefile('wb', bufsize)
        stdout = chan.makefile('r', bufsize)
        stderr = chan.makefile_stderr('r', bufsize)
        return stdin, stdout, stderr

    def invoke_shell(self, term='vt100', width=80, height=24, width_pixels=0,
                     height_pixels=0):
        """
        Start an interactive shell session on the SSH server.  A new `.Channel`
        is opened and connected to a pseudo-terminal using the requested
        terminal type and size.

        :param str term:
            the terminal type to emulate (for example, ``"vt100"``)
        :param int width: the width (in characters) of the terminal window
        :param int height: the height (in characters) of the terminal window
        :param int width_pixels: the width (in pixels) of the terminal window
        :param int height_pixels: the height (in pixels) of the terminal window
        :return: a new `.Channel` connected to the remote shell

        :raises SSHException: if the server fails to invoke a shell
        """
        chan = self._transport.open_session()
        chan.get_pty(term, width, height, width_pixels, height_pixels)
        chan.invoke_shell()
        return chan

    def open_sftp(self):
        """
        Open an SFTP session on the SSH server.

        :return: a new `.SFTPClient` session object
        """
        return self._transport.open_sftp_client()

    def get_transport(self):
        """
        Return the underlying `.Transport` object for this SSH connection.
        This can be used to perform lower-level tasks, like opening specific
        kinds of channels.

        :return: the `.Transport` for this connection
        """
        return self._transport

    def _auth(self, username, password, pkey, key_filenames, allow_agent,
              look_for_keys, gss_auth, gss_kex, gss_deleg_creds, gss_host):
        """
        Try, in order:

            - The key passed in, if one was passed in.
            - Any key we can find through an SSH agent (if allowed).
            - Any "id_rsa", "id_dsa" or "id_ecdsa" key discoverable in ~/.ssh/
              (if allowed).
            - Plain username/password auth, if a password was given.

        (The password might be needed to unlock a private key, or for
        two-factor authentication [for which it is required].)
        """
        saved_exception = None
        two_factor = False
        allowed_types = set()
        two_factor_types = set(['keyboard-interactive','password'])

        # If GSS-API support and GSS-PI Key Exchange was performed, we attempt
        # authentication with gssapi-keyex.
        if gss_kex and self._transport.gss_kex_used:
            try:
                self._transport.auth_gssapi_keyex(username)
                return
            except Exception as e:
                saved_exception = e

        # Try GSS-API authentication (gssapi-with-mic) only if GSS-API Key
        # Exchange is not performed, because if we use GSS-API for the key
        # exchange, there is already a fully established GSS-API context, so
        # why should we do that again?
        if gss_auth:
            try:
                self._transport.auth_gssapi_with_mic(username, gss_host,
                                                     gss_deleg_creds)
                return
            except Exception as e:
                saved_exception = e

        if pkey is not None:
            try:
                self._log(DEBUG, 'Trying SSH key %s' % hexlify(pkey.get_fingerprint()))
                allowed_types = set(self._transport.auth_publickey(username, pkey))
                two_factor = (allowed_types & two_factor_types)
                if not two_factor:
                    return
            except SSHException as e:
                saved_exception = e

        if not two_factor:
            for key_filename in key_filenames:
                for pkey_class in (RSAKey, DSSKey, ECDSAKey):
                    try:
                        key = pkey_class.from_private_key_file(key_filename, password)
                        self._log(DEBUG, 'Trying key %s from %s' % (hexlify(key.get_fingerprint()), key_filename))
                        allowed_types = set(self._transport.auth_publickey(username, key))
                        two_factor = (allowed_types & two_factor_types)
                        if not two_factor:
                            return
                        break
                    except SSHException as e:
                        saved_exception = e

        if not two_factor and allow_agent:
            if self._agent is None:
                self._agent = Agent()

            for key in self._agent.get_keys():
                try:
                    self._log(DEBUG, 'Trying SSH agent key %s' % hexlify(key.get_fingerprint()))
                    # for 2-factor auth a successfully auth'd key password will return an allowed 2fac auth method
                    allowed_types = set(self._transport.auth_publickey(username, key))
                    two_factor = (allowed_types & two_factor_types)
                    if not two_factor:
                        return
                    break
                except SSHException as e:
                    saved_exception = e

        if not two_factor:
            keyfiles = []
            rsa_key = os.path.expanduser('~/.ssh/id_rsa')
            dsa_key = os.path.expanduser('~/.ssh/id_dsa')
            ecdsa_key = os.path.expanduser('~/.ssh/id_ecdsa')
            if os.path.isfile(rsa_key):
                keyfiles.append((RSAKey, rsa_key))
            if os.path.isfile(dsa_key):
                keyfiles.append((DSSKey, dsa_key))
            if os.path.isfile(ecdsa_key):
                keyfiles.append((ECDSAKey, ecdsa_key))
            # look in ~/ssh/ for windows users:
            rsa_key = os.path.expanduser('~/ssh/id_rsa')
            dsa_key = os.path.expanduser('~/ssh/id_dsa')
            ecdsa_key = os.path.expanduser('~/ssh/id_ecdsa')
            if os.path.isfile(rsa_key):
                keyfiles.append((RSAKey, rsa_key))
            if os.path.isfile(dsa_key):
                keyfiles.append((DSSKey, dsa_key))
            if os.path.isfile(ecdsa_key):
                keyfiles.append((ECDSAKey, ecdsa_key))

            if not look_for_keys:
                keyfiles = []

            for pkey_class, filename in keyfiles:
                try:
                    key = pkey_class.from_private_key_file(filename, password)
                    self._log(DEBUG, 'Trying discovered key %s in %s' % (hexlify(key.get_fingerprint()), filename))
                    # for 2-factor auth a successfully auth'd key will result in ['password']
                    allowed_types = set(self._transport.auth_publickey(username, key))
                    two_factor = (allowed_types & two_factor_types)
                    if not two_factor:
                        return
                    break
                except (SSHException, IOError) as e:
                    saved_exception = e

        if password is not None:
            try:
                self._transport.auth_password(username, password)
                return
            except SSHException as e:
                saved_exception = e
        elif two_factor:
            try:
                self._transport.auth_interactive_dumb(username)
                return
            except SSHException as e:
                saved_exception = e

        # if we got an auth-failed exception earlier, re-raise it
        if saved_exception is not None:
            raise saved_exception
        raise SSHException('No authentication methods available')

    def _log(self, level, msg):
        self._transport._log(level, msg)
    class SSHClient( )源码
    • 基于用户名密码连接
    • 基于公钥密钥连接
  2. 创建 SFTPClient e.g. paramiko.SFTPClient.from_transport(transport) - 用于连接远程服务器并执行上传下载
     
    class SFTPClient(BaseSFTP, ClosingContextManager):
    """
    SFTP client object.

    Used to open an SFTP session across an open SSH `.Transport` and perform
    remote file operations.

    Instances of this class may be used as context managers.
    """
    def __init__(self, sock):
        """
        Create an SFTP client from an existing `.Channel`.  The channel
        should already have requested the ``"sftp"`` subsystem.

        An alternate way to create an SFTP client context is by using
        `from_transport`.

        :param .Channel sock: an open `.Channel` using the ``"sftp"`` subsystem

        :raises SSHException: if there's an exception while negotiating
            sftp
        """
        BaseSFTP.__init__(self)
        self.sock = sock
        self.ultra_debug = False
        self.request_number = 1
        # lock for request_number
        self._lock = threading.Lock()
        self._cwd = None
        # request # -> SFTPFile
        self._expecting = weakref.WeakValueDictionary()
        if type(sock) is Channel:
            # override default logger
            transport = self.sock.get_transport()
            self.logger = util.get_logger(transport.get_log_channel() + '.sftp')
            self.ultra_debug = transport.get_hexdump()
        try:
            server_version = self._send_version()
        except EOFError:
            raise SSHException('EOF during negotiation')
        self._log(INFO, 'Opened sftp connection (server version %d)' % server_version)

    @classmethod
    def from_transport(cls, t, window_size=None, max_packet_size=None):
        """
        Create an SFTP client channel from an open `.Transport`.

        Setting the window and packet sizes might affect the transfer speed.
        The default settings in the `.Transport` class are the same as in
        OpenSSH and should work adequately for both files transfers and
        interactive sessions.

        :param .Transport t: an open `.Transport` which is already authenticated
        :param int window_size:
            optional window size for the `.SFTPClient` session.
        :param int max_packet_size:
            optional max packet size for the `.SFTPClient` session..

        :return:
            a new `.SFTPClient` object, referring to an sftp session (channel)
            across the transport

        .. versionchanged:: 1.15
            Added the ``window_size`` and ``max_packet_size`` arguments.
        """
        chan = t.open_session(window_size=window_size,
                              max_packet_size=max_packet_size)
        if chan is None:
            return None
        chan.invoke_subsystem('sftp')
        return cls(chan)

    def _log(self, level, msg, *args):
        if isinstance(msg, list):
            for m in msg:
                self._log(level, m, *args)
        else:
            # escape '%' in msg (they could come from file or directory names) before logging
            msg = msg.replace('%','%%')
            super(SFTPClient, self)._log(level, "[chan %s] " + msg, *([self.sock.get_name()] + list(args)))

    def close(self):
        """
        Close the SFTP session and its underlying channel.

        .. versionadded:: 1.4
        """
        self._log(INFO, 'sftp session closed.')
        self.sock.close()

    def get_channel(self):
        """
        Return the underlying `.Channel` object for this SFTP session.  This
        might be useful for doing things like setting a timeout on the channel.

        .. versionadded:: 1.7.1
        """
        return self.sock

    def listdir(self, path='.'):
        """
        Return a list containing the names of the entries in the given ``path``.

        The list is in arbitrary order.  It does not include the special
        entries ``'.'`` and ``'..'`` even if they are present in the folder.
        This method is meant to mirror ``os.listdir`` as closely as possible.
        For a list of full `.SFTPAttributes` objects, see `listdir_attr`.

        :param str path: path to list (defaults to ``'.'``)
        """
        return [f.filename for f in self.listdir_attr(path)]

    def listdir_attr(self, path='.'):
        """
        Return a list containing `.SFTPAttributes` objects corresponding to
        files in the given ``path``.  The list is in arbitrary order.  It does
        not include the special entries ``'.'`` and ``'..'`` even if they are
        present in the folder.

        The returned `.SFTPAttributes` objects will each have an additional
        field: ``longname``, which may contain a formatted string of the file's
        attributes, in unix format.  The content of this string will probably
        depend on the SFTP server implementation.

        :param str path: path to list (defaults to ``'.'``)
        :return: list of `.SFTPAttributes` objects

        .. versionadded:: 1.2
        """
        path = self._adjust_cwd(path)
        self._log(DEBUG, 'listdir(%r)' % path)
        t, msg = self._request(CMD_OPENDIR, path)
        if t != CMD_HANDLE:
            raise SFTPError('Expected handle')
        handle = msg.get_binary()
        filelist = []
        while True:
            try:
                t, msg = self._request(CMD_READDIR, handle)
            except EOFError:
                # done with handle
                break
            if t != CMD_NAME:
                raise SFTPError('Expected name response')
            count = msg.get_int()
            for i in range(count):
                filename = msg.get_text()
                longname = msg.get_text()
                attr = SFTPAttributes._from_msg(msg, filename, longname)
                if (filename != '.') and (filename != '..'):
                    filelist.append(attr)
        self._request(CMD_CLOSE, handle)
        return filelist

    def listdir_iter(self, path='.', read_aheads=50):
        """
        Generator version of `.listdir_attr`.

        See the API docs for `.listdir_attr` for overall details.

        This function adds one more kwarg on top of `.listdir_attr`:
        ``read_aheads``, an integer controlling how many
        ``SSH_FXP_READDIR`` requests are made to the server. The default of 50
        should suffice for most file listings as each request/response cycle
        may contain multiple files (dependant on server implementation.)

        .. versionadded:: 1.15
        """
        path = self._adjust_cwd(path)
        self._log(DEBUG, 'listdir(%r)' % path)
        t, msg = self._request(CMD_OPENDIR, path)

        if t != CMD_HANDLE:
            raise SFTPError('Expected handle')

        handle = msg.get_string()

        nums = list()
        while True:
            try:
                # Send out a bunch of readdir requests so that we can read the
                # responses later on Section 6.7 of the SSH file transfer RFC
                # explains this
                # http://filezilla-project.org/specs/draft-ietf-secsh-filexfer-02.txt
                for i in range(read_aheads):
                    num = self._async_request(type(None), CMD_READDIR, handle)
                    nums.append(num)


                # For each of our sent requests
                # Read and parse the corresponding packets
                # If we're at the end of our queued requests, then fire off
                # some more requests
                # Exit the loop when we've reached the end of the directory
                # handle
                for num in nums:
                    t, pkt_data = self._read_packet()
                    msg = Message(pkt_data)
                    new_num = msg.get_int()
                    if num == new_num:
                        if t == CMD_STATUS:
                            self._convert_status(msg)
                    count = msg.get_int()
                    for i in range(count):
                        filename = msg.get_text()
                        longname = msg.get_text()
                        attr = SFTPAttributes._from_msg(
                            msg, filename, longname)
                        if (filename != '.') and (filename != '..'):
                            yield attr

                # If we've hit the end of our queued requests, reset nums.
                nums = list()

            except EOFError:
                self._request(CMD_CLOSE, handle)
                return


    def open(self, filename, mode='r', bufsize=-1):
        """
        Open a file on the remote server.  The arguments are the same as for
        Python's built-in `python:file` (aka `python:open`).  A file-like
        object is returned, which closely mimics the behavior of a normal
        Python file object, including the ability to be used as a context
        manager.

        The mode indicates how the file is to be opened: ``'r'`` for reading,
        ``'w'`` for writing (truncating an existing file), ``'a'`` for
        appending, ``'r+'`` for reading/writing, ``'w+'`` for reading/writing
        (truncating an existing file), ``'a+'`` for reading/appending.  The
        Python ``'b'`` flag is ignored, since SSH treats all files as binary.
        The ``'U'`` flag is supported in a compatible way.

        Since 1.5.2, an ``'x'`` flag indicates that the operation should only
        succeed if the file was created and did not previously exist.  This has
        no direct mapping to Python's file flags, but is commonly known as the
        ``O_EXCL`` flag in posix.

        The file will be buffered in standard Python style by default, but
        can be altered with the ``bufsize`` parameter.  ``0`` turns off
        buffering, ``1`` uses line buffering, and any number greater than 1
        (``>1``) uses that specific buffer size.

        :param str filename: name of the file to open
        :param str mode: mode (Python-style) to open in
        :param int bufsize: desired buffering (-1 = default buffer size)
        :return: an `.SFTPFile` object representing the open file

        :raises IOError: if the file could not be opened.
        """
        filename = self._adjust_cwd(filename)
        self._log(DEBUG, 'open(%r, %r)' % (filename, mode))
        imode = 0
        if ('r' in mode) or ('+' in mode):
            imode |= SFTP_FLAG_READ
        if ('w' in mode) or ('+' in mode) or ('a' in mode):
            imode |= SFTP_FLAG_WRITE
        if 'w' in mode:
            imode |= SFTP_FLAG_CREATE | SFTP_FLAG_TRUNC
        if 'a' in mode:
            imode |= SFTP_FLAG_CREATE | SFTP_FLAG_APPEND
        if 'x' in mode:
            imode |= SFTP_FLAG_CREATE | SFTP_FLAG_EXCL
        attrblock = SFTPAttributes()
        t, msg = self._request(CMD_OPEN, filename, imode, attrblock)
        if t != CMD_HANDLE:
            raise SFTPError('Expected handle')
        handle = msg.get_binary()
        self._log(DEBUG, 'open(%r, %r) -> %s' % (filename, mode, hexlify(handle)))
        return SFTPFile(self, handle, mode, bufsize)

    # Python continues to vacillate about "open" vs "file"...
    file = open

    def remove(self, path):
        """
        Remove the file at the given path.  This only works on files; for
        removing folders (directories), use `rmdir`.

        :param str path: path (absolute or relative) of the file to remove

        :raises IOError: if the path refers to a folder (directory)
        """
        path = self._adjust_cwd(path)
        self._log(DEBUG, 'remove(%r)' % path)
        self._request(CMD_REMOVE, path)

    unlink = remove

    def rename(self, oldpath, newpath):
        """
        Rename a file or folder from ``oldpath`` to ``newpath``.

        :param str oldpath: existing name of the file or folder
        :param str newpath: new name for the file or folder

        :raises IOError: if ``newpath`` is a folder, or something else goes
            wrong
        """
        oldpath = self._adjust_cwd(oldpath)
        newpath = self._adjust_cwd(newpath)
        self._log(DEBUG, 'rename(%r, %r)' % (oldpath, newpath))
        self._request(CMD_RENAME, oldpath, newpath)

    def mkdir(self, path, mode=o777):
        """
        Create a folder (directory) named ``path`` with numeric mode ``mode``.
        The default mode is 0777 (octal).  On some systems, mode is ignored.
        Where it is used, the current umask value is first masked out.

        :param str path: name of the folder to create
        :param int mode: permissions (posix-style) for the newly-created folder
        """
        path = self._adjust_cwd(path)
        self._log(DEBUG, 'mkdir(%r, %r)' % (path, mode))
        attr = SFTPAttributes()
        attr.st_mode = mode
        self._request(CMD_MKDIR, path, attr)

    def rmdir(self, path):
        """
        Remove the folder named ``path``.

        :param str path: name of the folder to remove
        """
        path = self._adjust_cwd(path)
        self._log(DEBUG, 'rmdir(%r)' % path)
        self._request(CMD_RMDIR, path)

    def stat(self, path):
        """
        Retrieve information about a file on the remote system.  The return
        value is an object whose attributes correspond to the attributes of
        Python's ``stat`` structure as returned by ``os.stat``, except that it
        contains fewer fields.  An SFTP server may return as much or as little
        info as it wants, so the results may vary from server to server.

        Unlike a Python `python:stat` object, the result may not be accessed as
        a tuple.  This is mostly due to the author's slack factor.

        The fields supported are: ``st_mode``, ``st_size``, ``st_uid``,
        ``st_gid``, ``st_atime``, and ``st_mtime``.

        :param str path: the filename to stat
        :return:
            an `.SFTPAttributes` object containing attributes about the given
            file
        """
        path = self._adjust_cwd(path)
        self._log(DEBUG, 'stat(%r)' % path)
        t, msg = self._request(CMD_STAT, path)
        if t != CMD_ATTRS:
            raise SFTPError('Expected attributes')
        return SFTPAttributes._from_msg(msg)

    def lstat(self, path):
        """
        Retrieve information about a file on the remote system, without
        following symbolic links (shortcuts).  This otherwise behaves exactly
        the same as `stat`.

        :param str path: the filename to stat
        :return:
            an `.SFTPAttributes` object containing attributes about the given
            file
        """
        path = self._adjust_cwd(path)
        self._log(DEBUG, 'lstat(%r)' % path)
        t, msg = self._request(CMD_LSTAT, path)
        if t != CMD_ATTRS:
            raise SFTPError('Expected attributes')
        return SFTPAttributes._from_msg(msg)

    def symlink(self, source, dest):
        """
        Create a symbolic link (shortcut) of the ``source`` path at
        ``destination``.

        :param str source: path of the original file
        :param str dest: path of the newly created symlink
        """
        dest = self._adjust_cwd(dest)
        self._log(DEBUG, 'symlink(%r, %r)' % (source, dest))
        source = bytestring(source)
        self._request(CMD_SYMLINK, source, dest)

    def chmod(self, path, mode):
        """
        Change the mode (permissions) of a file.  The permissions are
        unix-style and identical to those used by Python's `os.chmod`
        function.

        :param str path: path of the file to change the permissions of
        :param int mode: new permissions
        """
        path = self._adjust_cwd(path)
        self._log(DEBUG, 'chmod(%r, %r)' % (path, mode))
        attr = SFTPAttributes()
        attr.st_mode = mode
        self._request(CMD_SETSTAT, path, attr)

    def chown(self, path, uid, gid):
        """
        Change the owner (``uid``) and group (``gid``) of a file.  As with
        Python's `os.chown` function, you must pass both arguments, so if you
        only want to change one, use `stat` first to retrieve the current
        owner and group.

        :param str path: path of the file to change the owner and group of
        :param int uid: new owner's uid
        :param int gid: new group id
        """
        path = self._adjust_cwd(path)
        self._log(DEBUG, 'chown(%r, %r, %r)' % (path, uid, gid))
        attr = SFTPAttributes()
        attr.st_uid, attr.st_gid = uid, gid
        self._request(CMD_SETSTAT, path, attr)

    def utime(self, path, times):
        """
        Set the access and modified times of the file specified by ``path``.  If
        ``times`` is ``None``, then the file's access and modified times are set
        to the current time.  Otherwise, ``times`` must be a 2-tuple of numbers,
        of the form ``(atime, mtime)``, which is used to set the access and
        modified times, respectively.  This bizarre API is mimicked from Python
        for the sake of consistency -- I apologize.

        :param str path: path of the file to modify
        :param tuple times:
            ``None`` or a tuple of (access time, modified time) in standard
            internet epoch time (seconds since 01 January 1970 GMT)
        """
        path = self._adjust_cwd(path)
        if times is None:
            times = (time.time(), time.time())
        self._log(DEBUG, 'utime(%r, %r)' % (path, times))
        attr = SFTPAttributes()
        attr.st_atime, attr.st_mtime = times
        self._request(CMD_SETSTAT, path, attr)

    def truncate(self, path, size):
        """
        Change the size of the file specified by ``path``.  This usually
        extends or shrinks the size of the file, just like the `~file.truncate`
        method on Python file objects.

        :param str path: path of the file to modify
        :param size: the new size of the file
        :type size: int or long
        """
        path = self._adjust_cwd(path)
        self._log(DEBUG, 'truncate(%r, %r)' % (path, size))
        attr = SFTPAttributes()
        attr.st_size = size
        self._request(CMD_SETSTAT, path, attr)

    def readlink(self, path):
        """
        Return the target of a symbolic link (shortcut).  You can use
        `symlink` to create these.  The result may be either an absolute or
        relative pathname.

        :param str path: path of the symbolic link file
        :return: target path, as a `str`
        """
        path = self._adjust_cwd(path)
        self._log(DEBUG, 'readlink(%r)' % path)
        t, msg = self._request(CMD_READLINK, path)
        if t != CMD_NAME:
            raise SFTPError('Expected name response')
        count = msg.get_int()
        if count == 0:
            return None
        if count != 1:
            raise SFTPError('Readlink returned %d results' % count)
        return _to_unicode(msg.get_string())

    def normalize(self, path):
        """
        Return the normalized path (on the server) of a given path.  This
        can be used to quickly resolve symbolic links or determine what the
        server is considering to be the "current folder" (by passing ``'.'``
        as ``path``).

        :param str path: path to be normalized
        :return: normalized form of the given path (as a `str`)

        :raises IOError: if the path can't be resolved on the server
        """
        path = self._adjust_cwd(path)
        self._log(DEBUG, 'normalize(%r)' % path)
        t, msg = self._request(CMD_REALPATH, path)
        if t != CMD_NAME:
            raise SFTPError('Expected name response')
        count = msg.get_int()
        if count != 1:
            raise SFTPError('Realpath returned %d results' % count)
        return msg.get_text()

    def chdir(self, path=None):
        """
        Change the "current directory" of this SFTP session.  Since SFTP
        doesn't really have the concept of a current working directory, this is
        emulated by Paramiko.  Once you use this method to set a working
        directory, all operations on this `.SFTPClient` object will be relative
        to that path. You can pass in ``None`` to stop using a current working
        directory.

        :param str path: new current working directory

        :raises IOError: if the requested path doesn't exist on the server

        .. versionadded:: 1.4
        """
        if path is None:
            self._cwd = None
            return
        if not stat.S_ISDIR(self.stat(path).st_mode):
            raise SFTPError(errno.ENOTDIR, "%s: %s" % (os.strerror(errno.ENOTDIR), path))
        self._cwd = b(self.normalize(path))

    def getcwd(self):
        """
        Return the "current working directory" for this SFTP session, as
        emulated by Paramiko.  If no directory has been set with `chdir`,
        this method will return ``None``.

        .. versionadded:: 1.4
        """
        # TODO: make class initialize with self._cwd set to self.normalize('.')
        return self._cwd and u(self._cwd)

    def _transfer_with_callback(self, reader, writer, file_size, callback):
        size = 0
        while True:
            data = reader.read(32768)
            writer.write(data)
            size += len(data)
            if len(data) == 0:
                break
            if callback is not None:
                callback(size, file_size)
        return size

    def putfo(self, fl, remotepath, file_size=0, callback=None, confirm=True):
        """
        Copy the contents of an open file object (``fl``) to the SFTP server as
        ``remotepath``. Any exception raised by operations will be passed
        through.

        The SFTP operations use pipelining for speed.

        :param fl: opened file or file-like object to copy
        :param str remotepath: the destination path on the SFTP server
        :param int file_size:
            optional size parameter passed to callback. If none is specified,
            size defaults to 0
        :param callable callback:
            optional callback function (form: ``func(int, int)``) that accepts
            the bytes transferred so far and the total bytes to be transferred
            (since 1.7.4)
        :param bool confirm:
            whether to do a stat() on the file afterwards to confirm the file
            size (since 1.7.7)

        :return:
            an `.SFTPAttributes` object containing attributes about the given
            file.

        .. versionadded:: 1.10
        """
        with self.file(remotepath, 'wb') as fr:
            fr.set_pipelined(True)
            size = self._transfer_with_callback(
                reader=fl, writer=fr, file_size=file_size, callback=callback
            )
        if confirm:
            s = self.stat(remotepath)
            if s.st_size != size:
                raise IOError('size mismatch in put!  %d != %d' % (s.st_size, size))
        else:
            s = SFTPAttributes()
        return s

    def put(self, localpath, remotepath, callback=None, confirm=True):
        """
        Copy a local file (``localpath``) to the SFTP server as ``remotepath``.
        Any exception raised by operations will be passed through.  This
        method is primarily provided as a convenience.

        The SFTP operations use pipelining for speed.

        :param str localpath: the local file to copy
        :param str remotepath: the destination path on the SFTP server. Note
            that the filename should be included. Only specifying a directory
            may result in an error.
        :param callable callback:
            optional callback function (form: ``func(int, int)``) that accepts
            the bytes transferred so far and the total bytes to be transferred
        :param bool confirm:
            whether to do a stat() on the file afterwards to confirm the file
            size

        :return: an `.SFTPAttributes` object containing attributes about the given file

        .. versionadded:: 1.4
        .. versionchanged:: 1.7.4
            ``callback`` and rich attribute return value added.
        .. versionchanged:: 1.7.7
            ``confirm`` param added.
        """
        file_size = os.stat(localpath).st_size
        with open(localpath, 'rb') as fl:
            return self.putfo(fl, remotepath, file_size, callback, confirm)

    def getfo(self, remotepath, fl, callback=None):
        """
        Copy a remote file (``remotepath``) from the SFTP server and write to
        an open file or file-like object, ``fl``.  Any exception raised by
        operations will be passed through.  This method is primarily provided
        as a convenience.

        :param object remotepath: opened file or file-like object to copy to
        :param str fl:
            the destination path on the local host or open file object
        :param callable callback:
            optional callback function (form: ``func(int, int)``) that accepts
            the bytes transferred so far and the total bytes to be transferred
        :return: the `number <int>` of bytes written to the opened file object

        .. versionadded:: 1.10
        """
        file_size = self.stat(remotepath).st_size
        with self.open(remotepath, 'rb') as fr:
            fr.prefetch(file_size)
            return self._transfer_with_callback(
                reader=fr, writer=fl, file_size=file_size, callback=callback
            )

        return size

    def get(self, remotepath, localpath, callback=None):
        """
        Copy a remote file (``remotepath``) from the SFTP server to the local
        host as ``localpath``.  Any exception raised by operations will be
        passed through.  This method is primarily provided as a convenience.

        :param str remotepath: the remote file to copy
        :param str localpath: the destination path on the local host
        :param callable callback:
            optional callback function (form: ``func(int, int)``) that accepts
            the bytes transferred so far and the total bytes to be transferred

        .. versionadded:: 1.4
        .. versionchanged:: 1.7.4
            Added the ``callback`` param
        """
        with open(localpath, 'wb') as fl:
            size = self.getfo(remotepath, fl, callback)
        s = os.stat(localpath)
        if s.st_size != size:
            raise IOError('size mismatch in get!  %d != %d' % (s.st_size, size))

    ###  internals...

    def _request(self, t, *arg):
        num = self._async_request(type(None), t, *arg)
        return self._read_response(num)

    def _async_request(self, fileobj, t, *arg):
        # this method may be called from other threads (prefetch)
        self._lock.acquire()
        try:
            msg = Message()
            msg.add_int(self.request_number)
            for item in arg:
                if isinstance(item, long):
                    msg.add_int64(item)
                elif isinstance(item, int):
                    msg.add_int(item)
                elif isinstance(item, (string_types, bytes_types)):
                    msg.add_string(item)
                elif isinstance(item, SFTPAttributes):
                    item._pack(msg)
                else:
                    raise Exception('unknown type for %r type %r' % (item, type(item)))
            num = self.request_number
            self._expecting[num] = fileobj
            self.request_number += 1
        finally:
            self._lock.release()
        self._send_packet(t, msg)
        return num

    def _read_response(self, waitfor=None):
        while True:
            try:
                t, data = self._read_packet()
            except EOFError as e:
                raise SSHException('Server connection dropped: %s' % str(e))
            msg = Message(data)
            num = msg.get_int()
            self._lock.acquire()
            try:
                if num not in self._expecting:
                    # might be response for a file that was closed before responses came back
                    self._log(DEBUG, 'Unexpected response #%d' % (num,))
                    if waitfor is None:
                        # just doing a single check
                        break
                    continue
                fileobj = self._expecting[num]
                del self._expecting[num]
            finally:
                self._lock.release()
            if num == waitfor:
                # synchronous
                if t == CMD_STATUS:
                    self._convert_status(msg)
                return t, msg
            if fileobj is not type(None):
                fileobj._async_response(t, msg, num)
            if waitfor is None:
                # just doing a single check
                break
        return None, None

    def _finish_responses(self, fileobj):
        while fileobj in self._expecting.values():
            self._read_response()
            fileobj._check_exception()

    def _convert_status(self, msg):
        """
        Raises EOFError or IOError on error status; otherwise does nothing.
        """
        code = msg.get_int()
        text = msg.get_text()
        if code == SFTP_OK:
            return
        elif code == SFTP_EOF:
            raise EOFError(text)
        elif code == SFTP_NO_SUCH_FILE:
            # clever idea from john a. meinel: map the error codes to errno
            raise IOError(errno.ENOENT, text)
        elif code == SFTP_PERMISSION_DENIED:
            raise IOError(errno.EACCES, text)
        else:
            raise IOError(text)

    def _adjust_cwd(self, path):
        """
        Return an adjusted path if we're emulating a "current working
        directory" for the server.
        """
        path = b(path)
        if self._cwd is None:
            return path
        if len(path) and path[0:1] == b_slash:
            # absolute path
            return path
        if self._cwd == b_slash:
            return self._cwd + path
        return self._cwd + b_slash + path
    class SFTPClient( )源码
    • 基于用户名密码上传下载
    • 基于公钥密钥上传下载

安装 paramiko 

pip3 install paramiko

操作 paramiko 

SSHClient

  1. SSHClient - 用于连接远程服务器并执行基本命令 
     
    import paramiko

ssh = paramiko.SSHClient()
ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
ssh.connect(hostname='172.16.201.134', port=22, username='janice', password='janice123') 

stdin, stdout, stderr = ssh.exec_command('ls -la')

results = stdout.read()
print(results.decode())
ssh.close()
    paramiko.SSHClient( )密码登入
    import paramiko

private_key = paramiko.RSAKey.from_private_key_file('/home/auto/.ssh/id_rsa')

ssh = paramiko.SSHClient()
ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
ssh.connect(hostname='172.16.201.134', port=22, username='user', pkey=private_key)

stdin, stdout, stderr = ssh.exec_command('ls -la')

results = stdout.read()
print(results.decode())
ssh.close()
    paramiko.SSHClient( )公钥密钥连接
  2. 创建 Transport 对象来连接
     
    import paramiko

ip_port = ('172.16.201.134',22,)
transport = paramiko.Transport(ip_port)
transport.connect(username='user',password='user')

ssh = paramiko.SSHClient()
ssh._transport = transport

stdin, stdout, stderr = ssh.exec_command('df')
results = stdout.read() # 获取命令结果

print(results.decode())
transport.close()
    paramiko.Transport(ip_port)密码登入
    import paramiko

private_key = paramiko.RSAKey.from_private_key_file('/Users/jcchoiling/.ssh/vm.key')

ip_port = ('172.16.201.134',22,)
transport = paramiko.Transport(ip_port)
transport.connect(username='user',pkey=private_key)

ssh = paramiko.SSHClient()
ssh._transport = transport

stdin, stdout, stderr = ssh.exec_command('df')
results = stdout.read() # 获取命令结果

print(results.decode())
transport.close()
    paramiko.Transport(ip_port)公钥密钥连接

     

SFPTClient

  1. SFTPClient - 用于连接远程服务器并执行上传下载
    第一步：创建 transport 通道 e.g. paramiko.Transport(ip_port) 连接对象，负责上传文件到目标服务器端
    第二步：transport 需要连接上服务器端，输入用户名和密码
    第三步：创建 SFTPClient 对象然后把 transport作为参数传入 e.g. paramiko.SFTPClient.from_transport(transport)
    第四步：可以调用 sftp.put/ sftp.get 方法来上传和下载文件
            四、一）sftp.put(localfile, remotefile) 从本地上传到远端
            四、二）sftp.get(remotefile, localfile) 从远端下载到本地
    第五步：关闭 transport 通道
     
    import paramiko
ip_port = ('172.16.201.134',22,)
transport = paramiko.Transport(ip_port)
transport.connect(username='user',password='user')

sftp = paramiko.SFTPClient.from_transport(transport)
sftp.put('/Users/jcchoiling/Desktop/movies.dat','/home/user/m1.dat')
sftp.get('/home/user/start.sh','/Users/jcchoiling/Desktop/start.sh')

transport.close()
    paramiko.SFTPClient.from_transport(transport)密码登入
    import paramiko

private_key = paramiko.RSAKey.from_private_key_file('/Users/jcchoiling/.ssh/vm.key')

ip_port = ('172.16.201.134',22,)
transport = paramiko.Transport(ip_port)
transport.connect(username='user',pkey=private_key)

sftp = paramiko.SFTPClient.from_transport(transport)
sftp.put('/Users/jcchoiling/Desktop/movies.dat','/home/user/m1.dat')
sftp.get('/home/user/start.sh','/Users/jcchoiling/Desktop/start.sh')

transport.close()
    paramiko.SFTPClient.from_transport(transport)公钥密钥连接

 

上下文操作应用

 

 

 

 

 

初探堡垒机

 

 

 

 

 

本周作业

作业：开发一个由数据库管理的主机管理系统,主机分组、分用户权限管理

  1. 所有的用户操作日志要保留在数据库中
  2. 每个用户登录堡垒机后，只需要选择具体要访问的设置，就连接上了，不需要再输入目标机器的访问密码
  3. 允许用户对不同的目标设备有不同的访问权限，例:
  4. 对10.0.2.34 有mysql 用户的权限
  5. 对192.168.3.22 有root用户的权限
  6. 对172.33.24.55 没任何权限
  7. 分组管理，即可以对设置进行分组，允许用户访问某组机器，但对组里的不同机器依然有不同的访问权限 

考核题

试说明你写这个作业的思路：

  1. 首先看到管理主机组，这表明可能会有多于一台服务器，然后假设我是管理100台 Hadoop 服务器的管理员。
  2. 然后设计数据库的表结构：
    • 有本地用户 User
    • 管理组表 Groups
    • 本地用户和组的关系表 User-Groups
    • 服务器表 Hosts
    • 远程用户表 RemoteUsers
    • 管理组表、远程用户和服务器的关系表 Hosts-Groups-RemoteUsers
    • 记录表 Audit Log
  3. 然后可以从功能方面思考：
    • 查看用戶信息
    • 创建群組
    • 创建用戶
    • 创建服務器
    • 创建遠程用戶
    • 刪除群組
    • 刪除用戶
    • 刪除服務器
    • 新增用戶到指定群組
    • 新增服務器到指定群組
    • 初始化數據庫
    • 刪除數據庫
    • 遠程連接　
  4. 最后是把功能的流程关连在一起，比如说，你会想像当管理员一打开程序，第一步是什么、下一步又会是什么，这样的思路去设计你程序的流程图。
    • 用户登入 (登入需要认证，可以添加登入3次锁定帐号)
    • 登入后看到功能列表选单
    • 每一个功能用一个函数来表达
    • 当程序遇上非如期的输入时的处理方法 (Error Handling)
    • 优雅地退出程序 (Exit program)

[知识点：重点是如果用 Python 操作数据库，从数据库中读写数据。]

数据库表结构：

 

程序运行结果：

 

总结

第五阶段主要是学习了如何用 Python 来实现网络编程，第一部份是介绍了基本的网络协议，其中重点是 TCP/IP 协议，学习如何写服务器端和客户端的 socket，还数据可以通个 TCP/IP 来输送和接收数据；第二部份介绍了 Python中的线程、进程和协程。然后还学了线程锁与进程锁；学了自定义线程池。第三部份学了消息队列的概念，分别是 Python 内置的 Queue 功能和 RabbitMQ的功能。具体介绍了 RabbitMQ 的发布和订阅、主题模式和 RPC 通信。第四部份学了数据库的操作：分别是 MySQL 中的原生 SQL话句和如何用 Python 的 pymysql 来操作 MySQL 数据库。在了这个基础之后，便深入介绍 Python 中的 ORM，当中最具代表性的就是 SQLAlchemy 模块，学习如何用 Python 调用 SQLAlchemy 中的功能来操作 MySQL数据库，这是一个很有意思的单元。 

 

 

 

參考资料 

银角大王：1) MySQL 操作

金角大王：1) Python之路,Day10~11 - 那就做个堡垒机吧

　　　　　2) python 之路，Day11 - sqlalchemy ORM

　　　　   3) 金角大王教你如何做个堡垒机

OReilly.Essential.SQLAlchemy

 

posted @ 2016-10-24 00:23  無情  阅读(618)  评论(0)  编辑  收藏  举报