EVE Online Third Party Development

 

 第一部分：price_history表

# 建表语句

CREATE TABLE IF NOT EXISTS `price_history` (
  `regionID`  INT    NOT NULL,
  `typeID`    INT    NOT NULL,
  `date`      DATE   NOT NULL,
  `lowPrice`  DOUBLE NOT NULL,
  `highPrice` DOUBLE NOT NULL,
  `avgPrice`  DOUBLE NOT NULL,
  `volume`    INT    NOT NULL,
  `orders`    INT    NOT NULL
  COMMENT 'regionID,typeID,date,lowPrice,highPrice,avgPrice,volume,orders',
  INDEX `regionID`(`regionID`),
  INDEX `typeID`(`typeID`),
  INDEX `date`(`date`),
  INDEX `lowPrice`(`lowPrice`),
  INDEX `highPrice`(`highPrice`),
  INDEX `avgPrice`(`avgPrice`),
  INDEX `volume`(`volume`),
  INDEX `orders`(`orders`)
)
  ENGINE = InnoDB
  DEFAULT CHARSET = UTF8MB4;

 考虑到price字段使用FLOAT/DOUBLE类型会有精度问题（FLOAT 4个字节，有效数字6位；DOUBLE 8个字节，有效数字16位。）

# 优化以后的建表语句，关于DECIMAL请参见 https://dev.mysql.com/doc/refman/5.7/en/precision-math-decimal-characteristics.html

CREATE TABLE IF NOT EXISTS `price_history` (
  `regionID`  INT            NOT NULL,
  `typeID`    INT            NOT NULL,
  `date`      DATE           NOT NULL,
  `lowPrice`  DECIMAL(17, 4) NOT NULL,
  `highPrice` DECIMAL(17, 4) NOT NULL,
  `avgPrice`  DECIMAL(17, 4) NOT NULL
  COMMENT 'The max(highPrice) is 700,000,000,000',
  `volume`    INT            NOT NULL,
  `orders`    INT            NOT NULL
  COMMENT 'regionID,typeID,date,lowPrice,highPrice,avgPrice,volume,orders',
  INDEX `regionID`(`regionID`),
  INDEX `typeID`(`typeID`),
  INDEX `date`(`date`),
  INDEX `lowPrice`(`lowPrice`),
  INDEX `highPrice`(`highPrice`),
  INDEX `avgPrice`(`avgPrice`),
  INDEX `volume`(`volume`),
  INDEX `orders`(`orders`)
)
  ENGINE = InnoDB
  DEFAULT CHARSET = UTF8MB4;

 接下来进行数据的导入，表空间占用累计为70223M。

# 数据导入
LOAD DATA LOCAL INFILE '/root/priceHistory.txt' INTO TABLE price_history FIELDS TERMINATED BY ',' LINES TERMINATED BY '\n';

MySQL [eveonline_price]> LOAD DATA LOCAL INFILE '/root/priceHistory.txt' INTO TABLE price_history FIELDS TERMINATED BY ',' LINES TERMINATED BY '\n';
Query OK, 269257439 rows affected, 14686 warnings (5 hours 25 min 41.31 sec)
Records: 269257439  Deleted: 0  Skipped: 0  Warnings: 14686

 第二部分：typeid表

首先，从网上下载比较新的typeID文件，然后进行适当的处理：首先去掉第一和第二行，然后将空格分割的文件进行处理，以逗号进行分割。

# 下载下来的文件保存为download.txt，处理后文件保存为typeID.txt
with open('typeID.txt', mode='w', encoding="UTF-8") as out:
    
    for line in  open('download.txt', mode='r', encoding="UTF-8"):
    
        line = re.sub(r'\s+', ' ', line).split()
        key= line[0]
        name = ' '.join(line[1:])  
        
        out.write(key + ',' + name + "\n")

建立typeid表

# MySQL数据库字段支持大小写，但是底层一律转换成小写（typeID和typeid是同一字段）

CREATE TABLE IF NOT EXISTS `typeid` (
  `typeID`   MEDIUMINT PRIMARY KEY,
  `typeName` VARCHAR(90) NOT NULL,
  INDEX `typeName`(`typeName`)
)
  ENGINE = InnoDB
  DEFAULT CHARACTER SET = UTF8MB4;

经过对文件的读取判断，typeName字段最长为78个字符，这里留90字符作为保留量。接下来进行数据的导入，

LOAD DATA LOCAL INFILE '/root/typeID.txt' INTO TABLE typeid FIELDS TERMINATED BY ',' LINES TERMINATED BY '\n';

 

 

 

 

Importing market data into Excel，如何将市场数据导入Excel

USING ESI WITH GOOGLE SHEETS，将ESI和Google Sheets配合使用

Marketplace，CCP官方论坛交易市场

EVE Online Intelligence Data，Tools and data you need to succeed in EVE Online

EVE Trade，暂时不知道做什么用

 

EVE Online price history download link，大小 

Development Blog，开发者日志

 

MySQL部分知识：

posted @ 2018-04-17 16:10  又是火星人  阅读(337)  评论(0)  编辑  收藏  举报