duckdb
练习学习sql的新选择-用duckdb代替mysql个人最佳分析数据库
01.duckdb和依赖的parquet、httpfs插件
02.测试duckdb是否能正常使用OSS,
03.在postgresql中使用 pg_duckdb 访问oss内的parquet文件
duckdb扩展
duckdb_mysql. DuckDB to directly read and write data from a MySQL database instance
sqlite_scanner. DuckDB to directly read and write data from a SQLite database file
postgres_scanner DuckDB to directly read and write data from a running Postgres database instance.
duckdb_httpfs http, https, s3
duckdb_spatial
Postgres extension for DuckDB
pg_duckdb https://github.com/duckdb/pg_duckdb
parqut数据
步骤
1.MongoDB数据导出json - MongoDB的全量数据 --json
2.Duckdb读取json
create table duck_t1 as SELECT * FROM read_json_auto('t1.json');
3.Duckdb映射远端MySQL mysql_dat库，并起一个数据库别名 mysql_alias
ATTACH 'host=192.168.1.1 user=admin password=123456 port=3306 database=mysql_dat' AS mysql_alias (TYPE mysql_scanner);
4.Duckdb中操作mysql数据库
create table mysql_alias.t1 as SELECT * EXCLUDE('_id') FROM me.t1;
扩展下载： linux_amd64_gcc4 windows_amd64 osx_arm64 osx_amd64
https://extensions.duckdb.org/v1.0.0/linux_amd64_gcc4/httpfs.duckdb_extension.gz
Duckdb第三方扩展
ClickHouse SQL Dialect Macros 使用CK的 SQL 方言
下载： https://community-extensions.duckdb.org/v1.0.0/linux_amd64_gcc4/chsql.duckdb_extension.gz
解压： gzip -dk chsql.duckdb_extension.gz
,-k选项表示保留解压前的原始文件,
参考
把MongoDB的全量数据导入到MySQL里 https://blog.51cto.com/hcymysql/10226864
https://duckdb.org/docs/extensions/core_extensions.html
https://community-extensions.duckdb.org/list_of_extensions.html
https://github.com/lmangani/duckdb-extension-clickhouse-sql
