< 2025年3月 > 日 一 二 三 四 五 六 23 24 25 26 27 28 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 1 2 3 4 5

导航

• 博客园
• 首页
• 新随笔
• 新文章
• 联系
• 管理

• 随笔 - 754
• 文章 - 38
• 评论 - 0
• 阅读 - 54032

01 2025 档案

• Spark for ML - Study Notes 5
  摘要： 阅读全文
  posted @ 2025-01-31 11:27 ZhangZhihuiAAA 阅读(1) 评论(0) 推荐(0) 编辑
• Spark for ML - Study Notes 4
  摘要：import pandas as pd # 示例数据 data = { "X1": [1, 2, 3, 4, 5], "X2": [2, 4, 6, 8, 10], # X2 是 X1 的两倍，完全共线 "X3": [5, 3, 4, 2, 1] } df = pd.DataFrame(data) 阅读全文
  posted @ 2025-01-30 11:56 ZhangZhihuiAAA 阅读(2) 评论(0) 推荐(0) 编辑
• VSCode - How to stop terminal from inheriting virtual environments?
  摘要： 阅读全文
  posted @ 2025-01-30 09:38 ZhangZhihuiAAA 阅读(2) 评论(0) 推荐(0) 编辑
• Spark for ML - Study Notes 3
  摘要：from pyspark.sql import SparkSession from pyspark.ml.feature import HashingTF, IDF, Tokenizer spark = SparkSession.builder.appName("TF-IDF Example").g 阅读全文
  posted @ 2025-01-28 17:37 ZhangZhihuiAAA 阅读(3) 评论(0) 推荐(0) 编辑
• Spark for ML - Study Notes 2
  摘要：from pyspark.sql import SparkSession spark = SparkSession.builder.appName("HDFS Read Example").getOrCreate() # Define the HDFS path hdfs_path = "hdfs: 阅读全文
  posted @ 2025-01-26 22:33 ZhangZhihuiAAA 阅读(2) 评论(0) 推荐(0) 编辑
• Spark for ML - Study Notes 1
  摘要： 阅读全文
  posted @ 2025-01-26 21:42 ZhangZhihuiAAA 阅读(1) 评论(0) 推荐(0) 编辑
• Singular Value Decomposition (SVD)
  摘要：Singular Value Decomposition (SVD) is a powerful mathematical technique used in linear algebra to factorize a matrix into three simpler matrices. It i 阅读全文
  posted @ 2025-01-26 16:37 ZhangZhihuiAAA 阅读(7) 评论(0) 推荐(0) 编辑
• Pearson or Spearman correlation
  摘要：Pearson vs. Spearman Correlation: When to Use Each? Both Pearson and Spearman correlations measure the relationship between two variables, but they ar 阅读全文
  posted @ 2025-01-26 16:16 ZhangZhihuiAAA 阅读(11) 评论(0) 推荐(0) 编辑
• Spark
  摘要：1. Download: https://spark.apache.org/downloads.html 2. Install: (base) zzh@ZZHPC:~/Downloads/sfw$ tar -xvzf spark-3.5.4-bin-hadoop3.tgz 3. Set enviro 阅读全文
  posted @ 2025-01-26 12:30 ZhangZhihuiAAA 阅读(5) 评论(0) 推荐(0) 编辑
• Airflow - Study Notes 6
  摘要：1. First, we will set up the imports that are required for the dashboard view: from __future__ import annotations from typing import TYPE_CHECKING fro 阅读全文
  posted @ 2025-01-20 13:16 ZhangZhihuiAAA 阅读(4) 评论(0) 推荐(0) 编辑
• Airflow - AirflowTimetableInvalid: Exactly 5, 6 or 7 columns has to be specified for iterator expression.
  摘要：from airflow.decorators import ( dag, task, ) from pendulum import datetime @dag( schedule="@none", start_date=datetime(2025, 1, 1), catchup=False, de 阅读全文
  posted @ 2025-01-19 23:18 ZhangZhihuiAAA 阅读(30) 评论(0) 推荐(0) 编辑
• Airflow - Study Notes 5
  摘要：To create a new connection, select the + to add a new record. from airflow.decorators import ( dag, task, ) from pendulum import datetime @dag( start_ 阅读全文
  posted @ 2025-01-19 22:38 ZhangZhihuiAAA 阅读(1) 评论(0) 推荐(0) 编辑
• Airflow - Study Notes 4
  摘要：To retrieve these images, I frequently make use of the NASA Astronomy Picture of the Day API (https://apod.nasa.gov/apod/astropix.html) to gather a ne 阅读全文
  posted @ 2025-01-19 15:35 ZhangZhihuiAAA 阅读(3) 评论(0) 推荐(0) 编辑
• Airflow - Study Notes 3
  摘要：(.venv) frank@ZZHUBT:~/venvs/my_airflow_project$ airflow config get-value core executor SequentialExecutor 阅读全文
  posted @ 2025-01-19 13:46 ZhangZhihuiAAA 阅读(2) 评论(0) 推荐(0) 编辑
• Airflow - Study Notes 2
  摘要：rank@ZZHUBT:~$pip install airflowctl rank@ZZHUBT:~$ airflowctl init my_airflow_project --build-start ...... ebserver | [2025-01-18 20:46:08 +0800] [1 阅读全文
  posted @ 2025-01-18 21:51 ZhangZhihuiAAA 阅读(2) 评论(0) 推荐(0) 编辑
• Airflow - Study Notes 1
  摘要：Apache Airflow is known within the data engineering community as the go-to open source platform for “developing, scheduling, and monitoring batch-orie 阅读全文
  posted @ 2025-01-18 20:16 ZhangZhihuiAAA 阅读(4) 评论(0) 推荐(0) 编辑
• PySpark - Study Notes 2
  摘要：For the purpose of this book, we use the Docker version of PySpark, running on a single machine. If you have a version of PySpark installed on a distr 阅读全文
  posted @ 2025-01-18 11:16 ZhangZhihuiAAA 阅读(1) 评论(0) 推荐(0) 编辑
• PySpark - Study Notes 1
  摘要：frank@ZZHUBT:~$ docker pull jupyter/pyspark-notebook docker run --name pyspark-notebook -p 8888:8888 -v ~/dkvols/pyspark-notebook/:/home/jovyan/work/ 阅读全文
  posted @ 2025-01-17 21:20 ZhangZhihuiAAA 阅读(3) 评论(0) 推荐(0) 编辑
• DuckDB - Study Notes 11
  摘要：(zpy310) frank@ZZHUBT:~$pip install duckdb (zpy310) frank@ZZHUBT:~$ pip install harlequin ...... Successfully installed MarkupSafe-3.0.2 click-8.1.8 阅读全文
  posted @ 2025-01-17 13:19 ZhangZhihuiAAA 阅读(8) 评论(0) 推荐(0) 编辑
• DuckDB - Study Notes 10
  摘要：import duckdb records = duckdb.read_csv("data/C11/Pedestrian_Counting_System_Monthly_counts_per_hour_may_2009_to_14_dec_2022.csv") records.show(max_wi 阅读全文
  posted @ 2025-01-16 21:53 ZhangZhihuiAAA 阅读(3) 评论(0) 推荐(0) 编辑
• DuckDB - Study Notes 9
  摘要：In this section, we are going to explore some of the DuckDB niceties – little shortcuts and tweaks to make your use of DuckDB easier. D CREATE OR REPL 阅读全文
  posted @ 2025-01-16 18:03 ZhangZhihuiAAA 阅读(7) 评论(0) 推荐(0) 编辑
• Virtualbox - Virtualized CPU missing features AVX, AVX2 and FMA
  摘要：(duckdb_book) frank@ZZHUBT:~$ python Python 3.13.1 (main, Jan 15 2025, 18:12:47) [GCC 11.4.0] on linux Type "help", "copyright", "credits" or "license 阅读全文
  posted @ 2025-01-16 10:56 ZhangZhihuiAAA 阅读(96) 评论(0) 推荐(0) 编辑
• DuckDB - Study Notes 8
  摘要：Requirements: duckdb emoji ibis-framework ibis-framework[duckdb] jupysql jupyter-lsp jupyterlab pandas plotly polars-lts-cpu pyarrow sqlparse pi_relat 阅读全文
  posted @ 2025-01-15 20:52 ZhangZhihuiAAA 阅读(4) 评论(0) 推荐(0) 编辑
• VSCode - Python formatter doesn't work
  摘要：I installed VSCode extension Black Formatter, but it seemed not work. Later, I found that it only work when there's no syntax error and it can't forma 阅读全文
  posted @ 2025-01-15 20:09 ZhangZhihuiAAA 阅读(3) 评论(0) 推荐(0) 编辑
• DuckDB - Study Notes 7
  摘要：import duckdb duckdb.sql("SELECT 'duck' AS animal, 'quack!' AS greeting") ┌─────────┬──────────┐ │ animal │ greeting │ │ varchar │ varchar │ ├──────── 阅读全文
  posted @ 2025-01-15 17:41 ZhangZhihuiAAA 阅读(2) 评论(0) 推荐(0) 编辑
• Pip - Installing plotly stuck
  摘要：Installing plotly stuck in a Python venv virtual environment in WSL2 Ubuntu. pip install -i https://pypi.org/simple package_namepip install -i https:/ 阅读全文
  posted @ 2025-01-15 11:40 ZhangZhihuiAAA 阅读(22) 评论(0) 推荐(0) 编辑
• DuckDB - Study Notes 6
  摘要：DuckDB’s nested data types: LIST, MAP, and STRUCT. D SELECT [7,8,9] AS list_int; ┌───────────┐ │ list_int │ │ int32[] │ ├───────────┤ │ [7, 8, 9] │ └─ 阅读全文
  posted @ 2025-01-14 21:35 ZhangZhihuiAAA 阅读(5) 评论(0) 推荐(0) 编辑
• DuckDB - Study Notes 5
  摘要：D SELECT * FROM duckdb_extensions(); ┌──────────────────┬─────────┬───────────┬──────────────┬───┬───────────────────┬───────────────────┬──────────── 阅读全文
  posted @ 2025-01-14 17:53 ZhangZhihuiAAA 阅读(1) 评论(0) 推荐(0) 编辑
• Virtualbox - NAT Port Forwarding
  摘要：Can VM communicate with the host in NAT mode? No, in NAT mode in VirtualBox, the virtual machine (VM) cannot directly communicate with the host unless 阅读全文
  posted @ 2025-01-14 00:14 ZhangZhihuiAAA 阅读(11) 评论(0) 推荐(0) 编辑
• Ubuntu - How to update snap store
  摘要：frank@ZZHUBT:~$ sudo snap refresh snap-store error: cannot refresh "snap-store": snap "snap-store" has running apps (snap-store), pids: 2979 frank@ZZH 阅读全文
  posted @ 2025-01-13 21:43 ZhangZhihuiAAA 阅读(15) 评论(0) 推荐(1) 编辑
• Star Schema and Snowflake schema
  摘要：The Star Schema and Snowflake Schema are two types of dimensional models used in data warehousing to organize data for analytical queries. Both schema 阅读全文
  posted @ 2025-01-13 10:06 ZhangZhihuiAAA 阅读(11) 评论(0) 推荐(0) 编辑
• DuckDB - Study Notes 4
  摘要：block range index, BRIN adaptive radix tree, ART To download the necessary dataset for this project, please follow these instructions: 1. Go to https: 阅读全文
  posted @ 2025-01-09 23:39 ZhangZhihuiAAA 阅读(10) 评论(0) 推荐(0) 编辑
• DuckDB - Study Notes 3
  摘要：Data Wrangling CREATE OR REPLACE TABLE web_log_text (raw_text VARCHAR); COPY web_log_text FROM 'access.log' (DELIM ''); SELECT regexp_extract(raw_text 阅读全文
  posted @ 2025-01-09 18:27 ZhangZhihuiAAA 阅读(6) 评论(0) 推荐(0) 编辑
• DuckDB - Study Notes 2
  摘要：Parameters The parameters listed below are used in the read_csv function to configure the CSV Rejects Table. NameDescriptionTypeDefault store_rejects 阅读全文
  posted @ 2025-01-09 00:20 ZhangZhihuiAAA 阅读(14) 评论(0) 推荐(0) 编辑
• VSCode - Change default terminal from Powershell to WSL shell
  摘要：To change the default terminal in Visual Studio Code (VSCode) to the WSL (Windows Subsystem for Linux) shell instead of PowerShell, follow these steps 阅读全文
  posted @ 2025-01-08 13:49 ZhangZhihuiAAA 阅读(9) 评论(0) 推荐(0) 编辑
• VSCode - Can't open in WSL2 Ubuntu Desktop
  摘要：frank@ZZHPC:~$ code --version To use Visual Studio Code with the Windows Subsystem for Linux, please install Visual Studio Code in Windows and uninsta 阅读全文
  posted @ 2025-01-08 13:12 ZhangZhihuiAAA 阅读(2) 评论(0) 推荐(0) 编辑
• DuckDB - Study Notes 1
  摘要：https://duckdb.org/docs/installation Windows: PS C:\Users\ZhangZhihui> winget install DuckDB.cli 已找到 DuckDB CLI [DuckDB.cli] 版本 1.1.3 此应用程序由其所有者授权给你。 阅读全文
  posted @ 2025-01-07 22:27 ZhangZhihuiAAA 阅读(8) 评论(0) 推荐(0) 编辑
• Flameshot - Install on Windows
  摘要：PS C:\Users\ZhangZhihui> winget install flameshot “msstore”源要求在使用前查看以下协议。 Terms of Transaction: https://aka.ms/microsoft-store-terms-of-transaction 源要 阅读全文
  posted @ 2025-01-07 20:53 ZhangZhihuiAAA 阅读(8) 评论(0) 推荐(0) 编辑
• WSL2 Ubuntu has no software center
  摘要：frank@ZZHPC:~$sudo apt list gnome-software [sudo] password for frank: Listing... Done gnome-software/noble 46.0-1ubuntu2 amd64 frank@ZZHPC:~$ frank@Z 阅读全文
  posted @ 2025-01-07 14:18 ZhangZhihuiAAA 阅读(13) 评论(0) 推荐(0) 编辑
• WSL2 Ubuntu Desktop favorites bar not showing
  摘要：The settings showed by clicking the settings icon: frank@ZZHPC:~$ sudo apt install gnome-shell-extension-dashtodock Reading package lists... Done Buil 阅读全文
  posted @ 2025-01-07 13:10 ZhangZhihuiAAA 阅读(6) 评论(0) 推荐(0) 编辑
• WSL2 - Install Ubuntu Desktop and XRDP
  摘要：Step 1 – Installing Ubuntu Desktop sudo apt updatesudo apt upgrade -y sudo apt install ubuntu-desktop -y 在GNOME安装过程中，“waiting for automatic snapd rest 阅读全文
  posted @ 2025-01-07 09:56 ZhangZhihuiAAA 阅读(57) 评论(0) 推荐(0) 编辑
• WSL2 - Install an X Server and set DISPLAY
  摘要：Since WSL2 doesn’t natively support GUI apps, you need an X server to render the GUI on Windows. Install VcXsrv: Download and install VcXsrv from here 阅读全文
  posted @ 2025-01-06 21:47 ZhangZhihuiAAA 阅读(49) 评论(0) 推荐(0) 编辑
• WSL2 Installation
  摘要：C:\Program Files\WSL是系统自带的，与下面WSL功能是否启用无关，不能将基删除或移走，否则系统服务WSL Service将无法运行，wsl命令也无法执行。 Enable the WSL feature: Install: PS C:\Users\ZhangZhihui> wsl - 阅读全文
  posted @ 2025-01-06 21:11 ZhangZhihuiAAA 阅读(40) 评论(0) 推荐(0) 编辑
• VirtualBox - Create a Ubuntu Virtual Machine on Windows
  摘要：The Paravirtualization Interface setting in VirtualBox determines how the guest operating system interacts with the underlying host system for perform 阅读全文
  posted @ 2025-01-06 14:41 ZhangZhihuiAAA 阅读(6) 评论(0) 推荐(0) 编辑
• Adobe Reader - How to move bookmarks panel to the left side?
  摘要：With the latest update to Adobe Acrobat, the bookmarks side panel that used to live on the left side of the screen has moved to the right. The only wa 阅读全文
  posted @ 2025-01-06 11:07 ZhangZhihuiAAA 阅读(18) 评论(0) 推荐(0) 编辑
• 新电脑Windows设置过程中如何跳过微软帐户注册
  摘要：１、关闭网络 ２、按Shift+F10打开命令行窗口 ３、输入'oobe\bypassnro'，回车，电脑会自动重启。 ４、重启后在选择网络时会出现一个“我没有可用的internet连接“的连接，点击此连接就可以无须注册微软帐户继续设置。 阅读全文
  posted @ 2025-01-04 16:46 ZhangZhihuiAAA 阅读(67) 评论(0) 推荐(0) 编辑
• VirtualBox - Permission denied when accessing the shared folder
  摘要：zzh@ZZHUBT:~$cd /zzhwin bash: cd: /zzhwin: Permission denied zzh@ZZHUBT:~$ sudo su root@ZZHUBT:/home/zzh# cd /zzhwin root@ZZHUBT:/zzhwin# ls -al tota 阅读全文
  posted @ 2025-01-01 15:19 ZhangZhihuiAAA 阅读(4) 评论(0) 推荐(0) 编辑