本人DO计划体系导航 - Trello
优先级高
// OS总结篇章
[link] 与程序员相关的CPU缓存知识
一些经典的kaggle,数据科学的问题还是需要收集案例学习一下。
PyTorch学习资源整理:
- 最先看,PyTorch官方60min入门指南
- 对照着看,PyTorch深度学习:60分钟入门(官方指南的翻译)
- 中文版PyTorch
- 香港科大的3天入门PyTorch
- 莫烦:等什么, 赶快抱紧 PyTorch 的大腿!
- 韩国人在github上写的PyTorch学习指南
- PyTorch在github上的tutorials
- PyTorch doc
- PyTorch Forums
- 知乎:新手如何入门PyTorch
- 简书作者:SherlockLiao写的PyTorch入门指南
- PyTorch资料整合
- 吐血整理:PyTorch项目代码与资源列表 | 资源下载
c++ 并行机器学习
百万量级时,常用的数据当然可以提前存放在gpu mem里。
A Roadmap of Parallel Sorting Algorithms using GPU Computing
堆排序的并行优势感人。附送官方代码:
https://developer.nvidia.com/gpugems/GPUGems/gpugems_ch37.html
一、三个月内完成
python需要系统的过一遍。
tensorflow也要熟练使用各种特性,包括caffe。
machine leanring相关的算法需要再彻底整理一遍。
动物行为识别:http://www.mousemotorlab.org/deeplabcut
二、今年尽量过一遍
React Native摄像头功能实现!
以下的都暂时忽略。
### 大数据年底职位 ###
技术栈:Java, Scala, hadoop, spark, steaming/hive/tensorflow
Docker 学习
https://app.codility.com/programmers/
### 数据分析系统 ###
目前想做的是:Jan 12, 2019
将自己的知识体系彻底整理一遍,把大部分实体书都电子化。
大数据结合gpu这块是个有意思的地方,需要research。
前言
既然是实践体系,就要有手头工具书的效果。开这一篇章,源于一个经历,share with YOU。
自认为Machine Learning and Statistical Inference在IT界还算玩得溜,初次闯荡Data Science也是resume纷飞广撒网,终被一猎头相中,抛来一test。
打开aws一瞧,data size = 50G! ... 蒙了。原以为下载本地,用Python折腾一圈也就是一天的活儿。不过下载这50G的parquet files也不太现实。
Google一圈后,才明白这是scala+spark+aws analysis api的套路。虽然熟悉这套路也就十天半月的事情,但这次机会也算是飞了。
顿悟,纵然理论玩得溜,实验做得欢,但与工业界脱轨实则是一大方向性错误。故,除了理论体系还需一套实践体系支撑自我。
教学资源
环境配置
Ubuntu + Visual Studio Code with vim mode.
-
- MS出品
- 应付多种语言:c/c++, python, js, c#
- 界面舒服
VSCode's GitLens,实时查看修改记录,棒棒哒!
Expo.io: 发布测试容器。
文章导航
Goto: 我的软件工程导航
我的工具箱
Ubuntu Kernel
首先,/boot空间还是尽量空间分配大点吧,至少一个G。
内核下载:https://kernel.ubuntu.com/~kernel-ppa/mainline/
内核挂掉不要慌:Ubuntu 16.04 Linux系统内核升级方法
Qt-creator

#
# qmake configuration for win32-msvc2008
#
# Written for Microsoft VC2005.NET
#
MAKEFILE_GENERATOR = MSVC.NET
TEMPLATE = app
CONFIG += qt warn_on release incremental flat link_prl precompile_header autogen_precompile_source copy_dir_files debug_and_release debug_and_release_target embed_manifest_dll embed_manifest_exe
QT += core gui
DEFINES += UNICODE WIN32 QT_LARGEFILE_SUPPORT
QMAKE_COMPILER_DEFINES += _MSC_VER=1500 WIN32
QMAKE_CC = cl
QMAKE_LEX = flex
QMAKE_LEXFLAGS =
QMAKE_YACC = byacc
QMAKE_YACCFLAGS = -d
QMAKE_CFLAGS = -nologo -Zm200 -Zc:wchar_t-
QMAKE_CFLAGS_WARN_ON = -W3
QMAKE_CFLAGS_WARN_OFF = -W0
QMAKE_CFLAGS_RELEASE = -MT
QMAKE_CFLAGS_RELEASE_WITH_DEBUGINFO += -MT -Zi
QMAKE_CFLAGS_DEBUG = -Zi -MDd
QMAKE_CFLAGS_YACC =
QMAKE_CFLAGS_LTCG = -GL
QMAKE_CFLAGS_MP = -MP
QMAKE_CXX = $QMAKE_CC
QMAKE_CXXFLAGS = $QMAKE_CFLAGS
QMAKE_CXXFLAGS_WARN_ON = $QMAKE_CFLAGS_WARN_ON -w34100 -w34189
QMAKE_CXXFLAGS_WARN_OFF = $QMAKE_CFLAGS_WARN_OFF
QMAKE_CXXFLAGS_RELEASE = $QMAKE_CFLAGS_RELEASE
QMAKE_CXXFLAGS_RELEASE_WITH_DEBUGINFO += $QMAKE_CFLAGS_RELEASE_WITH_DEBUGINFO
QMAKE_CXXFLAGS_DEBUG = $QMAKE_CFLAGS_DEBUG
QMAKE_CXXFLAGS_YACC = $QMAKE_CFLAGS_YACC
QMAKE_CXXFLAGS_LTCG = $QMAKE_CFLAGS_LTCG
QMAKE_CXXFLAGS_MP = $QMAKE_CFLAGS_MP
QMAKE_CXXFLAGS_STL_ON = -EHsc
QMAKE_CXXFLAGS_STL_OFF =
QMAKE_CXXFLAGS_RTTI_ON = -GR
QMAKE_CXXFLAGS_RTTI_OFF =
QMAKE_CXXFLAGS_EXCEPTIONS_ON = -EHsc
QMAKE_CXXFLAGS_EXCEPTIONS_OFF =
QMAKE_INCDIR =
QMAKE_INCDIR_QT = $[QT_INSTALL_HEADERS]
QMAKE_LIBDIR_QT = $[QT_INSTALL_LIBS]
QMAKE_RUN_CC = $(CC) -c $(CFLAGS) $(INCPATH) -Fo$obj $src
QMAKE_RUN_CC_IMP = $(CC) -c $(CFLAGS) $(INCPATH) -Fo$@ {1}lt;
QMAKE_RUN_CC_IMP_BATCH = $(CC) -c $(CFLAGS) $(INCPATH) -Fo$@ @<<
QMAKE_RUN_CXX = $(CXX) -c $(CXXFLAGS) $(INCPATH) -Fo$obj $src
QMAKE_RUN_CXX_IMP = $(CXX) -c $(CXXFLAGS) $(INCPATH) -Fo$@ {1}lt;
QMAKE_RUN_CXX_IMP_BATCH = $(CXX) -c $(CXXFLAGS) $(INCPATH) -Fo$@ @<<
QMAKE_LINK = link
QMAKE_LFLAGS = /NOLOGO
QMAKE_LFLAGS_RELEASE = /NODEFAULTLIB:msvcrt.lib /INCREMENTAL:NO
QMAKE_LFLAGS_RELEASE_WITH_DEBUGINFO = /NODEFAULTLIB:msvcrt.lib /DEBUG /OPT:REF
QMAKE_LFLAGS_DEBUG = /DEBUG
QMAKE_LFLAGS_CONSOLE = /SUBSYSTEM:CONSOLE
QMAKE_LFLAGS_WINDOWS = /SUBSYSTEM:WINDOWS \"/MANIFESTDEPENDENCY:type=\'win32\' name=\'Microsoft.Windows.Common-Controls\' version=\'6.0.0.0\' publicKeyToken=\'6595b64144ccf1df\' language=\'*\' processorArchitecture=\'*\'\"
QMAKE_LFLAGS_DLL = /DLL
QMAKE_LFLAGS_LTCG = /LTCG
QMAKE_LIBS_CORE = kernel32.lib user32.lib shell32.lib uuid.lib ole32.lib advapi32.lib ws2_32.lib
QMAKE_LIBS_GUI = gdi32.lib comdlg32.lib oleaut32.lib imm32.lib winmm.lib winspool.lib ws2_32.lib ole32.lib user32.lib advapi32.lib
QMAKE_LIBS_NETWORK = ws2_32.lib kernel32.lib user32.lib advapi32.lib gdi32.lib crypt32.lib
QMAKE_LIBS_OPENGL = glu32.lib opengl32.lib gdi32.lib user32.lib
QMAKE_LIBS_COMPAT = advapi32.lib shell32.lib comdlg32.lib user32.lib gdi32.lib ws2_32.lib
QMAKE_LIBS_QT_ENTRY = -lqtmain
QMAKE_MOC = $[QT_INSTALL_BINS]\\moc.exe
QMAKE_UIC = $[QT_INSTALL_BINS]\\uic.exe
QMAKE_IDC = $[QT_INSTALL_BINS]\\idc.exe
QMAKE_IDL = midl
QMAKE_LIB = lib /NOLOGO
QMAKE_RC = rc
QMAKE_ZIP = zip -r -9
QMAKE_COPY = copy /y
QMAKE_COPY_DIR = xcopy /s /q /y /i
QMAKE_MOVE = move
QMAKE_DEL_FILE = del
QMAKE_DEL_DIR = rmdir
QMAKE_CHK_DIR_EXISTS = if not exist
QMAKE_MKDIR = mkdir
VCPROJ_EXTENSION = .vcproj
VCSOLUTION_EXTENSION = .sln
VCPROJ_KEYWORD = Qt4VSv1.0
load(qt_config)

TEMPLATE = app CONFIG += console CONFIG -= app_bundle CONFIG -= qt SOURCES += main.cpp INCLUDEPATH += /usr/local/include/opencv LIBS += -L/usr/local/lib \ -lopencv_shape -lopencv_stitching -lopencv_objdetect \ -lopencv_superres -lopencv_videostab -lopencv_calib3d \ -lopencv_features2d -lopencv_highgui -lopencv_videoio \ -lopencv_imgcodecs -lopencv_video -lopencv_photo \ -lopencv_ml -lopencv_imgproc -lopencv_flann -lopencv_core
vim 配置
vim 难记的快捷键
ctrl+e & ctrl+y: 页面滚动
tab --> 4 blank spaces.
:set ts=4
:set expandtab
:%retab!
ctags 难记的快捷键
中文输入法 安装
控制台背景颜色
http://blog.csdn.net/linxing927/article/details/52956983
控制台 usename$hostname:path
https://askubuntu.com/questions/232086/remove-full-path-from-terminal
vim /etc/hostname
关闭Google"阻止了登录尝试"方法, 其他设备也能登录Gmail等谷歌服务
http://blog.csdn.net/sabrecode/article/details/50460382
python环境切换
source activate py27
文件夹的权限添加
sudo chown -R unsw:unsw /usr/local/
打开disk的运行权限
Ubuntu下面挂载windows NTFS格式分区没有可执行权限
端口杀死
lsof -i :8081 列出8081谁在用,然后杀掉,腾出地方。 kill -9 <PID>
Reference
关于编程ing的测试,极其重要!从两处加强:
- *.h
- googletest
- qt creator analysis
未整理
相当不错的技术栈学习计划。
Jobs:
(1)
3D AR应用。
(2)
金融分析。
java服务器 | 基本的back-end服务搭建,以及如何链接amazon api | 只需最基本的要求 |
python工程实践 | 基于机器学习的所有相关编程经验 | 可以考虑结合kaggle,达到精英级别,积累代码 |
Android摄像头 | react native的摄像头相关的编程 | 代码积累 |
神经网路识别 | 模型实践 | 自定义识别经验积累 |
python金融数据编程 | 过一遍书,写成英文博客。 | wordpress,github |
Big Data Engineer - 4 Month Contract - ETL Focus with AWS Glue/Talend
Ideally, you will have strong experience doing ETL (AWS Glue 是一项完全托管的提取、转换和加载 (ETL) 服务) with huge data sets, using tools such as Talend or AWS Glue. In addition, experience with Python, Linux, Shell scripting, Spark and other technologies in the Hadoop ecosystem will be valuable.
This team is responsible for working alongside a leading team of machine learning and data scientists to deliver analytics projects to large enterprises, such as Australia's biggest banks, insurance companies, telcos and retailers. The specific big data engineer contract is to assist with the ETL and data pipeline build with very large data sets.
You should have good knowledge of SQL datawarehouse environments and big data such as; Hadoop, Spark, hdfs, Hive, AWS, Talend/Informatica/AWS Glue, Teradata, amongst other enterprise data warehouse and big data tech.
Work Rights: Must have valid Australian Work Rights
Interviews and Start date: ASAP
Mode: PAYG or PTY Ltd
AI Engineer - Computer Vision
In this role, you will help us build novel architectures for classifying and understanding complex and dynamic visual environments. You will have access to the best sensor data in the world, and an incredible infrastructure for testing and validating your algorithms. We are creating new algorithms for segmentation, tracking, classification, and high-level scene understanding.
We're looking for engineers with advanced degrees and experience building perception pipelines that work with real data in rapidly changing and uncertain environments.
An ideal candidate would be also interested in working on broader artificial intelligence implementations, primarily new algorithms for reinforcement learning in real-world applications.
Qualifications
-
- BS or MS degree in computer science or related field
- Fluency in Python
- Experience with modern computer vision techniques
- Experience with programming and algorithm design
- Demonstrated ability to create functional realtime systems that solve difficult perception tasks
- Experience handling large data sets efficiently
- Strong mathematics skills and understanding of probabilistic techniques
Additional qualifications
-
- Experience with autonomous robots/robotics
- Fluency in C/C++
- Experience with realtime sensor fusion (e.g. lidar, camera, radar)
- Experience with novel pipelines and architectures for convolutional neural nets
- Experience with 3D data and representations (pointclouds, meshes, etc.)
ABOUT Remi
Remi is an Artificial Intelligence firm working on a wide range of complex projects in both industry and academia. Remi aims to bring the next generation of A.I Research and Solutions to market. We're looking for top talent that shares our love of A.I and wants to be part of a fast-paced, growing, and ambitious team.
Data Scientist, Energy - Digital McKinsey
Job description
Qualifications
- MBA or related superior academic achievement in a GEM-related field (such as oil & gas, petrochem, power, mining or similar) is strongly preferred
- 5+ years of experience in progressively leadership roles within the oil & gas, power or mining industry, ideally including logistics, scheduling and maintenance and supporting information technology
- Deep knowledge and expertise in the application of advanced technologies in GEM sector
- Strong knowledge of the competitive landscape
- Familiarity with the use of algorithms, machine learning, data architecture and technology systems deployed in GEM sector related value chains, and how to translate them into business decisions.
- Strong critical thinking and problem-solving skills
- Ability to influence
- Excellent presentation skills, including strong verbal and writing capabilities
Who You'll Work With
You’ll be based in one of our Australian offices and will be a part of Digital McKinsey, working closely our Global Energy & Materials (GEM) practice.
Digital McKinsey brings together the best of McKinsey’s digital capabilities to help our clients use digital technology to transform their businesses. As part of this group, you’ll join a global team working on everything from IT modernization and strategy to agile, cloud, cybersecurity, and digital transformation.
You’ll typically work on projects across all industries and functions and will be fully integrated with the rest of our global firm.You’ll also work with colleagues from across McKinsey & Company to help our clients deliver breakthrough products, experiences, and businesses, both on technology and non-technology topics.
Our office culture is casual, fun and social, with an emphasis on education and innovation. We have the freedom to try new ideas, experiment and are expected to be constantly learning and growing. There is also a strong emphasis on mentoring others in the group, enabling them to grow and learn.
What You'll Do
You will play a key consultative and client-facing role in the development of big data and advanced analytics as a refined, yet evolving, capability of our client teams who advise the top firms in the Global Energy & Materials (GEM) sector.
In this role you will visualize and articulate the possibilities that analytics and digital present within these industry sectors, generating a feeling of excitement and opportunity with our clients. You will have the opportunity to apply solutions in one or more areas, such as predictive maintenance, yield improvement, big data strategy, technological innovation and related areas. You will be expected to bring a recognized and unique knowledge of core business drivers within oil & gas, power and mining sector, including applying data and analytics.
You will exhibit depth of knowledge in advanced analytics, including knowing how to “translate” complex data into easily digestible action items within a business context, external and internal audiences. You will work closely with other data scientists and other analytics focused consultants to build digital solutions, leveraging strong working knowledge of value levers and digital capabilities.
Front End Software Engineer, Chrome OS
As a Front End Software Engineer at Google, you will specialize in building responsive and elegant web applications that scale to millions of users in dozens of languages.
In this role, you will work on Chrome OS, an open source and lightweight operating system designed for speed, simplicity and security, and Chromebooks, a newer, faster computer running Chrome OS as its operating system.
Google is and always will be an engineering company. We hire people with a broad set of technical skills who are ready to take on some of technology's greatest challenges and make an impact on millions, if not billions, of users. At Google, engineers not only revolutionize search, they routinely work on massive scalability and storage solutions, large-scale applications and entirely new platforms for developers around the world. From AdWords to Chrome, Android to YouTube, Social to Local, Google engineers are changing the world one technological achievement after another.
Responsibilities
Design and implement new user-facing features in Google’s products.
Write client-side code for web-based applications, creating fast, easy-to-use, high volume production applications.
Optimize web applications to maximize speed and scale.
Collaborate on scalability issues involving data.
Qualifications
Minimum qualifications:
BA/BS degree in Computer Science or related technical field, or equivalent practical experience.
Experience in web technologies (object-oriented JavaScript, HTML, CSS) and experience with HTML5 and CSS3.
Experience in software development; experience developing web-based applications.
Software development experience in one or more general purpose programming languages.
Preferred qualifications:
Master's Degree or PhD in related field.
Understanding of algorithms and data structures, and their time and space performance
Excellent problem-solving, analytical and troubleshooting skills; ability to work with minimum guidance.
Everything here starts from 2011
优先级低
Android计划书:
函数式编程与RxJava(附demo)【设计模式的必要性】
Android 插件框架机制之预热篇【理解框架】
最好的5个Android ORM框架【理解框架】
Android 必须知道2018年流行的框架库及开发语言,看这一篇就够了!【各种功能框架】
软件工程计划书:
Resource【待搬迁】
Android:
https://github.com/hieuapp/android-firebase-chat【已测试,没voice message】
Voice message相关:
Iphone:
https://github.com/relatedcode/RealtimeChat【功能全,可检测】
Home | BitTiger - 官网
【面试中】系统设计怎么考?系统设计题怎么答? - 系统设计方法
正则表达式判断数字:(整数和浮点)
Regular Expression approach: Check if a given string is a valid number (Integer or Floating Point) in Java
PYTHON
Python的两种退出方式
推荐方式:
import sys try: sys.exit(0) except: print 'die' finally: print 'cleanup'
Python-OpenCV形态学处理
Python下opencv使用笔记(六)(图像的形态学转换)
python-opencv的博客,有必要撸一遍!谢谢!
Python文件IO处理
展示了os模块列表。
文件和文件夹是否存在、创建文件夹、权限判断等。
python计算程序运行时间
- 在 Unix 系统中,建议使用 time.time(),
- 在 Windows 系统中,建议使用 time.clock()。
Python传参
Python调试技巧
# 记录log
logging.debug(...)
logging.info(...)
logging.warn(...)
logging.error(...)
logging.critical(...)
未整理
- 打印数组内的原始信息
n = int( input() )
print(*range(1, n + 1), sep="")
Ref: https://www.hackerrank.com/challenges/python-print/tutorial
- 一个组合:map(<lambda>, <List>)
N = int( input() ) A = [0,1] for i in range(2,N): A.append(A[i-1]+A[i-2]) print map(lambda a: a*a*a, A)[:N]
与reduce比较:可见这是一个pre-result作为当前参数的一种迭代方式。
from functools import reduce
def add(x, y) : # 两数相加 return x + y >>> reduce(add, [1,2,3,4,5]) # 计算列表和:1+2+3+4+5 15 >>> reduce(lambda x, y: x+y, [1,2,3,4,5]) # 使用 lambda 匿名函数 15
两者的结合
def charToNumber(s): def charToNum(str): return int(str) def numToNumber(x,y): return x+y return reduce(numToNumber,list(map(charToNum,s))) print(charToNumber('345789'))
- 正则表达式 筛选邮件地址
import re mailList = list()
for i in range(int(input())): mailLst.append(input())
print ( sorted(list( filter(lambda x: re.search(r'^[\w\d-]+@[A-Za-z0-9]+\.\w?\w?\w$', x), mailList) )) )
- filter: 理解为只接受“返回true/false”函数的map;
- re.search(pattern, string, flags)
- 正则表达式:30分钟入门教程
JAVA
时间操作
多态
C++中,如果父类中的函数前边标有virtual,才显现出多态。
如果父类func是virtual的,则
Super *p =new Sub(); p->func(); // 调用子类的func
如果不是virtual的,p->func将调用父类原来的函数。
Java中,不管写不写virtual都是多态的,子类的同名函数会override父类的。
【推荐】国内首个AI IDE,深度理解中文开发场景,立即下载体验Trae
【推荐】编程新体验,更懂你的AI,立即体验豆包MarsCode编程助手
【推荐】抖音旗下AI助手豆包,你的智能百科全书,全免费不限次数
【推荐】轻量又高性能的 SSH 工具 IShell:AI 加持,快人一步
· AI与.NET技术实操系列:向量存储与相似性搜索在 .NET 中的实现
· 基于Microsoft.Extensions.AI核心库实现RAG应用
· Linux系列:如何用heaptrack跟踪.NET程序的非托管内存泄露
· 开发者必知的日志记录最佳实践
· SQL Server 2025 AI相关能力初探
· 震惊!C++程序真的从main开始吗?99%的程序员都答错了
· 【硬核科普】Trae如何「偷看」你的代码?零基础破解AI编程运行原理
· 单元测试从入门到精通
· 上周热点回顾(3.3-3.9)
· winform 绘制太阳,地球,月球 运作规律