papering - 博客园

2017年11月1日

Dropout: A Simple Way to Prevent Neural Networks fromOverfitting

摘要： https://www.cs.toronto.edu/~hinton/absps/JMLRdropout.pdf Deep neural nets with a large number of parameters are very powerful machine learning systems 阅读全文

posted @ 2017-11-01 15:25 papering 阅读(590) 评论(0) 推荐(0) 编辑

Write Custom Java to Create LZO Files

摘要： https://cwiki.apache.org/confluence/display/Hive/LanguageManual+LZO LanguageManual LZO LanguageManual LZO LanguageManual LZO Skip to end of metadata C 阅读全文

posted @ 2017-11-01 00:03 papering 阅读(402) 评论(0) 推荐(0) 编辑

2017年10月31日

Avro schemas are defined with JSON . This facilitates implementation in languages that already have JSON libraries.

摘要： https://avro.apache.org/docs/current/ Introduction Apache Avro™ is a data serialization system. Avro provides: Rich data structures. A compact, fast, 阅读全文

posted @ 2017-10-31 23:45 papering 阅读(195) 评论(0) 推荐(0) 编辑

Storage Types and Storage Policies

摘要： https://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-hdfs/ArchivalStorage.html Introduction Archival Storage is a solution to decouple gr 阅读全文

posted @ 2017-10-31 23:38 papering 阅读(290) 评论(0) 推荐(0) 编辑

splittability A SequenceFile can be split by Hadoop and distributed across map jobs whereas a GZIP file cannot be.

摘要： splittability CompressedStorage CompressedStorage CompressedStorage Skip to end of metadata Created by Confluence Administrator, last modified by Left 阅读全文

posted @ 2017-10-31 23:26 papering 阅读(301) 评论(0) 推荐(0) 编辑

This means that only a small number of nodes must be read from disk to retrieve an item.

摘要： http://cis.stvincent.edu/html/tutorials/swd/btree/btree.html Introduction A B-tree is a specialized multiway tree designed especially for use on disk. 阅读全文

posted @ 2017-10-31 21:51 papering 阅读(161) 评论(0) 推荐(0) 编辑

Apache Kafka® is a distributed streaming platform. What exactly does that mean?

摘要： https://kafka.apache.org/intro.html 阅读全文

posted @ 2017-10-31 17:03 papering 阅读(153) 评论(0) 推荐(0) 编辑

(t,p,o) t:p>=o there cannot be more consumer instances in a consumer group than partitions

摘要： https://kafka.apache.org/intro.html Kafka as a Messaging System How does Kafka's notion of streams compare to a traditional enterprise messaging syste 阅读全文

posted @ 2017-10-31 12:03 papering 阅读(189) 评论(0) 推荐(0) 编辑

Goldilocks

摘要： limit 阅读全文

posted @ 2017-10-31 11:22 papering 阅读(146) 评论(0) 推荐(0) 编辑

Sqoop

摘要： rmds mapper 阅读全文

posted @ 2017-10-31 11:21 papering 阅读(113) 评论(0) 推荐(0) 编辑

Spark SQL includes a cost-based optimizer, columnar storage and code generation to make queries fast.

摘要： https://spark.apache.org/sql/ Performance & Scalability Spark SQL includes a cost-based optimizer, columnar storage and code generation to make querie 阅读全文

posted @ 2017-10-31 00:10 papering 阅读(162) 评论(0) 推荐(0) 编辑

2017年10月30日

Sparksql 取代 Hive？

摘要： sparksql hive https://databricks.com/blog/2014/07/01/shark-spark-sql-hive-on-spark-and-the-future-of-sql-on-spark.html https://cwiki.apache.org/conflu 阅读全文

posted @ 2017-10-30 23:43 papering 阅读(674) 评论(0) 推荐(0) 编辑

Eliminates these repeated computation in multi aggregations query

摘要： https://github.com/pingcap/tispark/commit/dcca23bfa1aa0c356a4280d82bc8301c0de08318 阅读全文

posted @ 2017-10-30 20:56 papering 阅读(221) 评论(0) 推荐(0) 编辑

rule-based optimizer cost-based optimizer

摘要： SQL processing uses the following main components to execute a SQL query: The Parser checks both syntax and semantic analysis. The Optimizer uses cost 阅读全文

posted @ 2017-10-30 20:01 papering 阅读(465) 评论(0) 推荐(0) 编辑

Groovy

摘要： https://en.wikipedia.org/wiki/Groovy_(programming_language) Groovy offers simple, consistent syntax for handling lists and maps, reminiscent of Java's 阅读全文

posted @ 2017-10-30 19:54 papering 阅读(200) 评论(0) 推荐(0) 编辑

Interface AutoCloseable

摘要： https://docs.oracle.com/javase/tutorial/essential/exceptions/try.html https://docs.oracle.com/javase/8/docs/api/java/lang/AutoCloseable.html An object 阅读全文

posted @ 2017-10-30 17:20 papering 阅读(239) 评论(0) 推荐(0) 编辑

throws ? catch checked unchecked

摘要： ThrowableClass Error (unchecked) Exception IOException (checked) RuntimeException (unchecked) 阅读全文

posted @ 2017-10-30 14:58 papering 阅读(171) 评论(0) 推荐(0) 编辑

18 ？ 21

摘要： This is for MyHDFSCat = 18+3 阅读全文

posted @ 2017-10-30 10:04 papering 阅读(318) 评论(0) 推荐(0) 编辑

2017年10月29日

$wc

摘要： $ wc --helpUsage: wc [OPTION]... [FILE]... or: wc [OPTION]... --files0-from=FPrint newline, word, and byte counts for each FILE, and a total line ifmo 阅读全文

posted @ 2017-10-29 16:43 papering 阅读(250) 评论(0) 推荐(0) 编辑

file descriptor 0 1 2 一切皆文件 stdout stderr stdin /dev/null 沉默是金 pipes 禁止输出屏蔽 stdout 和 stderr 输入输出重定向重定向文件描述符

摘要：读取标准输入 #include "thread.h" #include "thread-sync.h" int n, count = 0; mutex_t lk = MUTEX_INIT(); void Tproduce() { while (1) { retry: mutex_lock(&lk); 阅读全文

posted @ 2017-10-29 16:08 papering 阅读(836) 评论(0) 推荐(0) 编辑

# $ # root

摘要： # 阅读全文

posted @ 2017-10-29 15:59 papering 阅读(127) 评论(0) 推荐(0) 编辑

inode ls -li 显示索引节点

摘要： ls 阅读全文

posted @ 2017-10-29 14:04 papering 阅读(436) 评论(0) 推荐(0) 编辑

again shell

摘要： sh UNIX shell Bourne Again Shell 阅读全文

posted @ 2017-10-29 11:27 papering 阅读(154) 评论(0) 推荐(0) 编辑

操作系统资源管理 zookeeper yarn 进程管理分布式 yarn诞生背景

摘要： zookeeper 信息保管员 YARN 简介 https://www.ibm.com/developerworks/cn/data/library/bd-yarn-intro/index.html 阅读全文

posted @ 2017-10-29 11:10 papering 阅读(227) 评论(0) 推荐(0) 编辑

2017年10月28日

Xen

摘要： https://www.xenproject.org/ The Xen ProjectTM is the leading open source virtualization platform that is powering some of the largest clouds in produc 阅读全文

posted @ 2017-10-28 21:13 papering 阅读(187) 评论(0) 推荐(0) 编辑

I/O 复用 multiplexing data race 同步 coroutine 协程

摘要：小结： 1、A file descriptor is considered ready if it is possible to perform the corresponding I/O operation (e.g., read(2)) without blocking. UNIX网络编程卷1：阅读全文

posted @ 2017-10-28 12:58 papering 阅读(333) 评论(0) 推荐(0) 编辑

flume topology design . tier num 分层数目

摘要： 32：+：1 x：1 x＜＝8 https://flume.apache.org/FlumeUserGuide.html#flume-topology-design Flume topology design 【拓扑分层原因 0-缓冲1-路由】 The first step in designin 阅读全文

posted @ 2017-10-28 09:26 papering 阅读(218) 评论(0) 推荐(0) 编辑

Page (computer memory) Memory segmentation Page table 虚拟地址到物理地址的转换

摘要： A page, memory page, or virtual page is a fixed-length contiguous block of virtual memory, described by a single entry in the page table. It is the sm 阅读全文

posted @ 2017-10-28 02:48 papering 阅读(537) 评论(0) 推荐(0) 编辑

2017年10月27日

service oriented architecture 构造分布式计算的应用程序的方法面向服务的架构分解技术

摘要： zh.wikipedia.org/wiki/面向服务的架构【程序功能做为服务】面向服务的体系结构（英语：service-oriented architecture）是构造分布式計算的应用程序的方法。它将应用程序功能作为服务发送给最终用户或者其他服务。它采用开放标准、与软件资源进行交互并采用表示阅读全文

posted @ 2017-10-27 22:31 papering 阅读(318) 评论(0) 推荐(0) 编辑

WildFly JBoss 应用程序服务器

摘要： https://en.wikipedia.org/wiki/WildFly 【实现基于面向服务的架构SOA的web应用和服务】 WildFly,[1] formerly known as JBoss AS, or simply JBoss, is an application server auth 阅读全文

posted @ 2017-10-27 22:23 papering 阅读(236) 评论(0) 推荐(0) 编辑

Object/Relational Mapping 数学关系反面向对象

摘要：【hibernate ORM 是对象关系映射框架事实上的持久化存储引擎】 http://docs.jboss.org/hibernate/orm/5.2/userguide/html_single/Hibernate_User_Guide.html Object/Relational Mappin 阅读全文

posted @ 2017-10-27 21:46 papering 阅读(253) 评论(0) 推荐(0) 编辑

non-blocking I/O

摘要： https://en.wikipedia.org/wiki/New_I/O_(Java) zh.wikipedia.org/wiki/Java_NIO Java NIO API提供在java.nio套件或其子套件中。緩衝區的基本資料類型字元集的編碼器和解碼器基於Perl風格的正則表達式（在 j 阅读全文

posted @ 2017-10-27 21:25 papering 阅读(172) 评论(0) 推荐(0) 编辑

jetty netty

摘要： jetty netty 阅读全文

posted @ 2017-10-27 16:54 papering 阅读(275) 评论(0) 推荐(0) 编辑

Azkaban_Oozie_action

摘要： http://azkaban.github.io/azkaban/docs/2.5/ There is no reason why MySQL was chosen except that it is a widely used DB. We are looking to implement com 阅读全文

posted @ 2017-10-27 16:50 papering 阅读(228) 评论(0) 推荐(0) 编辑

权限过大 ssh协议通过pem文件登陆

摘要： root@oneweek:~# ssh -i uat.pem root@110.5.15.6@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ WARNING: UNPROTECTED PRIVATE KEY FILE! @@@@ 阅读全文

posted @ 2017-10-27 16:06 papering 阅读(457) 评论(0) 推荐(0) 编辑

交易准实时预警 kafka topic 主题异常交易主题低延迟 event topic alert topic 内存算法测试

摘要： https://www.ibm.com/developerworks/cn/opensource/os-cn-kafka/index.html 周明耀2015 年 6 月 10 日发布示例：网络游戏示例：网络游戏假设我们正在开发一个在线网络游戏平台，这个平台需要支持大量的在线用户实时操作，玩阅读全文

posted @ 2017-10-27 14:46 papering 阅读(562) 评论(0) 推荐(0) 编辑

flink_action 多级别的抽象同时处理无界限（流）和有界限（批处理）数据集

摘要：实践 3）通过连表mysql，字段值IS NOT NULL，实现数据分流 CREATE TEMPORARY TABLE `nocache_table_test` ( `device_id` STRING) WITH ( 'connector' = 'rds', 'username' = 'acfo 阅读全文

posted @ 2017-10-27 14:43 papering 阅读(269) 评论(0) 推荐(0) 编辑

netty

摘要： netty Netty: Home http://netty.io/ Netty is an asynchronous event-driven network application framework for rapid development of maintainable high perf 阅读全文

posted @ 2017-10-27 13:31 papering 阅读(208) 评论(0) 推荐(0) 编辑

1970 ted codd

摘要： 1970 阅读全文

posted @ 2017-10-27 12:54 papering 阅读(146) 评论(0) 推荐(0) 编辑

Data dictionary

摘要： Data dictionary 阅读全文

posted @ 2017-10-27 10:11 papering 阅读(148) 评论(0) 推荐(0) 编辑