Java系列整理---Python Java Scala 区别
本文仅从语法知识点上阐述三者之间的区别，因为该三种语言在现在工作中处理大数据时用的较多，简单记录一下，并逐步丰富与巩固
1. 基本数据结构用法
1.1 Array 数组
I. Python
主要是见于Numpy,同时Pandas中的DataFrame一起处理数据
II.Java
III.Scala
1.2 List 列表
I. Python
II.Java
III.Scala
1.3 Set集合
I. Python
II.Java
III.Scala
1.4 Dict字典、Map映射
I. Python
II.Java
III.Scala
Java的List、Set、Map
class TestDataStruct {
// List<T>, ArrayList<T>;
// Set<E>, HashSet<T>
// Map<String, T>, HashMap<String, T>
public void testList() {
List<String> list = new ArrayList<String>();
list.add("aaa");
list.add("bbb");
list.add("ccc");
System.out.println(list);
for(String sss: list) {
System.out.println(sss);
}
System.out.println(list.contains("aaa"));
Boolean s1 = list.remove("aaa");
String s2 = list.remove(0);
System.out.println(list);
}
public void testSet() {
Set<String> set = new HashSet<String>();
set.add("aaa");
set.remove("aaa");
set.add("bbb");
for(String sss: set) {
System.out.println(sss);
}
}
public void testMap() {
Map<String, Object> map = new HashMap<String, Object>();
map.put("aaa", "111");
map.put("bbb", "222");
map.remove("aaa");
for(Map.Entry<String, Object> entry: map.entrySet()) {
System.out.println("Key = " + entry.getKey() + ", Value = " + entry.getValue());
}
}
}
Scala的List、Set、Map
其中 Scala的可以来源于java包，或来自 scala.collection.mutable
class testBaseDataStruct {
def testList(): Unit={
var list1:List[String] = List[String]("111", "222")
list1 :+= "ddd"
list1 :+= "ccc"
list1.foreach((item: String) => println(item))
println(list1.drop(2))
var list:util.List[String] = new util.ArrayList[String]()
list.add("aaa");
list.add("bbb");
list.forEach((item: String) => println(item))
}
def testSet(): Unit={
var set: Set[String] = Set[String]("###", "222")
set += ("aaa")
set.foreach((item: String) => println(item))
var set2:mutable.Set[String] = mutable.Set[String]()
set2.add("ttt")
set2.add("####")
set.foreach((item: String) => println(item))
}
def testMap(): Unit={
var map: Map[String, Any] = Map[String, Any]()
map += ("zzz" -> "000000")
map += ("ttt" -> "11111")
map.foreach(println)
var map2: mutable.Map[String, Any] = mutable.HashMap[String, Any]()
map2.put("aaa", "1111111111")
map2.put("bbb", "2222222222")
println(map2)
map2.remove("aaa")
println(map2)
}
}
2. 流程控制语句
2.1 if/else
2.2 while/for
I. Python
II.Java
III.Scala
2.4 switch(Java)与match(Scala)
I. Python
使用if/else实现
II.Java
III.Scala
3. 面向对象
1. 类是抽象的，具有相同属性和方法(行为)的集合
2. 对象是具体的，具有某些属性和方法(行为)的
3. 创建的对象的过程，叫实例化
3.1 类的初始化或类的构造函数
I. Python
II.Java
III.Scala
3.2 类的访问控制
主要涉及 public、protect、privite在父类和子类中的访问范围
Public: 所有类都可见；三种语言概念一致
Protect: python与scala当前类及子类都可以访问，java是当前类及子类都可以访问，同时所在的同名的包中的类也可以访问
Privite: 仅限当前类，子类不可访问，且不可以被重写；三种语言概念一致
I. Python
Public：变量不是_、__开头
Protect：_ 单下划线表示， 例如： _instance
Privite：__双下划线表示例如：__instance
II.Java
略
III.Scala
略
3.3 类的属性与方法
I. Python
有静态属性/方法，实例(动态)属性/方法，类属性/方法, 且都可以被重写
II.Java
有静态属性/方法，实例(动态)属性/方法
java 中类属性需要先定义，才能在具体函数中使用！！！; interface 中一般不定义属性，更多的是接口函数,仅有方法名
静态属性/方法不能被直接重写，实例属性/方法需要在构造函数中才能重写
III.Scala
没有静态属性/方法，动态属性/方法的概念
但是有实例属性/方法(相当于实例属性/方法) 和 对象属性/方法(相当于静态属性/方法)，且都可以被重写
3.4 接口
I. Python
通过定义类实现，方法体可有可无
II.Java
通过interface定义实现，且无方法体，一般仅定义方法，因为接口中定义的属性后不能被重写
III.Scala
通过trait定义实现，方法体可有可无
3.5 继承
三者概念基本一致:
1).继承了父类的基本属性和方法
2).可以继续实现自己的属性和方法
3).方法重写：同名函数重写，用另一种实现方案实现父类的方法
I. Python
有单继承、多重继承、多层继承，都是通过class类本身定义
II.Java
有单继承、多层继承，通过class类本身定义
但多重继承是通过继承多个接口即多个interface
III.Scala
同Java概念
3.6 重写与重载
重写：子类继承父类后对父类的属性或方法进行重写，
同时基类的私有方法能被子类重写
重载：同一个类中，方法名相同，但是参数不一样(参数类型、数量)
I. Python
仅重写，没有重载
II.Java
静态字段不能被重写，实例方法仅在构造函数中才能被重写
III.Scala
必须使用override修饰
3.7 多态
多态：
目的是为了让代码更加，降低耦合
有继承或实现特质（接口）
父类引用指向子类对象或接口指向实现类
方法需要重写
三者概念一致, 略
I. Python
II.Java
III.Scala
下面给出三种不同语言对工厂类模式的实现，1）继承 2）属性、方法是否可被重写
Python工厂类实现
# coding=utf-8
import re
class BaseSite:
url_patterns = []
def __init__(self):
super(BaseSite, self).__init__()
def process(self, url, html):
params = {
'url': url,
'html': html,
'params': ''
}
return params
class Douban(BaseSite):
url_patterns = ['https://movie.douban.com']
def __init__(self):
super(BaseSite, self).__init__()
def process(self, url, html):
params = {
'url': url,
'html': html,
'params': 'douban'
}
return params
class Tencent(BaseSite):
url_patterns = ['https?://www.qq.com']
def __init__(self):
super(BaseSite, self).__init__()
def process(self, url, html):
params = {
'url': url,
'html': html,
'params': 'qq'
}
return params
class Factory:
def __init__(self):
self.site_list = []
def init_factory(self):
self.site_list.append(Douban())
self.site_list.append(Tencent())
def get_site(self, url):
for site in self.site_list:
for pattern in site.url_patterns:
if re.search(pattern, url):
return site
return BaseSite()
if __name__ == '__main__':
factory = Factory()
factory.init_factory()
url = 'https://www.qq.com'
html = '<html></html>'
site = factory.get_site(url)
params = site.process(url, html)
print(params)
Java工厂类实现
import java.util.ArrayList;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.regex.*;
class Site {
// 静态属性/方法不能被直接重写，实例属性/方法需要在构造函数中才能操作(近似重写)
List<String> urlPatterns = new ArrayList<String>();
// 初始化时，不要设置值，否则会被继承到子类
// public Site() {
// this.urlPatterns.add("https?://www.common.com");
// }
public Map<String, Object> process(String url, String html) {
Map<String, Object> map = new HashMap<String, Object>();
// map.put("url", url);
// map.put("html", html);
return map;
}
}
class Baidu extends Site {
// 静态属性/方法不能被直接重写，实例属性/方法需要在构造函数中才能操作(近似重写)
public Baidu() {
this.urlPatterns.add(".*baidu.*");
}
public Map<String, Object> process(String url, String html) {
Map<String, Object> map = new HashMap<String, Object>();
map.put("url", url);
map.put("html", html);
map.put("params", "baidu");
return map;
}
}
class Tencent extends Site {
// 静态属性/方法不能被直接重写，实例属性/方法需要在构造函数中才能操作(近似重写)
public Tencent() {
this.urlPatterns.add(".*qq.*");
}
public Map<String, Object> process(String url, String html) {
Map<String, Object> map = new HashMap<String, Object>();
map.put("url", url);
map.put("html", html);
map.put("params", "qq");
return map;
}
}
class SiteFactory {
List<Site> siteList = new ArrayList<Site>();
public void initFactory() {
this.siteList.add(new Baidu());
this.siteList.add(new Tencent());
}
public Site getSite(String url) {
Site tempSite = new Site();
for(Site site: this.siteList) {
for(String urlPattern: site.urlPatterns) {
if(Pattern.matches(urlPattern, url)) {
return site;
}
}
}
return tempSite;
}
}
public class Factory {
public static void main(String[] args) {
SiteFactory siteFactory = new SiteFactory();
siteFactory.initFactory();
String url = "https://www.baidu.com";
Site site = siteFactory.getSite(url);
System.out.println(site);
Map<String, Object> map = site.process(url, "<html></html>");
for(Map.Entry<String, Object> entry : map.entrySet()) {
System.out.println("Key = " + entry.getKey() + ", Value = " + entry.getValue());
}
}
public void test() {
Site site = new Site();
System.out.println(site.urlPatterns);
Map<String, Object> map = site.process("https://www.baidu.com", "<html></html>");
for(Map.Entry<String, Object> entry : map.entrySet()) {
System.out.println("Key = " + entry.getKey() + ", Value = " + entry.getValue());
}
}
}
Scala工厂类实现
import scala.collection.mutable
import scala.util.matching.Regex
class BaseSite {
// 属性和方法都可以被重写
val urlPatterns = List[String]()
def process(url: String, html: String): mutable.Map[String, Any]={
var map = mutable.HashMap[String, Any]()
map.put("url", url)
map.put("html", html)
map.put("params", "")
map
}
}
class BaseBaidu extends BaseSite {
// 属性和方法都可以被重写
override val urlPatterns = List[String](".*baidu.*")
override def process(url: String, html: String): mutable.Map[String, Any] = {
var map = super.process(url, html)
map.put("params", "baidu")
map
}
}
class BaseFactoryList {
var siteList = List[BaseSite]()
def initFactory(): Unit= {
siteList = siteList :+ new BaseBaidu()
}
def getSite(url: String): BaseSite = {
var tempSite: BaseSite = new BaseSite()
println(siteList)
for (site: BaseSite <- siteList) {
for (urlPattern: String <- site.urlPatterns) {
val pattern = new Regex(urlPattern)
tempSite = site
if (pattern.matches(url)) {
return site
}
}
}
tempSite
}
}
object BaseFactory {
def main(args: Array[String]): Unit = {
val siteFactory: BaseFactoryList = new BaseFactoryList()
siteFactory.initFactory()
val url: String = "https://www.baidu.com"
val site: BaseSite = siteFactory.getSite(url)
val map = site.process(url, "<html></html>")
map.foreach(println)
}
// 缺少对各种数据结构的操作
}
4.其他模块封装
4.1 时间处理封装
package com.jsw.kg; import java.text.DateFormat; import java.text.ParseException; import java.text.SimpleDateFormat; import java.util.Date; import java.util.Calendar; public class TimeUtil { public static String UniformFormat = "yyyy-MM-dd HH:mm:SS"; public static String timeFormat(Date date, String format) { DateFormat fm = new SimpleDateFormat(format); return fm.format(date); } public static String timeFormat(Date date) { return TimeUtil.timeFormat(date, UniformFormat); } public static String timeFormat() { return TimeUtil.timeFormat(new Date(), UniformFormat); } public static String timeFormat(Calendar calendar, String format) { Date date = calendarToDate(calendar); return TimeUtil.timeFormat(date, format); } public static String timeFormat(Calendar calendar) { return TimeUtil.timeFormat(calendar, UniformFormat); } /** * 时间转换： 字符串转Date时间, 然后再转其他时间字符串 */ public static String timeFormat(String source, String sourceFormat, String format) { Date date = timeStringToDate(source, sourceFormat); return timeFormat(date, format); } /** * 时间转换： 字符串转Date时间, 然后再转默认通用时间字符串 */ public static String timeFormat(String source, String sourceFormat) { return timeFormat(source, sourceFormat, UniformFormat); } /** * 时间转换： 字符串转Date时间 */ public static Date timeStringToDate(String source, String sourceFormat) { DateFormat fm = new SimpleDateFormat(sourceFormat); Date date = null; try { date = fm.parse(source); } catch (ParseException e) { e.printStackTrace(); } return date; } public static Calendar dateToCalendar(Date date) { Calendar calendar = Calendar.getInstance(); calendar.setTime(date); return calendar; } public static Date calendarToDate(Calendar calendar) { Date date = calendar.getTime(); return date; } public static long timeStamp(Date date) { return date.getTime(); } public static long timeStamp(Calendar calendar) { // return calendar.getTimeInMillis(); return timeStamp(calendarToDate(calendar)); } public static long timeStamp() { return timeStamp(new Date()); } public static Date timeStampToDate(long time) { return new Date(time); } public static void main(String[] args) { Date date = new Date(); System.out.println(timeFormat(date)); System.out.println(timeStamp(date)); System.out.println(timeStamp(timeStringToDate("2005-06-09", "yyyy-MM-dd"))); System.out.println("###############"); System.out.println(new Date().getTime()); System.out.println(timeStampToDate(1386665666777L)); } }
4.2 正则封装
scala代码
package com.jsw.kg import scala.util.matching.Regex object ScalaRegUtil { def search(pattern: String, string: String): Boolean={ val r:Regex = new Regex(pattern) r.pattern.matcher(string).matches } def findAll(pattern: String, string: String): List[String]={ val r:Regex = new Regex(pattern) var list = List[String]() if(r.pattern.matcher(string).matches) { val m = r.findAllIn(string) while(m.hasNext) { list = list :+ m.next() } } list } def replaceAll(string: String, pattern: String, replace: String): String = { val r:Regex = new Regex(pattern) r.replaceAllIn(string, replace) } def main(args: Array[String]): Unit = { val s = ScalaRegUtil.search("(abc).*", "abcdef") println(s) val s1 = ScalaRegUtil.replaceAll("abcdef", "(abc).*", "aaa") println(s1) val s2 = ScalaRegUtil.findAll("(abc).*", "abcdef") println(s2) } }
Java代码
package com.jsw.kg; import java.util.ArrayList; import java.util.List; import java.util.regex.Matcher; import java.util.regex.Pattern; public class JavaRegUtil { public static Matcher getMatcher(String pattern, String string) { Pattern r = Pattern.compile(pattern); Matcher m = r.matcher(string); return m; } public static Boolean search(String pattern, String string) { return JavaRegUtil.getMatcher(pattern, string).matches(); } public static List<String> findAll(String pattern, String string) { List<String> items = new ArrayList<String>(); Matcher m = JavaRegUtil.getMatcher(pattern, string); if(m.matches()) { int count = m.groupCount(); for(int i=1; i<=count; i++) { items.add(m.group(i)); } } return items; } public static String replaceAll(String string, String pattern, String replace) { Matcher m = JavaRegUtil.getMatcher(pattern, string); if(m.matches()) { return m.replaceAll(replace); } return ""; } public static void main(String[] args) { String line = "This order was placed for QT3000! OK?"; String pattern = ".*(order.*placed).*"; System.out.println(JavaRegUtil.search(pattern, line)); System.out.println(JavaRegUtil.findAll(pattern, line)); } }
4.3 文件处理封装
scala代码
package com.jsw.kg; import java.io.{BufferedWriter, File, FileWriter} import scala.io.Source object ScalaFileUtil{ def readFile(path: String): Iterator[String]={ val f = Source.fromFile(path) for (line: String <- f.getLines()) yield line } def wrieFile(path: String, lines: List[String], append: Boolean=false): Unit = { val writer = new BufferedWriter(new FileWriter(path, append)) writer.write(lines.mkString("\n") + "\n") writer.close() } def getFileListObj(dir: File): Array[File] = { val fp = dir.listFiles val d = fp.filter(_.isDirectory) val f = fp.filter(_.isFile) f ++ d.flatMap(getFileListObj(_)) } def getFileList(dir: String): List[String]={ getFileListObj(new File(dir)).map(_.getPath).toList } def remove(path: String): Boolean={ return JavaFileUtil.remove(path) } def main(args: Array[String]): Unit = { val path = "E:\\LocalCode\\allcode" getFileList(path).foreach(println) // writeFile(path, List("aaa", "bbb", "ccc")) // val lines = readFile(path) // lines.foreach(println) } }
Java代码
package com.jsw.kg; import java.io.*; import java.util.ArrayList; import java.util.List; public class JavaFileUtil { public static List<String> readFile(String path) throws IOException { List<String> lines = new ArrayList<String>(); if(!exists(path)) { return lines; } BufferedReader reader = new BufferedReader(new FileReader(path)); String line = reader.readLine(); while (line != null) { line = reader.readLine(); lines.add(line); } reader.close(); return lines; } public static void writeFile(String path, List<String> lines) throws IOException { if(!exists(path)) { return; } BufferedWriter writer = new BufferedWriter(new FileWriter(path)); for (String line: lines){ writer.write(line); } writer.close(); } public static Boolean exists(String path) { File file = new File(path); //获取其file对象 return file.exists(); } public static List<String> getFileList(String path, List<String> files) { if (!exists(path)) { return files; } File file = new File(path); //获取其file对象 File[] fs = file.listFiles(); //遍历path下的文件和目录，放在File数组中 for(File f: fs) { //遍历File[]数组 if(f.isDirectory()) getFileList(f.getPath(), files); else { files.add(f.getPath()); } } return files; } public static void mkdir(String path) { File file = new File(path); // 现在创建目录 file.mkdirs(); } public static boolean remove(String path) { if(!exists(path)) { return false; } File file = new File(path); // 现在创建目录 if(file.isFile()) { System.out.println("delete file: " + file.getPath()); return file.delete(); } else { for(File f: file.listFiles()) { remove(f.getPath()); } System.out.println("delete dir: " + file.getPath()); return file.delete(); } } public static void main(String[] args) { String path = "E:\\Program Files\\eclipse\\kg\\src"; // JavaFileUtil.remove(path); List<String> files = new ArrayList<String>(); JavaFileUtil.getFileList(path, files); for (String file: files) { System.out.println(file); } } }
注意事项：
1）关于创建maven时可以指定架构：
目录级别：project--->package--->class(object) 使用maven时，可以统一基本代码框架
2）关于maven打包指定mainClass, pom.xml中添加如下配置：
<build> <plugins> <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-jar-plugin</artifactId> <version>3.0.2</version> <configuration> <archive> <manifest> <addClasspath>true</addClasspath> <!-- 此处为程序主入口--> <mainClass>com.jsw.kg.App</mainClass> </manifest> </archive> </configuration> </plugin> </plugins> </build>
3）IDEA打包设置如下：
本地实现具体如下：(分开打包)
正常运行如下：
https://github.com/jiangsiwei2018/BigData.git
实例代码git仓库地址
