Quartz分布式定时任务
前言：
项目需要执行定时任务,该类定时任务只需要实现类似Spring原生的@Scheudle注解的定时方法即可，无需考虑分片、刷新及重启，且因项目是多实例，所以需要考虑实现分布式，考察了目前开源的几款分布式定时任务产品
xxl-job 需要部署server端，考虑项目不需要依赖这么重的第三方的server，所以被pass
elastic-job-lite 这个需要的zk版本要3.6以上 公司已有的zk环境是3.4.5 所以不想再单独部署zk，所以也被pass
quartz 分布式定时任务是依靠数据库悲观锁的机制实现的，只需要依赖数据库，项目本身是有数据库存储依赖的，对项目的侵入性较小
以下为改造过程
-
引入maven依赖
<dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-quartz</artifactId> <version>2.3.2.RELEASE</version> </dependency> <dependency> <groupId>org.projectlombok</groupId> <artifactId>lombok</artifactId> </dependency> <dependency> <groupId>cn.hutool</groupId> <artifactId>hutool-all</artifactId> <version>5.8.25</version> </dependency> -
增加配置application.properties
# 定时任务配置 spring.quartz.job-store-type=jdbc spring.quartz.wait-for-jobs-to-complete-on-shutdown=true spring.quartz.overwrite-existing-jobs=true spring.quartz.startup-delay=10S spring.quartz.properties.org.quartz.scheduler.instanceName=exampleInstanceName spring.quartz.properties.org.quartz.scheduler.instanceId=AUTO spring.quartz.properties.org.quartz.jobStore.class=org.quartz.impl.jdbcjobstore.JobStoreTX spring.quartz.properties.org.quartz.jobStore.driverDelegateClass=org.quartz.impl.jdbcjobstore.StdJDBCDelegate spring.quartz.properties.org.quartz.jobStore.tablePrefix=QRTZ_ spring.quartz.properties.org.quartz.jobStore.useProperties=false spring.quartz.properties.org.quartz.jobStore.isClustered=true spring.quartz.properties.org.quartz.jobStore.clusterCheckinInterval=5000 spring.quartz.properties.org.quartz.threadPool.class=org.quartz.simpl.SimpleThreadPool spring.quartz.properties.org.quartz.threadPool.threadCount=10 spring.quartz.properties.org.quartz.threadPool.threadPriority=5 spring.quartz.properties.org.quartz.threadPool.threadsInheritContextClassLoaderOfInitializingThread=true -
执行sql
项目中存在数据库的连接配置,存在spring-boot-starter-jdbc依赖
# # In your Quartz properties file, you'll need to set # org.quartz.jobStore.driverDelegateClass = org.quartz.impl.jdbcjobstore.StdJDBCDelegate # # # By: Ron Cordell - roncordell # I didn't see this anywhere, so I thought I'd post it here. This is the script from Quartz to create the tables in a MySQL database, modified to use INNODB instead of MYISAM. DROP TABLE IF EXISTS QRTZ_FIRED_TRIGGERS; DROP TABLE IF EXISTS QRTZ_PAUSED_TRIGGER_GRPS; DROP TABLE IF EXISTS QRTZ_SCHEDULER_STATE; DROP TABLE IF EXISTS QRTZ_LOCKS; DROP TABLE IF EXISTS QRTZ_SIMPLE_TRIGGERS; DROP TABLE IF EXISTS QRTZ_SIMPROP_TRIGGERS; DROP TABLE IF EXISTS QRTZ_CRON_TRIGGERS; DROP TABLE IF EXISTS QRTZ_BLOB_TRIGGERS; DROP TABLE IF EXISTS QRTZ_TRIGGERS; DROP TABLE IF EXISTS QRTZ_JOB_DETAILS; DROP TABLE IF EXISTS QRTZ_CALENDARS; CREATE TABLE QRTZ_JOB_DETAILS( SCHED_NAME VARCHAR(120) NOT NULL, JOB_NAME VARCHAR(190) NOT NULL, JOB_GROUP VARCHAR(190) NOT NULL, DESCRIPTION VARCHAR(250) NULL, JOB_CLASS_NAME VARCHAR(250) NOT NULL, IS_DURABLE VARCHAR(1) NOT NULL, IS_NONCONCURRENT VARCHAR(1) NOT NULL, IS_UPDATE_DATA VARCHAR(1) NOT NULL, REQUESTS_RECOVERY VARCHAR(1) NOT NULL, JOB_DATA BLOB NULL, PRIMARY KEY (SCHED_NAME,JOB_NAME,JOB_GROUP)) ENGINE=InnoDB; CREATE TABLE QRTZ_TRIGGERS ( SCHED_NAME VARCHAR(120) NOT NULL, TRIGGER_NAME VARCHAR(190) NOT NULL, TRIGGER_GROUP VARCHAR(190) NOT NULL, JOB_NAME VARCHAR(190) NOT NULL, JOB_GROUP VARCHAR(190) NOT NULL, DESCRIPTION VARCHAR(250) NULL, NEXT_FIRE_TIME BIGINT(13) NULL, PREV_FIRE_TIME BIGINT(13) NULL, PRIORITY INTEGER NULL, TRIGGER_STATE VARCHAR(16) NOT NULL, TRIGGER_TYPE VARCHAR(8) NOT NULL, START_TIME BIGINT(13) NOT NULL, END_TIME BIGINT(13) NULL, CALENDAR_NAME VARCHAR(190) NULL, MISFIRE_INSTR SMALLINT(2) NULL, JOB_DATA BLOB NULL, PRIMARY KEY (SCHED_NAME,TRIGGER_NAME,TRIGGER_GROUP), FOREIGN KEY (SCHED_NAME,JOB_NAME,JOB_GROUP) REFERENCES QRTZ_JOB_DETAILS(SCHED_NAME,JOB_NAME,JOB_GROUP)) ENGINE=InnoDB; CREATE TABLE QRTZ_SIMPLE_TRIGGERS ( SCHED_NAME VARCHAR(120) NOT NULL, TRIGGER_NAME VARCHAR(190) NOT NULL, TRIGGER_GROUP VARCHAR(190) NOT NULL, REPEAT_COUNT BIGINT(7) NOT NULL, REPEAT_INTERVAL BIGINT(12) NOT NULL, TIMES_TRIGGERED BIGINT(10) NOT NULL, PRIMARY KEY (SCHED_NAME,TRIGGER_NAME,TRIGGER_GROUP), FOREIGN KEY (SCHED_NAME,TRIGGER_NAME,TRIGGER_GROUP) REFERENCES QRTZ_TRIGGERS(SCHED_NAME,TRIGGER_NAME,TRIGGER_GROUP)) ENGINE=InnoDB; CREATE TABLE QRTZ_CRON_TRIGGERS ( SCHED_NAME VARCHAR(120) NOT NULL, TRIGGER_NAME VARCHAR(190) NOT NULL, TRIGGER_GROUP VARCHAR(190) NOT NULL, CRON_EXPRESSION VARCHAR(120) NOT NULL, TIME_ZONE_ID VARCHAR(80), PRIMARY KEY (SCHED_NAME,TRIGGER_NAME,TRIGGER_GROUP), FOREIGN KEY (SCHED_NAME,TRIGGER_NAME,TRIGGER_GROUP) REFERENCES QRTZ_TRIGGERS(SCHED_NAME,TRIGGER_NAME,TRIGGER_GROUP)) ENGINE=InnoDB; CREATE TABLE QRTZ_SIMPROP_TRIGGERS ( SCHED_NAME VARCHAR(120) NOT NULL, TRIGGER_NAME VARCHAR(190) NOT NULL, TRIGGER_GROUP VARCHAR(190) NOT NULL, STR_PROP_1 VARCHAR(512) NULL, STR_PROP_2 VARCHAR(512) NULL, STR_PROP_3 VARCHAR(512) NULL, INT_PROP_1 INT NULL, INT_PROP_2 INT NULL, LONG_PROP_1 BIGINT NULL, LONG_PROP_2 BIGINT NULL, DEC_PROP_1 NUMERIC(13,4) NULL, DEC_PROP_2 NUMERIC(13,4) NULL, BOOL_PROP_1 VARCHAR(1) NULL, BOOL_PROP_2 VARCHAR(1) NULL, PRIMARY KEY (SCHED_NAME,TRIGGER_NAME,TRIGGER_GROUP), FOREIGN KEY (SCHED_NAME,TRIGGER_NAME,TRIGGER_GROUP) REFERENCES QRTZ_TRIGGERS(SCHED_NAME,TRIGGER_NAME,TRIGGER_GROUP)) ENGINE=InnoDB; CREATE TABLE QRTZ_BLOB_TRIGGERS ( SCHED_NAME VARCHAR(120) NOT NULL, TRIGGER_NAME VARCHAR(190) NOT NULL, TRIGGER_GROUP VARCHAR(190) NOT NULL, BLOB_DATA BLOB NULL, PRIMARY KEY (SCHED_NAME,TRIGGER_NAME,TRIGGER_GROUP), INDEX (SCHED_NAME,TRIGGER_NAME, TRIGGER_GROUP), FOREIGN KEY (SCHED_NAME,TRIGGER_NAME,TRIGGER_GROUP) REFERENCES QRTZ_TRIGGERS(SCHED_NAME,TRIGGER_NAME,TRIGGER_GROUP)) ENGINE=InnoDB; CREATE TABLE QRTZ_CALENDARS ( SCHED_NAME VARCHAR(120) NOT NULL, CALENDAR_NAME VARCHAR(190) NOT NULL, CALENDAR BLOB NOT NULL, PRIMARY KEY (SCHED_NAME,CALENDAR_NAME)) ENGINE=InnoDB; CREATE TABLE QRTZ_PAUSED_TRIGGER_GRPS ( SCHED_NAME VARCHAR(120) NOT NULL, TRIGGER_GROUP VARCHAR(190) NOT NULL, PRIMARY KEY (SCHED_NAME,TRIGGER_GROUP)) ENGINE=InnoDB; CREATE TABLE QRTZ_FIRED_TRIGGERS ( SCHED_NAME VARCHAR(120) NOT NULL, ENTRY_ID VARCHAR(95) NOT NULL, TRIGGER_NAME VARCHAR(190) NOT NULL, TRIGGER_GROUP VARCHAR(190) NOT NULL, INSTANCE_NAME VARCHAR(190) NOT NULL, FIRED_TIME BIGINT(13) NOT NULL, SCHED_TIME BIGINT(13) NOT NULL, PRIORITY INTEGER NOT NULL, STATE VARCHAR(16) NOT NULL, JOB_NAME VARCHAR(190) NULL, JOB_GROUP VARCHAR(190) NULL, IS_NONCONCURRENT VARCHAR(1) NULL, REQUESTS_RECOVERY VARCHAR(1) NULL, PRIMARY KEY (SCHED_NAME,ENTRY_ID)) ENGINE=InnoDB; CREATE TABLE QRTZ_SCHEDULER_STATE ( SCHED_NAME VARCHAR(120) NOT NULL, INSTANCE_NAME VARCHAR(190) NOT NULL, LAST_CHECKIN_TIME BIGINT(13) NOT NULL, CHECKIN_INTERVAL BIGINT(13) NOT NULL, PRIMARY KEY (SCHED_NAME,INSTANCE_NAME)) ENGINE=InnoDB; CREATE TABLE QRTZ_LOCKS ( SCHED_NAME VARCHAR(120) NOT NULL, LOCK_NAME VARCHAR(40) NOT NULL, PRIMARY KEY (SCHED_NAME,LOCK_NAME)) ENGINE=InnoDB; CREATE INDEX IDX_QRTZ_J_REQ_RECOVERY ON QRTZ_JOB_DETAILS(SCHED_NAME,REQUESTS_RECOVERY); CREATE INDEX IDX_QRTZ_J_GRP ON QRTZ_JOB_DETAILS(SCHED_NAME,JOB_GROUP); CREATE INDEX IDX_QRTZ_T_J ON QRTZ_TRIGGERS(SCHED_NAME,JOB_NAME,JOB_GROUP); CREATE INDEX IDX_QRTZ_T_JG ON QRTZ_TRIGGERS(SCHED_NAME,JOB_GROUP); CREATE INDEX IDX_QRTZ_T_C ON QRTZ_TRIGGERS(SCHED_NAME,CALENDAR_NAME); CREATE INDEX IDX_QRTZ_T_G ON QRTZ_TRIGGERS(SCHED_NAME,TRIGGER_GROUP); CREATE INDEX IDX_QRTZ_T_STATE ON QRTZ_TRIGGERS(SCHED_NAME,TRIGGER_STATE); CREATE INDEX IDX_QRTZ_T_N_STATE ON QRTZ_TRIGGERS(SCHED_NAME,TRIGGER_NAME,TRIGGER_GROUP,TRIGGER_STATE); CREATE INDEX IDX_QRTZ_T_N_G_STATE ON QRTZ_TRIGGERS(SCHED_NAME,TRIGGER_GROUP,TRIGGER_STATE); CREATE INDEX IDX_QRTZ_T_NEXT_FIRE_TIME ON QRTZ_TRIGGERS(SCHED_NAME,NEXT_FIRE_TIME); CREATE INDEX IDX_QRTZ_T_NFT_ST ON QRTZ_TRIGGERS(SCHED_NAME,TRIGGER_STATE,NEXT_FIRE_TIME); CREATE INDEX IDX_QRTZ_T_NFT_MISFIRE ON QRTZ_TRIGGERS(SCHED_NAME,MISFIRE_INSTR,NEXT_FIRE_TIME); CREATE INDEX IDX_QRTZ_T_NFT_ST_MISFIRE ON QRTZ_TRIGGERS(SCHED_NAME,MISFIRE_INSTR,NEXT_FIRE_TIME,TRIGGER_STATE); CREATE INDEX IDX_QRTZ_T_NFT_ST_MISFIRE_GRP ON QRTZ_TRIGGERS(SCHED_NAME,MISFIRE_INSTR,NEXT_FIRE_TIME,TRIGGER_GROUP,TRIGGER_STATE); CREATE INDEX IDX_QRTZ_FT_TRIG_INST_NAME ON QRTZ_FIRED_TRIGGERS(SCHED_NAME,INSTANCE_NAME); CREATE INDEX IDX_QRTZ_FT_INST_JOB_REQ_RCVRY ON QRTZ_FIRED_TRIGGERS(SCHED_NAME,INSTANCE_NAME,REQUESTS_RECOVERY); CREATE INDEX IDX_QRTZ_FT_J_G ON QRTZ_FIRED_TRIGGERS(SCHED_NAME,JOB_NAME,JOB_GROUP); CREATE INDEX IDX_QRTZ_FT_JG ON QRTZ_FIRED_TRIGGERS(SCHED_NAME,JOB_GROUP); CREATE INDEX IDX_QRTZ_FT_T_G ON QRTZ_FIRED_TRIGGERS(SCHED_NAME,TRIGGER_NAME,TRIGGER_GROUP); CREATE INDEX IDX_QRTZ_FT_TG ON QRTZ_FIRED_TRIGGERS(SCHED_NAME,TRIGGER_GROUP); commit; -
以下是代码列表清单
实现逻辑：
1> ScheduleJobInit 初始化时加载所有AbstractJob 的子类
2> 获取子类的 beanName ，及ScheduleTag注解注解的方法名称methodName,及注解的cron()方法
3> 根据beanName+methodName 生产jobId
4> 如果外部有 beanName+methodName+cron 配置取外部配置,没值取注解的cron()值
5> 具体需要定时执行的类和方法 需满足 在spring容器里、实现AbstractJob 接口、方法上加上ScheduleTag(cron="") 注解信息即可
-
ScheduleJobEntity job信息类
@Data public class ScheduleJobEntity { public static final String JOB_PARAM_KEY = "JOB_PARAM_KEY"; private String jobId; private String beanName; private String methodName; private String params; private String cronExpression; public String getJobId() { return StrUtil.join("_", beanName, methodName); } } -
ScheduleUtils schedule工具类
/** * 定时任务工具类 */ public class ScheduleUtils { private final static String JOB_NAME = "job_"; private final static String JOB_PARAMS_NAME = "params"; private final static String JOB_CRON_NAME = "cron"; /** * 获取触发器key */ public static TriggerKey getTriggerKey(String jobId) { return TriggerKey.triggerKey(JOB_NAME + jobId); } /** * 获取jobKey */ public static JobKey getJobKey(String jobId) { return JobKey.jobKey(JOB_NAME + jobId); } public static void initJob(Scheduler scheduler, Class jobClass, Map<String, String> properties) { ScheduleJobEntity scheduleJob = new ScheduleJobEntity(); scheduleJob.setBeanName(StringUtils.uncapitalize(jobClass.getSimpleName())); Method[] methods = MethodUtils.getMethodsWithAnnotation(jobClass, ScheduleTag.class); for (Method method : methods) { ScheduleTag scheduleTag = method.getAnnotation(ScheduleTag.class); String params = ""; String cron = ""; if (scheduleTag != null) { params = scheduleTag.params(); cron = scheduleTag.corn(); } if (CollectionUtil.isNotEmpty(properties)) { String paramsDynamic = properties.get(joinKey(scheduleJob.getBeanName(), method.getName(), JOB_PARAMS_NAME)); String cronDynamic = properties.get(joinKey(scheduleJob.getBeanName(), method.getName(), JOB_CRON_NAME)); if (StringUtils.isNotEmpty(paramsDynamic)) { params = paramsDynamic; } if (StringUtils.isNotEmpty(cronDynamic)) { cron = cronDynamic; } } scheduleJob.setMethodName(method.getName()); scheduleJob.setParams(params); scheduleJob.setCronExpression(cron); CronTrigger cronTrigger = ScheduleUtils.getCronTrigger(scheduler, scheduleJob.getJobId()); if (cronTrigger == null) { createScheduleJob(scheduler, scheduleJob); } else { updateScheduleJob(scheduler, scheduleJob); } } } private static String joinKey(String beanName, String methodName, String key) { return StrUtil.join("_", beanName, methodName, key); } /** * 获取表达式触发器 */ public static CronTrigger getCronTrigger(Scheduler scheduler, String jobId) { try { return (CronTrigger) scheduler.getTrigger(getTriggerKey(jobId)); } catch (SchedulerException e) { throw new QeteshException(e, "获取定时任务CronTrigger出现异常"); } } /** * 创建定时任务 */ public static void createScheduleJob(Scheduler scheduler, ScheduleJobEntity scheduleJob) { try { // 构建job信息 JobDetail jobDetail = JobBuilder.newJob(ScheduleJob.class).withIdentity(getJobKey(scheduleJob.getJobId())).build(); // 表达式调度构建器 CronScheduleBuilder scheduleBuilder = CronScheduleBuilder.cronSchedule(scheduleJob.getCronExpression()) .withMisfireHandlingInstructionDoNothing(); // 按新的cronExpression表达式构建一个新的trigger CronTrigger trigger = TriggerBuilder.newTrigger().withIdentity(getTriggerKey(scheduleJob.getJobId())).withSchedule(scheduleBuilder).build(); // 放入参数，运行时的方法可以获取 jobDetail.getJobDataMap().put(ScheduleJobEntity.JOB_PARAM_KEY, JSONUtil.toJsonStr(scheduleJob)); scheduler.scheduleJob(jobDetail, trigger); } catch (SchedulerException e) { throw new QeteshException(e, "创建定时任务失败"); } } /** * 更新定时任务 */ public static void updateScheduleJob(Scheduler scheduler, ScheduleJobEntity scheduleJob) { try { TriggerKey triggerKey = getTriggerKey(scheduleJob.getJobId()); // 表达式调度构建器 CronScheduleBuilder scheduleBuilder = CronScheduleBuilder.cronSchedule(scheduleJob.getCronExpression()) .withMisfireHandlingInstructionDoNothing(); CronTrigger trigger = getCronTrigger(scheduler, scheduleJob.getJobId()); // 按新的cronExpression表达式重新构建trigger trigger = trigger.getTriggerBuilder().withIdentity(triggerKey).withSchedule(scheduleBuilder).build(); // 参数 trigger.getJobDataMap().put(ScheduleJobEntity.JOB_PARAM_KEY, JSONUtil.toJsonStr(scheduleJob)); scheduler.rescheduleJob(triggerKey, trigger); } catch (SchedulerException e) { throw new QeteshException(e, "更新定时任务失败"); } } } -
ScheduleRunnable 具体任务执行反射线程类
/** * 执行定时任务 */ public class ScheduleRunnable implements Runnable { private final Object target; private final Method method; private final String params; public ScheduleRunnable(String beanName, String methodName, String params) throws NoSuchMethodException, SecurityException { this.target = SpringContextHolder.getBean(beanName); this.params = params; if (StringUtils.isNotBlank(params)) { this.method = target.getClass().getDeclaredMethod(methodName, String.class); } else { this.method = target.getClass().getDeclaredMethod(methodName); } } @Override public void run() { try { ReflectionUtils.makeAccessible(method); if (StringUtils.isNotBlank(params)) { method.invoke(target, params); } else { method.invoke(target); } } catch (Exception e) { throw new QeteshException(e, "执行定时任务失败"); } } } -
ScheduleJob job类
/** * 定时任务 */ @Slf4j public class ScheduleJob extends QuartzJobBean { private static final ExecutorService service = Executors.newSingleThreadExecutor(); @Override protected void executeInternal(JobExecutionContext context) throws JobExecutionException { String jsonJob = context.getMergedJobDataMap().getString(ScheduleJobEntity.JOB_PARAM_KEY); ScheduleJobEntity scheduleJob = JSONUtil.toBean(jsonJob, ScheduleJobEntity.class); // 任务开始时间 long startTime = System.currentTimeMillis(); try { // 执行任务 log.info("任务准备执行，任务ID：" + scheduleJob.getJobId()); ScheduleRunnable task = new ScheduleRunnable(scheduleJob.getBeanName(), scheduleJob.getMethodName(), scheduleJob.getParams()); Future<?> future = service.submit(task); future.get(); // 任务执行总时长 long times = System.currentTimeMillis() - startTime; log.info("任务执行完毕，任务ID：" + scheduleJob.getJobId() + " 总共耗时：" + times + "毫秒"); } catch (Exception e) { log.error("任务执行失败，任务ID：" + scheduleJob.getJobId(), e); // 任务执行总时长 long times = System.currentTimeMillis() - startTime; log.info("任务执行完毕，任务ID：" + scheduleJob.getJobId() + " 总共耗时：" + times + "毫秒"); } } } -
ScheduleTag 注解类
@Retention(RetentionPolicy.RUNTIME) @Target({ElementType.METHOD, ElementType.TYPE}) @Documented @Inherited public @interface ScheduleTag { String corn() default ""; String params() default ""; } -
AbstractJob job接口类,如果需要定时执行的方法需要继承该接口
public interface AbstractJob { } -
ScheduleJobInit 初始化器
@Component @RequiredArgsConstructor(onConstructor = @__(@Autowired)) public class ScheduleJobInit implements ApplicationContextAware, InitializingBean { private final Scheduler scheduler; private final JobConfig jobConfig; private ApplicationContext applicationContext; @Override public void setApplicationContext(ApplicationContext applicationContext) throws BeansException { this.applicationContext = applicationContext; } @Override public void afterPropertiesSet() throws Exception { Collection<AbstractJob> jobList = applicationContext.getBeansOfType(AbstractJob.class).values(); if (CollectionUtil.isNotEmpty(jobList)) { jobList.forEach(job -> ScheduleUtils.initJob(scheduler, job.getClass(), jobConfig.getProperties())); } } } -
SimpleJob 简单示例job ,必须在spring容器内
@Slf4j @Component @RequiredArgsConstructor(onConstructor = @__(@Autowired)) public class SimpleJob implements AbstractJob { // 每5秒执行一次 @ScheduleTag(corn = "*/5 * * * * ?") public void test() { log.info("test start..."); } }
这里如果需要在配置文件里配置cron，需要增加如下配置
properties aicc.job.properties.simpleJob_test_cron=0 0/1 * * * ?
规则如下：
aicc.job.properties.
beanName(类名首字母小写).methodName(方法名).cron
