browserless webhooks 简单说明

以前说明browserless 的hooks，实际上browserless还支持webhooks 也是一个很不错的功能，可以进行统计分析
而且webhooks 的地址都是可以通过环境变量配置的

参考使用

  • docker 启动
docker run -d \
  --restart always \
  --name browserless \
  -e "QUEUE_ALERT_URL=https://my.website.com/queued" \
  -p 3000:3000
  ghcr.io/browserless/chromium

内部处理

  • webhooks 的定义

    可以看到提供了不少，而且都是可配置的

import { Config, fetchTimeout, noop } from '@browserless.io/browserless';
import { EventEmitter } from 'events';
 
export class WebHooks extends EventEmitter {
  constructor(protected config: Config) {
    super();
  }
 
  protected callURL(url: string | null) {
    if (url) {
      return fetchTimeout(url, {
        method: 'GET',
        timeout: 10000,
      }).catch(noop);
    }
 
    return;
  }
 
  public callFailedHealthURL() {
    const url = this.config.getFailedHealthURL();
    return this.callURL(url);
  }
 
  public callQueueAlertURL() {
    const url = this.config.getQueueAlertURL();
    return this.callURL(url);
  }
 
  public callRejectAlertURL() {
    const url = this.config.getRejectAlertURL();
    return this.callURL(url);
  }
 
  public callTimeoutAlertURL() {
    const url = this.config.getTimeoutAlertURL();
    return this.callURL(url);
  }
 
  public callErrorAlertURL(message: string) {
    const url = this.config.getErrorAlertURL();
 
    try {
      const fullURL = new URL(url ?? '');
      fullURL?.searchParams.set('error', message);
      return this.callURL(fullURL.href);
    } catch (err) {
      return console.error(
        `Issue calling error hook: "${err}". Did you set a working ERROR_ALERT_URL env variable?`,
      );
    }
  }
 
  /**
   * Implement any browserless-core-specific shutdown logic here.
   * Calls the empty-SDK stop method for downstream implementations.
   */
  public shutdown = async () => {
    await this.stop();
  };
 
  /**
   * Left blank for downstream SDK modules to optionally implement.
   */
  public stop = () => {};
}
  • 内部使用

包含了browserless 以及limiter 部分，以下说明limiter 的处理, 包含了请求队列的参数

public limit = <TArgs extends unknown[], TResult>(
  limitFn: LimitFn<TArgs, TResult>,
  overCapacityFn: ErrorFn<TArgs>,
  onTimeoutFn: ErrorFn<TArgs>,
  timeoutOverrideFn: (...args: TArgs) => number | undefined,
): LimitFn<TArgs, unknown> => {
  return (...args: TArgs) =>
    new Promise(async (res, rej) => {
      const timeout = timeoutOverrideFn(...args) ?? this.timeout;
      this.logQueue(
        `Adding to queue, max time allowed is ${timeout.toLocaleString()}ms`,
      );
 
      if (this.config.getHealthChecksEnabled()) {
        const { cpuOverloaded, memoryOverloaded } =
          await this.monitor.overloaded();
 
        if (cpuOverloaded || memoryOverloaded) {
          this.logQueue(`Health checks have failed, rejecting`);
          this.webhooks.callFailedHealthURL();
          this.metrics.addRejected();
          overCapacityFn(...args);
          return rej(new Error(`Health checks have failed, rejecting`));
        }
      }
 
      if (!this.hasCapacity) {
        this.logQueue(`Concurrency and queue is at capacity`);
        this.webhooks.callRejectAlertURL();
        this.metrics.addRejected();
        overCapacityFn(...args);
        return rej(
          new TooManyRequests(`Concurrency and queue is at capacity`),
        );
      }
 
      if (this.willQueue) {
        this.logQueue(`Concurrency is at capacity, queueing`);
        this.webhooks.callQueueAlertURL();
        this.metrics.addQueued();
      }
 
      const bound: () => Promise<TResult | unknown> = async () => {
        this.logQueue(`Starting new job`);
        this.metrics.addRunning();
 
        try {
          const result = await limitFn(...args);
          res(result);
          return;
        } catch (err) {
          rej(err);
          throw err;
        }
      };
 
      const job: Job = Object.assign(bound, {
        args,
        onTimeoutFn: () => onTimeoutFn(...args),
        start: Date.now(),
        timeout,
      });
 
      this.push(job);
 
      return bound;
    });
};

说明

hooks+ webhooks 都是很不错的browserless 统计分析或者扩展地方，值得试用下

参考资料

src/webhooks.ts
src/browserless.ts
src/limiter.ts
https://docs.browserless.io/Docker/webhooks
https://github.com/browserless/browserless

posted on 2024-07-02 08:00  荣锋亮  阅读(15)  评论(0)  编辑  收藏  举报

导航