Loki 部署
二、loki 部署
下载地址：
https://github.com/grafana/loki/releases/download/v2.4.1/loki-linux-amd64.zip
$ curl -O -L "https://github.com/grafana/loki/releases/download/v2.7.2/loki-linux-amd64.zip"
# extract the binary
$ unzip "loki-linux-amd64.zip"
# make sure it is executable
$ chmod a+x "loki-linux-amd64"
1、解压，下载官方配置模板并修改
源码安装loki:
mkdir /data/loki
cd /data/loki
unzip loki-linux-amd64
wget https://raw.githubusercontent.com/grafana/loki/master/cmd/loki/loki-local-config.yaml
mkdir -p /home/loki/index
mkdir -p /home/loki/chunks
chmod -R 777 /home/loki/index
chmod -R 777 /home/loki/chunks
vi /data/loki/loki-config.yaml
-----------------------------------------
auth_enabled: false
server:
http_listen_port: 3100
grpc_listen_port: 9095
grpc_server_max_recv_msg_size: 1572864000
grpc_server_max_send_msg_size: 1572864000
graceful_shutdown_timeout: 60s
http_server_read_timeout: 60s
http_server_write_timeout: 60s
http_server_idle_timeout: 120s
#grpc_server_max_recv_msg_size: 209715200
#grpc_server_max_send_msg_size: 209715200
grpc_server_max_concurrent_streams: 1024
log_level: info
health_check_target: true
ingester:
lifecycler:
address: 127.0.0.1
ring:
kvstore:
store: inmemory
replication_factor: 1
final_sleep: 0s
chunk_idle_period: 5m
chunk_retain_period: 30s
max_transfer_retries: 0
max_chunk_age: 20m
schema_config:
configs:
- from: 2018-04-15
store: boltdb
object_store: filesystem
schema: v9
index:
prefix: index_
period: 168h
storage_config:
boltdb:
directory: /home/loki/index
filesystem:
directory: /home/loki/chunks
limits_config:
enforce_metric_name: false
reject_old_samples: true
reject_old_samples_max_age: 720h
ingestion_rate_mb: 30
ingestion_burst_size_mb: 15
chunk_store_config:
max_look_back_period: 168h
table_manager:
chunk_tables_provisioning:
inactive_read_throughput: 0
inactive_write_throughput: 0
provisioned_read_throughput: 0
provisioned_write_throughput: 0
index_tables_provisioning:
inactive_read_throughput: 0
inactive_write_throughput: 0
provisioned_read_throughput: 0
provisioned_write_throughput: 0
retention_deletes_enabled: true
retention_period: 168h
-------------------------------------------------------
nohup ./loki-linux-amd64 --config.file=loki.yaml &
-------------------------------------------------------
docker版本loki安装：
docker search loki
docker pull grafana/loki
docker run -d \
--restart always \
--name loki \
--privileged=true \
-v /home/loki:/mnt/config \ #挂载配置文件目录
-v /home/loki/index:/opt/loki/index \ #挂载宿主机索引目录
-v /home/loki/chunks:/opt/loki/chunks \
-p 3100:3100 \
grafana/loki:2.8.0 \
-config.file=/mnt/config/loki-config.yaml # 使用挂载的配置文件
loki docker版本的loki 需要安装日志插件
Loki支持Docker插件，该插件将从Docker容器读取日志并将其发送到Loki。可以将插件配置为将日志发送到私有Loki实例或Grafana Cloud。
参考：https://github.com/grafana/loki/blob/master/docs/sources/clients/docker-driver/_index.md
主机安装docker插件
docker plugin install grafana/loki-docker-driver:latest --alias loki --grant-all-permissions
docker plugin ls
docker配置使用插件，修改/etc/docker/daemon.json并重启docker
{
"debug" : true,
"log-driver": "loki",
"log-opts": {
"loki-url": "http://8.8.8.8/loki/api/v1/push",
"loki-batch-size": "400"
}
}
-----------------------------------------------------------------------
Grafana 安装 docker版本：
docker search grafana
docker pull grafana/grafana
docker run -d --name=grafana \
--restart always \
-p 3000:3000 \
-v grafana-storage:/var/lib/grafana \
grafana/grafana
-------------------------------------------------------------------------
URL: IP:3000 admin/admin
--------------------------------------------------------------------------
在任意客户机节点安装 日志收集器：promtail
mkdir /home/promtail
vi /home/promtail/promtail-config.yaml
--------------------------------------------------
server:
http_listen_port: 9080
grpc_listen_port: 0
positions:
filename: /tmp/positions.yaml
clients:
- url: http://8.8.8.8:3100/loki/api/v1/push
scrape_configs:
- job_name: uat-pay-ddd
pipeline_stages:
static_configs:
- targets:
- 6.6.9.9
labels:
job: uat-pay-ddd
__path__: /var/log/**/*.log
- job_name: uat-factory-nginx
pipeline_stages:
static_configs:
- targets:
- 77.77.77.77
labels:
job: uat-factory-nginx
__path__: /var/log/nginx/**/*.log
--------------------------------------------
以微服务的包名为服务名称，还可以加很多个服务
-job_name: xxx
。。。。。。。
-job_name: kkk
。。。。。。。
docker run -d \
--name promtail \
--restart always \
--privileged=true \
-v /home/promtail:/mnt/config \
-v /opt/topdon/logs/product/app-produce-prod/:/var/log/app-produce-prod \
-v /opt/topdon/logs/product/app-user-prod/:/var/log/app-user-prod \
-v /opt/topdon/logs/product/app-vci-prod/:/var/log/app-vci-prod \
-v /opt/topdon/logs/product/topdon-auth-prod/:/var/log/topdon-auth-prod \
-v /opt/topdon/logs/product/topdon-user/:/var/log/topdon-user \
-v /opt/topdon/logs/product/topdon-dic-prod/:/var/log/topdon-dic-prod \
-v /opt/topdon/logs/product/topdon-gateway-prod/:/var/log/topdon-gateway-prod \
-v /opt/topdon/logs/product/topdon-plat-data-interaction-prod/:/var/log/topdon-plat-data-interaction-prod \
-v /opt/topdon/logs/product/topdon-plat-innerapi-prod/:/var/log/topdon-plat-innerapi-prod \
-v /opt/topdon/logs/product/topdon-plat-out-user-provide-prod/:/var/log/topdon-plat-out-user-provide-prod \
-v /opt/topdon/logs/product/topdon-plat-user-listener-prod/:/var/log/topdon-plat-user-listener-prod \
-v /opt/topdon/logs/product/topdon-plat-user-provide-prod/:/var/log/topdon-plat-user-provide-prod \
grafana/promtail:latest \
-config.file=/mnt/config/promtail-config.yaml
这样的好处是，grafana页面的loki会以每job每xxxx服务 的形式展示日志
如果想要收集指定目录及所有子目录下以.log结尾的日志文件，修改promtail-config.yaml 配置文件：
# cat promtail-config.yaml
......
__path__: /var/log/**/*.log
http://8.8.8.8:3100/metrics
http://8.8.8.8:3100/ready
配置grafana数据源
访问grafana，默认用户密码为admin/admin。
选择左侧设置—>Data Sources—>Add data source，搜索Loki配置HTTP URL为
完成后选择左侧设置—>Preferences，修改底部默认时区为Asia/Shanghai。
选择左侧Explore查看日志，可以基于文件名或标签查看：
========================================================================
