kubernets实战采坑1
1.NLog.config失效,日志ElasticSearch的Index不匹配
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 | <?xml version= "1.0" encoding= "utf-8" ?> <nlog xmlns= "http://www.nlog-project.org/schemas/NLog.xsd" xmlns:xsi= "http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation= "http://www.nlog-project.org/schemas/NLog.xsd NLog.xsd" autoReload= "true" > <extensions> <add assembly= "NLog.Targets.ElasticSearch" /> </extensions> <!-- optional, add some variabeles https: //github.com/nlog/NLog/wiki/Configuration-file#variables --> <variable name= "myvar" value= "myvalue" /> <targets async= "true" > <target xsi:type= "File" name= "File" fileName= "${environment:variable=NLOGDIR}/WebSite.log" archiveAboveSize= "20000000" archiveNumbering= "Rolling" encoding= "UTF-8" enableFileDelete= "true" maxArchiveFiles= "5" layout= "[${date:format=yyyy-MM-dd HH\:mm\:ss}][${level}] ${logger} ${message} ${exception:format=toString} ${aspnet-request:cookie=String:serverVariable=String:queryString=String:item=String:form=String:header=String}" /> <target name= "ElasticSearch" xsi:type= "ElasticSearch" ConnectionStringName= "ElasticSearchServerAddress" index= "platform-website-${date:format=yyyy.MM.dd}" documentType= "logevent" includeAllProperties= "true" layout= "[${date:format=yyyy-MM-dd HH\:mm\:ss}][${level}] ${logger} ${message} ${exception:format=toString}" > <field name= "MachineName" layout= "${machinename}" /> <!--<field name= "AllEventProperties" layout= "${all-event-properties:format=[key]=[value]:separator=;:includeCallerInformation=true}" />--> <field name= "Time" layout= "${longdate}" /> <field name= "level" layout= "${level:uppercase=true}" /> <field name= "logger" layout= " ${logger}" /> <field name= "message" layout= " ${message}" /> <field name= "exception" layout= " ${exception:format=toString}" /> <field name= "processid" layout= " ${processid}" /> <field name= "threadname" layout= " ${threadname}" /> <field name= "stacktrace" layout= " ${stacktrace}" /> <field name= "Properties" layout= "${machinename} ${longdate} ${level:uppercase=true} ${logger} ${message} ${exception}|${processid}|${stacktrace}|${threadname}" /> <field name= "WWNamespace" layout= "${environment:WWNamespace}" /> </target> <target name= "PerformanceLogES" xsi:type= "ElasticSearch" ConnectionStringName= "ElasticSearchServerAddress" index= "perflog-platform-website-${date:format=yyyy.MM.dd}" documentType= "logevent" includeAllProperties= "true" layout= "[${date:format=yyyy-MM-dd HH\:mm\:ss}][${level}] ${logger} ${message} ${exception:format=toString}" > <field name= "MachineName" layout= "${machinename}" /> <!--<field name= "AllEventProperties" layout= "${all-event-properties:format=[key]=[value]:separator=;:includeCallerInformation=true}" />--> <field name= "Time" layout= "${longdate}" /> <field name= "level" layout= "${level:uppercase=true}" /> <field name= "logger" layout= " ${logger}" /> <field name= "message" layout= " ${message}" /> <field name= "exception" layout= " ${exception:format=toString}" /> <field name= "processid" layout= " ${processid}" /> <field name= "threadname" layout= " ${threadname}" /> <field name= "stacktrace" layout= " ${stacktrace}" /> <field name= "Properties" layout= "${machinename} ${longdate} ${level:uppercase=true} ${logger} ${message} ${exception}|${processid}|${stacktrace}|${threadname}" /> <field name= "WWNamespace" layout= "${environment:WWNamespace}" /> </target> </targets> <rules> <!-- add your logging rules here --> <!--<logger name= "*" minlevel= "Trace" writeTo= "EventLog" />--> <logger name= "PerformanceLog" minlevel= "Trace" writeTo= "PerformanceLogES" final= "true" /> <logger name= "*" minlevel= "Trace" writeTo= "File" /> <logger name= "*" minlevel= "Trace" writeTo= "ElasticSearch" /> <!-- <logger name= "*" minlevel= "Trace" writeTo= "ColoredConsole" /> <logger name= "*" minlevel= "Trace" writeTo= "f" /> --> </rules> </nlog> |
- 查看项目NLog.config是否正确 =》正确
- Kubernets上,pods下NLog.config是否正确 =》错误
- 查看本地Container历史 =》是自己上传的image
- image制定的publish目录中NLog.config是否正确 =》错误
- 为什么项目中的NLog.config是正确的,但是发布的NLog.config是错误的呢?
- 找到错误的NLog.config哪儿来的 =》同解决方案中,其他某个project的NLog.config
- 本项目中是否添加了对应项目的依赖引用? =》是的,找到原因(引用了项目,对应项目的NLog.config会在publish时候加入,从而导致了替换了本项目的NLog.config)
- 还好那个项目的引用本项目不再使用,故删除那个项目的引用 ,重新打包到kubernetes =》 问题解决
2.kubernets的Ingress中配置的Host无法使用?
- 远程,根据host名远程访问服务 =》 失败
- 进入ingress有问题的服务的pods中,访问服务 =》成功,服务本身没有问题
- 进入其他服务的pods中,通过服务名来访问ingree有问题的服务 =》成功,服务间调用没有问题,应该是ingress配置出现了问题
- 仔细检查,是否Ingress的yaml文件有错误 =》 没有问题
- 修改Ingress中host的命名,删掉pods =》还是无法通过host访问服务
- 最终,找老大解决,给出原因:kubernets中有两个namespace,结果配置了同样的host名,导致kubernets无法解析应该向哪个服务发送请求,故无法访问。=》删除了另一个namespace中的ingress配置,则问题解决
- 疑问:之前已经修改过host名,为什么当时删掉pods,自动起新的pods,还是无法用新的host名来访问服务? =》 知道的大师,请指点下!感谢!
3.configmap修改了之后,无法生效?
- 重启pods,因为已有的pods中的环境变量不会改变;重启之后,环境变量才会用新配置的configmap
4.同一个namespace中服务之间相互引用,不需要配置外部的host,只需要根据服务名调用即可:http://[服务名]
- 获取所有服务:kubectl get service -n dev
5.服务出现问题
- 从kibana中查看日志=》没找到
- 进入pods,查看本地日志
- 如果是cronjob,则查看job历史,查找记录
- kubectl get cronjobs -n dev
- kubectl get jobs -n dev
- kubectl log job/*** -n dev
6.服务没有日志
- 进入服务对应的pods中,是否有NLog.config
- kubectl exec -ti **** /bin/bash -n dev
- 项目中是否有NLog.config
- NLog.config的属性:生成=》始终复制
- 查找项目Dockerfile中指定的发布地址,此目录下,是否有NLog.config
【推荐】编程新体验,更懂你的AI,立即体验豆包MarsCode编程助手
【推荐】凌霞软件回馈社区,博客园 & 1Panel & Halo 联合会员上线
【推荐】抖音旗下AI助手豆包,你的智能百科全书,全免费不限次数
【推荐】博客园社区专享云产品让利特惠,阿里云新客6.5折上折
【推荐】轻量又高性能的 SSH 工具 IShell:AI 加持,快人一步