.NET应用系统的国际化-基于Roslyn抽取词条、更新代码

上篇文章我们介绍了

VUE+.NET应用系统的国际化-多语言词条服务

系统国际化改造整体设计思路如下：

  1. 提供一个工具，识别前后端代码中的中文，形成多语言词条，按语言、界面、模块统一管理多有的多语言词条
  2. 提供一个翻译服务，批量翻译多语言词条
  3. 提供一个词条服务，支持后端代码在运行时根据用户登录的语言，动态获取对应的多语言文本
  4. 提供前端多语言JS生成服务，按界面动态生成对应的多语言JS文件，方便前端VUE文件使用。
  5. 提供代码替换工具，将VUE前端代码中的中文替换为$t("词条ID")，后端代码中的中文替换为TermService.Current.GetText("词条ID")

今天，我们在上篇文章的基础上，继续介绍基于Roslyn抽取词条、更新代码。

一、业务背景

先说一下业务背景，后端.NET代码中存在大量的中文提示和异常消息，甚至一些中文返回值文本。

这些中文文字都需要识别出来，抽取为多语言词条，同时将代码替换为调用多语言词条服务获取翻译后的文本。

例如：

private static void CheckMd5(string fileName, string md5Data)
{
      string md5Str = MD5Service.GetMD5(fileName);
      if (!string.Equals(md5Str, md5Data, StringComparison.OrdinalIgnoreCase))
      {
           throw new CustomException(PackageExceptionConst.FileMd5CheckFailed, "服务包文件MD5校验失败：" + fileName);
      }
}

代码中需要将“服务包文件MD5校验失败”这个文本做多语言改造。

这里通过调用多语言词条服务I18NTermService，根据线程上下文中设置的语言，获取对应的翻译文本。例如以下代码：

var text=T.Core.I18N.Service.TermService.Current.GetTextFormatted("词条ID"，"默认文本"); 

throw new CustomException(PackageExceptionConst.FileMd5CheckFailed, text + fileName);

以上背景下，我们准备使用Roslyn技术对代码进行中文扫描，对扫描出来的文本，做词条抽取、代码替换。

二、使用Roslyn技术对代码进行中文扫描

首先，我们先定义好代码中多语言词条的扫描结果类TermScanResult

 1  [Serializable]
 2     public class TermScanResult
 3     {
 4         public Guid Id { get; set; }
 5         public string OriginalText { get; set; }
 6 
 7         public string ChineseText { get; set; }
 8 
 9         public string SlnName { get; set; }
10 
11         public string ProjectName { get; set; }
12 
13         public string ClassFile { get; set; }
14 
15         public string MethodName { get; set; }
16 
17         public string Code { get; set; }
18 
19         public I18NTerm I18NTerm { get; set; }
20 
21         public string SlnPath { get; set; }
22 
23         public string ClassPath { get; set; }
24 28         public string SubSystemCode { get; set; }
29 
30         public override string ToString()
31         {
32             return Code;
33         }
34     }

上述代码中SubSystemCode是一个业务管理维度。大家忽略即可。

我们会以sln解决方案为单位，扫描代码中的中文文字。

以下是具体的实现代码

public async Task<List<TermScanResult>> CheckSln(string slnPath, System.ComponentModel.BackgroundWorker backgroundWorker, SubSystemFile subSystemFiles, string subSystem)
{
            var slnFile = new FileInfo(slnPath);
            var results = new List<TermScanResult>();

            MSBuildHelper.RegisterMSBuilder();
            var solution = await MSBuildWorkspace.Create().OpenSolutionAsync(slnPath);

            var subSystemInfo = subSystemFiles?.SubSystemSlnMappings.FirstOrDefault(w => w.SlnName.Select(s => s += ".sln").Contains(slnFile.Name.ToLower()));

            if (solution.Projects != null && solution.Projects.Count() > 0)
            {
                foreach (var project in solution.Projects.ToList())
                {
                    backgroundWorker.ReportProgress(10, $"扫描Project: {project.Name}");
                    var documents = project.Documents.Where(x => x.Name.Contains(".cs"));

                    if (project.Name.ToLower().Contains("test"))
                    {
                        continue;
                    }
                    var codeReplace = new CodeReplace();
                    foreach (var document in documents)
                    {
                        var tree = await document.GetSyntaxTreeAsync();
                        var root = tree.GetCompilationUnitRoot();
                        if (root.Members == null || root.Members.Count == 0) continue;
                        //member
                        var classDeclartions = root.DescendantNodes().Where(i => i is ClassDeclarationSyntax);

                        foreach (var classDeclare in classDeclartions)
                        {
                            var programDeclaration = classDeclare as ClassDeclarationSyntax;
                            if (programDeclaration == null) continue;

                            foreach (var memberDeclarationSyntax in programDeclaration.Members)
                            {
                                foreach (var item in GetLiteralStringExpression(memberDeclarationSyntax))
                                {
                                    var statementCode = item.Item1;
                                    foreach (var syntaxNode in item.Item3)
                                    {
                                        ExpressionSyntaxParser expressionSyntaxParser = new ExpressionSyntaxParser();
                                        var text = "";
                                        var expressionSyntax = expressionSyntaxParser
                                            .GetExpressionSyntaxVerifyRule(syntaxNode as ExpressionSyntax, statementCode);
                                        if (expressionSyntax != null)
                                        {
                                            // 排除
                                            if (expressionSyntaxParser.IsExcludeCaller(expressionSyntax, statementCode))
                                            {
                                                continue;
                                            }

                                            text = expressionSyntaxParser.GetExpressionSyntaxOriginalText(expressionSyntax, statementCode);
                                            if (expressionSyntax is Microsoft.CodeAnalysis.CSharp.Syntax.InterpolatedStringExpressionSyntax)
                                            {
                                                text = expressionSyntaxParser.GetExpressionSyntaxOriginalText(expressionSyntax, statementCode);

                                                if (expressionSyntax is Microsoft.CodeAnalysis.CSharp.Syntax.LiteralExpressionSyntax)
                                                {
                                                    if (!expressionSyntax.IsKind(SyntaxKind.StringLiteralExpression))
                                                    {
                                                        continue;
                                                    }
                                                    text = expressionSyntax.NormalizeWhitespace().ToString();
                                                }
                                            }
                                        }
                                        if (CheckChinese(text) == false) continue;
                                        if (string.IsNullOrWhiteSpace(text)) continue;
                                        if (string.IsNullOrWhiteSpace(text.Replace("\"", "").Trim())) continue;

                                        results.Add(new TermScanResult()
                                        {
                                            Id = Guid.NewGuid(),
                                            ClassPath = programDeclaration.SyntaxTree.FilePath,
                                            SlnPath = slnPath,
                                            OriginalText = text.Replace("\"", "").Trim(),
                                            ChineseText = text,
                                            SlnName = slnFile.Name,
                                            ProjectName = project.Name,
                                            ClassFile = programDeclaration.Identifier.Text,
                                            MethodName = item.Item2,
                                            Code = statementCode,
                                            SubSystemCode = subSystem
                                        });
                                    }
                                }
                            }
                        }
                    }
                }
            }

     return results;
}

上述代码中，我们先使用MSBuilder编译，构建 sln解决方案

MSBuildHelper.RegisterMSBuilder();
var solution = await MSBuildWorkspace.Create().OpenSolutionAsync(slnPath);

然后遍历solution下的各个Project中的class类

foreach (var project in solution.Projects.ToList())
var documents = project.Documents.Where(x => x.Name.Contains(".cs"));

然后遍历类中声明、成员、方法中的每行代码，通过正则表达式识别是否有中文字符

public static bool CheckChinese(string strZh)
{
            Regex re = new Regex(@"[\u4e00-\u9fa5]+");
            if (re.IsMatch(strZh))
            {
                return true;
            }
            return false;
}

如果存在中文字符，作为扫描后的结果，识别为多语言词条

results.Add(new TermScanResult()
{
        Id = Guid.NewGuid(),
        ClassPath = programDeclaration.SyntaxTree.FilePath,
        SlnPath = slnPath,
        OriginalText = text.Replace("\"", "").Trim(),
        ChineseText = text,
        SlnName = slnFile.Name,
        ProjectName = project.Name,
        ClassFile = programDeclaration.Identifier.Text,
        MethodName = item.Item2,
        Code = statementCode,        //管理维度                                  
        SubSystemCode = subSystem    //管理维度
});

TermScanResult中没有对词条属性赋值。

public I18NTerm I18NTerm { get; set; }

下一篇文章的代码中，我们会通过多语言翻译服务，将翻译后的文本放到I18NTerm 属性中，作为多语言词条。

三、代码替换

代码替换这块逻辑中，我们设计了一个类SourceWeaver，对上一步的代码扫描结果，进行代码替换

CodeScanReplace这个方法中完成了代码的二次扫描和替换
 /// <summary>
    /// 源代码替换服务
    /// </summary>
    public class SourceWeaver
    {
        List<CommonTermDto> commonTerms = new List<CommonTermDto>();
        List<CommonTermDto> commSubTerms = new List<CommonTermDto>();

        public SourceWeaver()
        {
            commonTerms = JsonConvert.DeserializeObject<List<CommonTermDto>>(File.ReadAllText("comm_data.json"));
            commSubTerms = JsonConvert.DeserializeObject<List<CommonTermDto>>(File.ReadAllText("comm_sub_data.json"));
        }
        public async Task CodeScanReplace(Tuple<List<I18NTerm>, List<TermScanResult>> result, System.ComponentModel.BackgroundWorker backgroundWorker)
        {
            try
            {
                backgroundWorker.ReportProgress(0, "正在对代码进行替换.");
                var termScanResultGroupBy = result.Item2.GroupBy(g => g.SlnName);
                foreach (var termScanResult in termScanResultGroupBy)
                {
                    var termScan = termScanResult.FirstOrDefault();
                    MSBuildHelper.RegisterMSBuilder();
                    var solution = await MSBuildWorkspace.Create().OpenSolutionAsync(termScan.SlnPath).ConfigureAwait(false);
                    if (solution.Projects.Any())
                    {
                        foreach (var project in solution.Projects.ToList())
                        {
                            if (project.Name.ToLower().Contains("test"))
                            {
                                continue;
                            }
                            var projectTermScanResults = result.Item2.Where(f => f.ProjectName == project.Name);

                            var documents = project.Documents.Where(x =>
                            {
                                return x.Name.Contains(".cs") && projectTermScanResults.Any(f => $"{f.ClassPath}" == x.FilePath);
                            });

                            foreach (var document in documents)
                            {
                                var tree = await document.GetSyntaxTreeAsync().ConfigureAwait(false);
                                var root = tree.GetCompilationUnitRoot();
                                if (root.Members.Count == 0) continue;

                                var classDeclartions = root.DescendantNodes()
                                    .Where(i => i is ClassDeclarationSyntax);
                                List<MemberDeclarationSyntax> syntaxNodes = new List<MemberDeclarationSyntax>();
                                foreach (var classDeclare in classDeclartions)
                                {
                                    if (!(classDeclare is ClassDeclarationSyntax programDeclaration)) continue;
                                    var className = programDeclaration.Identifier.Text;

                                    foreach (var method in programDeclaration.Members)
                                    {
                                        if (method is ConstructorDeclarationSyntax)
                                        {
                                            syntaxNodes.Add((ConstructorDeclarationSyntax)method);
                                        }
                                        else if (method is MethodDeclarationSyntax)
                                        {
                                            syntaxNodes.Add((MethodDeclarationSyntax)method);
                                        }
                                        else if (method is PropertyDeclarationSyntax)
                                        {
                                            syntaxNodes.Add(method);
                                        }
                                        else if (method is FieldDeclarationSyntax)
                                        {
                                            // 注：常量不支持
                                            syntaxNodes.Add(method);
                                        }
                                    }
                                }

                                var terms = termScanResult.Where(
                                    f => f.ProjectName == document.Project.Name && f.ClassPath == document.FilePath).ToList();
                                backgroundWorker.ReportProgress(10, $"正在检查{document.FilePath}文件.");
                                ReplaceNodesAndSave(root, syntaxNodes, terms, result, backgroundWorker, document.Name);
                            }
                        }
                    }
                }
            }
            catch (Exception ex)
            {
                LogUtils.LogError(string.Format("异常类型：{0}\r\n异常消息：{1}\r\n异常信息：{2}\r\n",
                    ex.GetType().Name, ex.Message, ex.StackTrace));
                backgroundWorker.ReportProgress(0, ex.Message);
            }
        }

        public async void ReplaceNodesAndSave(SyntaxNode classSyntaxNode, List<MemberDeclarationSyntax> syntaxNodes, IEnumerable<TermScanResult> terms, Tuple<List<I18NTerm>, List<TermScanResult>> result,
            System.ComponentModel.BackgroundWorker backgroundWorker, string className)
        {

            {//check pro是否存在词条
                if (AppConfig.Instance.IsCheckTermPro)
                {
                    backgroundWorker.ReportProgress(15, $"词条验证中.");
                    var termsCodes = terms.Select(f => f.I18NTerm.Code).ToList();
                    var size = 100;
                    var p = (result.Item2.Count() + size - 1) / size;

                    using DBHelper dBHelper = new DBHelper();
                    List<I18NTerm> items = new List<I18NTerm>();
                    for (int i = 0; i < p; i++)
                    {
                        var list = termsCodes
                            .Skip(i * size).Take(size);
                        Thread.Sleep(10);
                        var segmentItems = await dBHelper.GetTermsAsync(termsCodes).ConfigureAwait(false);
                        items.AddRange(segmentItems);
                    }

                    List<TermScanResult> termScans = new List<TermScanResult>();
                    foreach (var term in terms)
                    {
                        if (items.Any(f => f.Code == term.I18NTerm.Code))
                        {
                            termScans.Add(term);
                        }
                        else
                        {
                            backgroundWorker.ReportProgress(20, $"词条{term.OriginalText}未导入到词条库，该词条将忽略替换.");
                        }
                    }
                    terms = termScans;
                }
            }

            var newclassDeclare = classSyntaxNode;
            newclassDeclare = classSyntaxNode.ReplaceNodes(syntaxNodes,
                    (methodDeclaration, _) =>
                    {                     
                        MemberDeclarationSyntax newMemberDeclarationSyntax = methodDeclaration;
                        var className = ((ClassDeclarationSyntax)newMemberDeclarationSyntax.Parent).Identifier.Text;
                        List<StatementSyntax> statementSyntaxes = new List<StatementSyntax>();

                        switch (newMemberDeclarationSyntax)
                        {
                            case ConstructorDeclarationSyntax:
                                {
                                    var blockSyntax = (newMemberDeclarationSyntax as ConstructorDeclarationSyntax).NormalizeWhitespace().Body;
                                    if (blockSyntax == null)
                                    {
                                        break;
                                    }
                                    foreach (var statement in blockSyntax.Statements)
                                    {
                                        var nodeStatement = statement.DescendantNodes();

                                        statementSyntaxes.Add(new CodeReplace().ReplaceStatementNodes(statement,
                                            new ExpressionSyntaxParser().LiteralStringExpression(nodeStatement), terms, commonTerms, commSubTerms));
                                    }

                                    break;
                                }

                            case MethodDeclarationSyntax:
                                {
                                    var blockSyntax = (methodDeclaration as MethodDeclarationSyntax).NormalizeWhitespace().Body;
                                    if (blockSyntax == null)
                                    {
                                        break;
                                    }
                                    foreach (var statement in blockSyntax.Statements)
                                    {
                                        var nodeStatement = statement.DescendantNodes();
                                        statementSyntaxes.Add(new CodeReplace().ReplaceStatementNodes(statement,
                                               new ExpressionSyntaxParser().LiteralStringExpression(nodeStatement), terms, commonTerms, commSubTerms));
                                    }

                                    break;
                                }

                            case PropertyDeclarationSyntax:
                                {
                                    var propertyDeclarationSyntax = newMemberDeclarationSyntax as PropertyDeclarationSyntax;

                                    var nodeStatement = propertyDeclarationSyntax.DescendantNodes();

                                    return new CodeReplace().ReplacePropertyNodes(newMemberDeclarationSyntax as PropertyDeclarationSyntax,
                                        new ExpressionSyntaxParser().LiteralStringExpression(nodeStatement), terms, commonTerms, commSubTerms);
                                }

                            case FieldDeclarationSyntax:
                                {
                                    var fieldDeclarationSyntax = newMemberDeclarationSyntax as FieldDeclarationSyntax;
                                    var nodeStatement = fieldDeclarationSyntax.DescendantNodes();
                                    return new CodeReplace().ReplaceFiledNodes(fieldDeclarationSyntax,
                                           new ExpressionSyntaxParser().LiteralStringExpression(nodeStatement), terms, commonTerms, commSubTerms);
                                }
                        }
                        backgroundWorker.ReportProgress(50, $"解析并对类文件{className}中的方法做语句替换.");
                        // 替换方法内部
                        if (newMemberDeclarationSyntax is MethodDeclarationSyntax)
                        {
                            return new CodeReplace().ReplaceMethodDeclaration(newMemberDeclarationSyntax as MethodDeclarationSyntax, statementSyntaxes);
                        }
                        else if (newMemberDeclarationSyntax is ConstructorDeclarationSyntax)
                        {
                            return new CodeReplace().ReplaceConstructorDeclaration(newMemberDeclarationSyntax as ConstructorDeclarationSyntax, statementSyntaxes);
                        }
                        return newMemberDeclarationSyntax;
                    });

            var sourceStr = newclassDeclare.NormalizeWhitespace().GetText().ToString();
            File.WriteAllText(newclassDeclare.SyntaxTree.FilePath, sourceStr);
            backgroundWorker.ReportProgress(100, $"完成{className}的替换.");
        }
    }

关键的代码语义替换的实现代码：

 public StatementSyntax ReplaceStatementNodes(StatementSyntax statement, List<ExpressionSyntax> expressionSyntaxes, IEnumerable<TermScanResult> terms
            , List<CommonTermDto> commonTerms, List<CommonTermDto> commSubTerms)
        {
            var statementSyntax = statement.ReplaceNodes(expressionSyntaxes, (syntaxNode, _) =>
            {
                var statementStr = statement.NormalizeWhitespace().ToString();

                var argumentLists = statement.DescendantNodes().
                                               OfType<InvocationExpressionSyntax>();
                ExpressionSyntaxParser expressionSyntaxParser = new ExpressionSyntaxParser();
                return expressionSyntaxParser.ExpressionSyntaxTermReplace(syntaxNode, statementStr, terms, commonTerms, commSubTerms);

            });

            return statementSyntax;
        }

这里，我们抽象了一个ExpressionSyntaxParser 类，负责替换代码：

T.Core.I18N.Service.TermService.Current.GetTextFormatted
 public ExpressionSyntax ExpressionSyntaxTermReplace(ExpressionSyntax syntaxNode, string statementStr, IEnumerable<TermScanResult> terms
            , List<CommonTermDto> commonTerms, List<CommonTermDto> commSubTerms)
        {
            var expressionSyntax = GetExpressionSyntaxVerifyRule(syntaxNode, statementStr);
            var originalText = GetExpressionSyntaxOriginalText(expressionSyntax, statementStr);

            var I18Expr = "";
            var interpolationSyntaxes = syntaxNode.DescendantNodes().OfType<InterpolationSyntax>();         
            var term = terms.FirstOrDefault(i => i.ChineseText == originalText);

            if (term == null)
                return syntaxNode;
            string termcode = term.I18NTerm.Code;
if (syntaxNode is InterpolatedStringExpressionSyntax)
            {
                if (interpolationSyntaxes.Count() > 0)
                {
                    var parms = "";
                    foreach (var item in interpolationSyntaxes)
                    {
                        parms += $",{item.ToString().TrimStart('{').TrimEnd('}')}";
                    }
                    I18Expr = "$\"{T.Core.I18N.Service.TermService.Current.GetTextFormatted(\"" + termcode + "\", " + originalText + parms + ")}\"";
                    var token1 = SyntaxFactory.Token(default, SyntaxKind.StringLiteralToken, I18Expr, "", default);
                    return SyntaxFactory.LiteralExpression(SyntaxKind.StringLiteralExpression, token1);
                }
                else
                {

                    var startToken = SyntaxFactory.Token(SyntaxKind.InterpolatedStringStartToken);
                    if ((syntaxNode as InterpolatedStringExpressionSyntax).StringStartToken.Value == startToken.Value)
                    {
                        // 如果本身有"$"
                        I18Expr = "$\"{T.Core.I18N.Service.TermService.Current.GetText(\"" + termcode + "\"," + originalText + ")}";
                    }
                    else
                    {
                        // 如果没有"$"
                        I18Expr = "$\"{T.Core.I18N.Service.TermService.Current.GetText(\"" + termcode + "\",\\teld\"" + originalText + "\")}";
                        I18Expr = I18Expr.Replace("\\teld", "$");
                    }
                }
            }
            else
            {
                I18Expr = "$\"{T.Core.I18N.Service.TermService.Current.GetText(\"" + termcode + "\"," + originalText + ")}";
            }

            var token = SyntaxFactory.Token(default(SyntaxTriviaList), SyntaxKind.InterpolatedVerbatimStringStartToken, I18Expr, "$\"", default(SyntaxTriviaList));
            var literalExpressionSyntax = SyntaxFactory.InterpolatedStringExpression(token);
            return literalExpressionSyntax;
        }
T.Core.I18N.Service.TermService这个就是多语言词条服务类，这个类中提供了一个GetText的方法，通过词条编号，获取多语言文本。

代码完成替换后，打开VS，对工程引用多语言词条服务的Nuget包/dll，重新编译代码，手工校对替换后的代码即可。
以上是.NET应用系统的国际化-基于Roslyn抽取词条、更新代码的分享。


周国庆
2023/3/19







posted @ 2023-03-19 10:04  Eric zhou  阅读(663)  评论(0)  编辑  收藏  举报