记一次分布式服务框架错误调优

    最近业务组在开发程序时，遇到了一个诡异的错误，错误信息如下：

{"Disconnected before response received.tcp://139.217.0.107:8004ScsRemoteInvokeMessage: Teld.Sys.Service.Spi.IOrganizationService.GetType(...)"}

    内部错误信息是：

未能加载文件或程序集“Teld.Sys.Service.Core, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null”或它的某一个依赖项。系统找不到指定的文件。

    错误堆栈是：

Server stack trace:
  

在 Teld.Core.SCS.Communication.Messengers.RequestReplyMessenger`1.SendMessageAndWaitForResponse(IScsMessage message, Int32 timeoutMilliseconds) 位置 D:\Teld\05源代码\TFS\TTP\Main\20ISP\HSF\Src\Teld.Core.RPC.SCS\RPC\Impl\Communication\Messengers\RequestReplyMessenger.cs:行号 237
   在 Teld.Core.SCS.Communication.Messengers.RequestReplyMessenger`1.SendMessageAndWaitForResponse(IScsMessage message) 位置 D:\Teld\05源代码\TFS\TTP\Main\20ISP\HSF\Src\Teld.Core.RPC.SCS\RPC\Impl\Communication\Messengers\RequestReplyMessenger.cs:行号 195
   在 Teld.Core.SCSServices.Communication.RemoteInvokeProxy`2.Invoke(IMessage msg) 位置 D:\Teld\05源代码\TFS\TTP\Main\20ISP\HSF\Src\Teld.Core.RPC.SCS\RPC\SPI\Communication\RemoteInvokeProxy.cs:行号 118

Exception rethrown at [0]:
   在 System.Runtime.Remoting.Proxies.RealProxy.HandleReturnMessage(IMessage reqMsg, IMessage retMsg)
   在 System.Runtime.Remoting.Proxies.RealProxy.PrivateInvoke(MessageData& msgData, Int32 type)
   在 System.Object.GetType()
   在 System.Dynamic.DynamicMetaObject.get_RuntimeType()
   在 Microsoft.CSharp.RuntimeBinder.BinderHelper.IsWindowsRuntimeObject(DynamicMetaObject obj)
   在 Microsoft.CSharp.RuntimeBinder.CSharpInvokeMemberBinder.FallbackInvokeMember(DynamicMetaObject target, DynamicMetaObject[] args, DynamicMetaObject errorSuggestion)
   在 System.Dynamic.DynamicMetaObject.BindInvokeMember(InvokeMemberBinder binder, DynamicMetaObject[] args)
   在 System.Dynamic.InvokeMemberBinder.Bind(DynamicMetaObject target, DynamicMetaObject[] args)
   在 System.Dynamic.DynamicMetaObjectBinder.Bind(Object[] args, ReadOnlyCollection`1 parameters, LabelTarget returnLabel)
   在 System.Runtime.CompilerServices.CallSiteBinder.BindCore[T](CallSite`1 site, Object[] args)
   在 System.Dynamic.UpdateDelegates.UpdateAndExecute2[T0,T1,TRet](CallSite site, T0 arg0, T1 arg1)
   在 Teld.BIZ.User.Login.Service.LoginService.SetACCompayID(Object sessionInfo, ReqSource reqSource) 位置 C:\Users\dell\Desktop\Test\Teld.BIZ.User.Login.Service\LoginService.cs:行号 414
   在 CallSite.Target(Closure , CallSite , LoginService , Object , ReqSource )
   在 System.Dynamic.UpdateDelegates.UpdateAndExecuteVoid3[T0,T1,T2](CallSite site, T0 arg0, T1 arg1, T2 arg2)
   在 Teld.BIZ.User.Login.Service.LoginService.CreateSession(UserInfo userInfo, LoginInfo loginInfo) 位置 C:\Users\dell\Desktop\Test\Teld.BIZ.User.Login.Service\LoginService.cs:行号 329
   在 Teld.BIZ.User.Login.Service.LoginService.Login(LoginInfo loginInfo, LoginTypeEnum loginType) 位置 C:\Users\dell\Desktop\Test\Teld.BIZ.User.Login.Service\LoginService.cs:行号 97
   在 Teld.BIZ.User.Login.Service.LoginService.LoginWithPwd(LoginInfo loginInfo) 位置 C:\Users\dell\Desktop\Test\Teld.BIZ.User.Login.Service\LoginService.cs:行号 50
   在 Teld.BIZ.User.Login.Service.Tests.LoginServiceTests.Main() 位置 C:\Users\dell\Desktop\Test\Teld.BIZ.User.Login.ServiceTests\LoginServiceTests.cs:行号 51
   在 System.AppDomain._nExecuteAssembly(RuntimeAssembly assembly, String[] args)
   在 System.AppDomain.ExecuteAssembly(String assemblyFile, Evidence assemblySecurity, String[] args)
   在 Microsoft.VisualStudio.HostingProcess.HostProc.RunUsersAssembly()
   在 System.Threading.ThreadHelper.ThreadStart_Context(Object state)
   在 System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
   在 System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
   在 System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
   在 System.Threading.ThreadHelper.ThreadStart()

   先说一下系统背景，我司自主开发了一个SOA的分布式服务框架，并提供了一套客户端调用的SDK。SDK通过Remoting方式实现了远程的SOA服务调用，具体如下图：

    客户端（Consumer）发起远程调用的代码如下：

public class RemoteInvokeProxy<TProxy, TMessenger> : RealProxy where TMessenger : IMessenger
    {
        /// <summary>
        /// Messenger object that is used to send/receive messages.
        /// </summary>
        private readonly RequestReplyMessenger<TMessenger> _clientMessenger;
        private string _serviceVersion;
        public event ReportRemoteInvokeStatisticsEventHandler OnReportStatistics;
        private Func<IMessage, IMessage> errorHandler;
        private Func<IMessage, IMessage> sessionAttacher;
        /// <summary>
        /// Creates a new RemoteInvokeProxy object.
        /// </summary>
        /// <param name="clientMessenger">Messenger object that is used to send/receive messages</param>
        public RemoteInvokeProxy(RequestReplyMessenger<TMessenger> clientMessenger, Func<IMessage, IMessage> errorHandler = null,Func<IMessage, IMessage> sessionAttacher = null, string serviceVersion = "")
            : base(typeof(TProxy))
        {
            _clientMessenger = clientMessenger;
            _serviceVersion = serviceVersion;
            this.errorHandler = errorHandler;
            this.sessionAttacher = sessionAttacher;
        }

        public override System.Runtime.Remoting.ObjRef CreateObjRef(Type requestedType)
        {
            return base.CreateObjRef(requestedType);
        }

        public override object GetTransparentProxy()
        {
            return base.GetTransparentProxy();
        }

        public override void GetObjectData(System.Runtime.Serialization.SerializationInfo info, System.Runtime.Serialization.StreamingContext context)
        {
            base.GetObjectData(info, context);
        }

        /// <summary>
        /// Overrides message calls and translates them to messages to remote application.
        /// </summary>
        /// <param name="msg">Method invoke message (from RealProxy base class)</param>
        /// <returns>Method invoke return message (to RealProxy base class)</returns>
        public override IMessage Invoke(IMessage msg)
        {
            if (msg != null && sessionAttacher != null)
                msg = sessionAttacher(msg);
            DateTime start = DateTime.Now;
            ScsRemoteInvokeMessage requestMessage = null;
            var message = msg as IMethodCallMessage;
            if (message == null)
            {
                return null;
            }

            try
            {
                requestMessage = new ScsRemoteInvokeMessage
                {
                    ServiceClassName = typeof(TProxy).FullName,
                    MethodName = message.MethodName,
                    Parameters = message.Args,
                    Version = this._serviceVersion,
                    Properties = new System.Collections.Generic.Dictionary<string, string>(StringComparer.OrdinalIgnoreCase)
                };
                if (message.MethodBase!= null && message.MethodBase.IsGenericMethod)
                {
                    var tps= message.MethodBase.GetGenericArguments();
                    if (tps != null)
                    {
                        List<string> strTypes = new List<string>();
                       
                        foreach (var item in tps)
                        {
                            strTypes.Add(JsonConvert.SerializeObject(item));
                        }

                        requestMessage.GenericTypes = strTypes.ToArray();
                    }
                }

                if (msg.Properties != null)
                {
                    foreach (var item in msg.Properties.Keys)
                    {
                        requestMessage.Properties.Add(Convert.ToString(item), Convert.ToString(msg.Properties[item]));
                    }
                }

                var responseMessage = _clientMessenger.SendMessageAndWaitForResponse(requestMessage) as ScsRemoteInvokeReturnMessage;
                if (responseMessage == null)
                {
                    return null;
                }

                if (responseMessage.RemoteException != null)
                {
                    //return new ReturnMessage(responseMessage.RemoteException, message);
                    if (errorHandler != null)
                    {
                        var newValue = this.errorHandler.Invoke(msg) as ReturnMessage;
                        if (newValue != null)
                        {
                            if (newValue.Exception == null)
                                return InvokeResult.HandleResult(message, newValue.ReturnValue);
                            else
                                return new ReturnMessage(responseMessage.RemoteException, message);
                        }
                        else
                            return new ReturnMessage(responseMessage.RemoteException, message);
                    }
                    else
                        return new ReturnMessage(responseMessage.RemoteException, message);

                }
                else
                {
                    return InvokeResult.HandleResult(message, responseMessage.ReturnValue);
                }
            }
            catch (Exception ex)
            {
                if (errorHandler != null)
                {
                    var newValue = this.errorHandler.Invoke(msg);
                    if (newValue != null)
                        return newValue;
                    else
                        return new ReturnMessage(ex, message);
                }
                else
                    return new ReturnMessage(ex, message);
            }
            finally
            {
                try
                {
                    if (OnReportStatistics != null)
                        OnReportStatistics(new InvokeArgs(DateTime.Now - start, requestMessage));
                }
                catch
                { }
            }
        }

       
    }

    业务部门开发了一个OrganizationService服务，并部署到了SOA服务容器中，通过下面代码调用时，触发了上面的Error。

/// <summary>
        /// 获取核算组织
        /// </summary>
        /// <param name="companyId">公司内码</param>
        /// <param name="reqSource">请求来源</param>
        /// <returns></returns>
        private void SetACCompayID(dynamic sessionInfo, ReqSource reqSource)
        {
            if (reqSource == ReqSource.Backend && !string.IsNullOrEmpty(sessionInfo.CompanyId))
            {
                OrganizationInfo orgInfo = ServiceFactory<IOrganizationService>.GetService().GetOrganizationById(sessionInfo.CompanyId);

                if (orgInfo != null && string.IsNullOrEmpty(orgInfo.BusUnitID) == false)
                {
                    string busUnitIDs = string.Format("'{0}'", orgInfo.BusUnitID);

                    List<string> accompayIds = ServiceFactory<IBusinessUnitsSAPService>.GetService().GetBalanceCompanySys(busUnitIDs);

                    if (accompayIds != null && accompayIds.Count == 1)
                    {
                        sessionInfo.ACCompanyID = accompayIds[0];
                    }
                }
            }
        }

    此方法明明调用的是GetOrganizationById方法，    但为什么异常中显示的却是 IOrganizationService下的GetType方法呢？（通过异常信息可以看到：ScsRemoteInvokeMessage: Teld.Sys.Service.Spi.IOrganizationService.GetType(...)）

   通过仔细阅读代码发现，GetOrganizationById方法传入的参数是：sessionInfo.CompanyId，其中sessionInfo是dynamic类型的。因为没有强制转换类型，导致在运行时，需要获取IOrganizationService对象的类型，所以触发了GetType调用，但是此调用返回的类型在调用方没有，所以出现找不到程序集的错误。