Orleans基础知识以及使用示例

Orleans简介

Orleans 是一个与ABP齐名，支持有状态云生应用/服务水平伸缩的基于Virtual Actor 模型的.NET分布式应用框架。

Actor模型

简单来讲：Actor模型 = 状态 + 行为 + 消息。一个应用/服务由多个Actor组成，每个Actor都是一个独立的运行单元，拥有隔离的运行空间，在隔离的空间内，其有独立的状态和行为，不被外界干预，Actor之间通过消息进行交互，而同一时刻，每个Actor只能被单个线程执行，这样既有效避免了数据共享和并发问题，又确保了应用的伸缩性。

另外Actor基于事件驱动模型进行异步通信，性能良好。且位置透明，无论Actor是在本机亦或是在集群中的其他机器，都可以直接进行透明调用。

因此Actor模型赋予了应用/服务的生命力（有状态）、高并发的处理能力和弹性伸缩能力。

然而Actor模型作为一个偏底层的技术框架，对于开发者来说，需要有一定分布式应用的开发经验，才能用好Actor（包括Actor的生命周期管理，状态管理等等）。为了进一步简化分布式编程，微软的研究人员引入了 Virtual Actor 模型概念，简单来讲Virtual Actor模型是对Actor模型的进一步封装和抽象。其与Actor模型的最大的区别在于，Actor的物理实例完全被抽象出来，并由Virtual Actor所在的运行时自动管理。

Orleans 就是作为一款面向.NET的Virtual Actor模型的实现框架，提供了开发者友好的编程方式，简化了分布式应用的开发成本。在Orleans中Virtual Actor由Grain来体现。

Orleans框架的基本构成

Grains(颗粒)

Orleans模型里，每一个actor有专门的类代表它，叫做Grain，这个grain类就是模拟通信场景中的”人”

所有通过Orleans建立的应用程序的基本单位都是Grains. 也可以理解为任何Orleans程序都是由一个一个的Grain组成的. Grain是一个由用户自定义标识,行为和状态组成的实体. 标识是用户自定义的键(Key),其他应用程序或Grain通过键来调用该Grain. Grains是通过强类型接口(协议)与其他Grains或客户端进行通信. Graint是实现一个或多个这些接口的实例.

Orleans为了解决多线程带来的“资源竞争”等问题，在Orleans框架内，它保证每个grain类符合以下行为规范：

　　A．发往同一个grain类实例的任何消息都会在固定线程内执行。

　　B． grain类按照接受消息的先后，依次处理消息。在任意时间点，一个grain实例只处理一个消息。

　　C． grain实例内的字段属性，只能由实例本身访问。外界不能访问。

Silos(筒仓)

Silos是Orleans运行时的主要组件，Silos 是托管和执行 Grains 的容器。Orleans 通过 Silos 创建和管理 Grains 对象，并且执行 Grains 对象，客户端仅通过 Grains 定义的接口去调用。从而将 Grains 的对象状态封装起来，只公开 Grains 声明的接口方法。

Orleans 运行时会根据需要自行实例化或管理 Grains 对象。会将长期不使用的 Grains 对象从内存中释放。当 Grain 出现异常时会自动恢复。Orleans 运行时会自动管理 Grain 的整个生命周期，使得开发人员可以专注业务开发中。

通常, 一组silo是以集群方式运行的, 并以此来实现可伸缩性和容错性. 当这些silo作为集群方式运行的时候,silo之间彼此协调分配工作, 检测故障以及故障恢复. Orleans运行时使得集群中的Grian能够像在一个进程中一样彼此相互通信.

Clients(客户端)

客户端又称 Grains 客户端，即调用 Grains 程序代码。客户端分为两种：一种与 Silos 存在相同进程中，即共同托管的客户端；另外一种是运行 Silos 外的进程中，即外部客户端。

简单Orleans持久化解决方案示例

项目环境：.Net Core3.1+MySql+Orleans

示例项目源码地址：https://github.com/wswind/learn-orleans

此示例采用的是03.MultiGrain

包含 Grains 接口的类库 —— GrainInterfaces

包含 Grains 类库 —— Grains

Silos 控制台应用程序 —— Silo

Client 控制台应用程序 —— Client

Silo项目

作为Orleans服务端程序，单独启动，依赖Microsoft.Orleans.Server、Microsoft.Orleans.EventSourcing包，如果需要使用Mysql实现持久化存储，则还需依赖MySql.Data包。主要相关配置代码如下：

private static async Task<ISiloHost> StartSilo()
{
    // define the cluster configuration
    var builder = new SiloHostBuilder()
        // 因为是本地开发， silo 使用 localhost 集群
        .UseLocalhostClustering()
        //配置存储提供程序--内存存储
        .AddMemoryGrainStorage("DevStore")
        //配置存储提供程序--AdoNet持久化存储
        .AddAdoNetGrainStorage("OrleansStorage", options =>
        {
            //options.Invariant = "System.Data.SqlClient";
            options.Invariant = "MySql.Data.MySqlClient";
            //options.ConnectionString = "Server=.;Database=o3;Trusted_Connection=True;";
            options.ConnectionString = "Server=localhost;DataBase=graintest;uid=root;pwd=12345678;pooling=true;port=3306;CharSet=utf8mb3;sslMode=None;";
            options.UseJsonFormat = true;//指定使用Json格式序列化存储grain状态
        })
        //配置集群Id 和 服务Id
        .Configure<ClusterOptions>(options =>
        {
            options.ClusterId = "dev";//获取或设置群集标识
            options.ServiceId = "OrleansBasics";//获取或设置此服务的唯一标识符，该标识符应在部署和重新部署后继续存在
        })
        //应用程序部分：只需引用我们使用的 Grain 实现
        .ConfigureApplicationParts(parts => parts.AddApplicationPart(typeof(HelloGrain).Assembly).WithReferences())
        //配置日志输出到控制台
        .ConfigureLogging(logging => logging.AddConsole());

    var host = builder.Build();
    await host.StartAsync();
    return host;
}

GrainInterfaces项目

作为Orleans项目的公共接口，依赖Microsoft.Orleans.Core.Abstractions、Microsoft.Orleans.CodeGenerator.MSBuild、Microsoft.Orleans.Core包

定义Grain的行为接口：IHello : Orleans.IGrainWithIntegerKey

public interface IHello : Orleans.IGrainWithIntegerKey
{
    Task<string> SayHello(string greeting);
    Task AddCount();
    Task<int> GetCount();
}

Grains项目

作为Orleans的Grains的具体实现，依赖Microsoft.Orleans.Persistence.AdoNet、Microsoft.Orleans.CodeGenerator.MSBuild和Microsoft.Orleans.Core.Abstractions包。

定义需要持久化存储的Grain状态数据类：PersistentData

public class PersistentData
{
    public int Count { get; set; }
}

定义具体的Grain实体类：HelloGrain : Grain<PersistentData>, IHello

从Grain <T>继承的Grain类（其中T是需要持久化的特定于应用程序的状态数据类型）将从指定的存储区自动加载它们的状态。

同时需要指定持久化存储的配置名称[StorageProvider(ProviderName= "OrleansStorage")]

并重写OnActivateAsync()和OnDeactivateAsync()方法，确保Grain实例在每次激活和未激活状态下的数据持久化。

[StorageProvider(ProviderName= "OrleansStorage")]
public class HelloGrain : Grain<PersistentData>, IHello
{
    private readonly ILogger logger;

    public override Task OnActivateAsync()
    {
        this.ReadStateAsync();
        return base.OnActivateAsync();
    }

    public override Task OnDeactivateAsync()
    {
        this.WriteStateAsync();
        return base.OnDeactivateAsync();
    }

    public HelloGrain(ILogger<HelloGrain> logger)
    {
        this.logger = logger;
        
    }

    public async Task AddCount()
    {
        this.State.Count ++;
        await this.WriteStateAsync();
    }

    public Task<int> GetCount()
    {
        return Task.FromResult(this.State.Count);
    }

    Task<string> IHello.SayHello(string greeting)
    {
        logger.LogInformation($"\n SayHello message received: greeting = '{greeting}'");
        return Task.FromResult($"\n Client said: '{greeting}', so HelloGrain says: Hello!");
    }
}

Client项目

作为Orleans项目的客户端程序，单独启动，负责调用Orleans的Silo服务端，依赖Microsoft.Orleans.Client包。主要相关配置代码如下：

private static async Task<IClusterClient> ConnectClient()
{
    IClusterClient client;
    client = new ClientBuilder()
        // 这里配置与 Silo 相同
        .UseLocalhostClustering()
        //与 Silo 配置的服务一样，否则客户端会连接失败
        .Configure<ClusterOptions>(options =>
        {
            options.ClusterId = "dev";
            options.ServiceId = "OrleansBasics";
        })
        .ConfigureLogging(logging => logging.AddConsole())
        .Build();

    await client.Connect();
    Console.WriteLine("Client successfully connected to silo host \n");
    return client;
}

创建好客户端连接之后，即可通过客户端连接执行多个Grains的操作（节省篇幅，只列出部分关键代码）

private static async Task DoClientWork(IClusterClient client)
{
   
    var client1 = client.GetGrain<IHello>(0);
    var client2 = client.GetGrain<IHello>(1);

    //https://dotnet.github.io/orleans/Documentation/grains/grain_identity.html
    var id1 = client1.GetGrainIdentity().GetPrimaryKeyLong(out string keyExt);
    var id2 = client2.GetGrainIdentity().GetPrimaryKeyLong(out string keyExt2);

    Console.WriteLine(id1);
    Console.WriteLine(keyExt);
    Console.WriteLine(id2);
    Console.WriteLine(keyExt2);

    await client1.AddCount();
    var count1 = await client1.GetCount();
    Console.WriteLine("count1:{0}", count1);

    await client2.AddCount();
    var count2 = await client2.GetCount();
    Console.WriteLine("count2:{0}", count2);
    await client2.AddCount();
    var count3 = await client2.GetCount();
    Console.WriteLine("count3:{0}", count3);

}

数据库初始化

要使Orleans代码在给定的关系数据库后端发挥作用，还需要初始化一个对应的数据库，并与代码兼容。这是通过运行供应商特定的数据库创建脚本来完成的。这些脚本位于OrleansSqlUtils NuGet包中，随每个Orleans版本一起发布。目前有两个数据库脚本：

SQL Server - CreateOrleansTables_SqlServer.sql。AdoInvariant是System.Data.SqlClient。

MySQL - CreateOrleansTables_MySql.sql。AdoInvariant是MySql.Data.MySqlClient。

但因为此次示例是在Mac上使用Rider开发，不方便打开NuGet包中的内容，可以通过以下地址找到对应数据库初始化脚本：

https://github.com/dotnet/orleans/tree/main/src/AdoNet/Shared/MySQL-Main.sql

/*
Implementation notes:

1) The general idea is that data is read and written through Orleans specific queries.
   Orleans operates on column names and types when reading and on parameter names and types when writing.

2) The implementations *must* preserve input and output names and types. Orleans uses these parameters to reads query results by name and type.
   Vendor and deployment specific tuning is allowed and contributions are encouraged as long as the interface contract
   is maintained.

3) The implementation across vendor specific scripts *should* preserve the constraint names. This simplifies troubleshooting
   by virtue of uniform naming across concrete implementations.

5) ETag for Orleans is an opaque column that represents a unique version. The type of its actual implementation
   is not important as long as it represents a unique version. In this implementation we use integers for versioning

6) For the sake of being explicit and removing ambiguity, Orleans expects some queries to return either TRUE as >0 value
   or FALSE as =0 value. That is, affected rows or such does not matter. If an error is raised or an exception is thrown
   the query *must* ensure the entire transaction is rolled back and may either return FALSE or propagate the exception.
   Orleans handles exception as a failure and will retry.

7) The implementation follows the Extended Orleans membership protocol. For more information, see at:
        https://docs.microsoft.com/dotnet/orleans/implementation/cluster-management
        https://github.com/dotnet/orleans/blob/main/src/Orleans.Core/SystemTargetInterfaces/IMembershipTable.cs
*/
-- This table defines Orleans operational queries. Orleans uses these to manage its operations,
-- these are the only queries Orleans issues to the database.
-- These can be redefined (e.g. to provide non-destructive updates) provided the stated interface principles hold.
CREATE TABLE OrleansQuery
(
    QueryKey VARCHAR(64) NOT NULL,
    QueryText VARCHAR(8000) NOT NULL,

    CONSTRAINT OrleansQuery_Key PRIMARY KEY(QueryKey)
);

https://github.com/dotnet/orleans/blob/main/src/AdoNet/Orleans.Persistence.AdoNet/MySQL-Persistence.sql

-- The design criteria for this table are:
--
-- 1. It can contain arbitrary content serialized as binary, XML or JSON. These formats
-- are supported to allow one to take advantage of in-storage processing capabilities for
-- these types if required. This should not incur extra cost on storage.
--
-- 2. The table design should scale with the idea of tens or hundreds (or even more) types
-- of grains that may operate with even hundreds of thousands of grain IDs within each
-- type of a grain.
--
-- 3. The table and its associated operations should remain stable. There should not be
-- structural reason for unexpected delays in operations. It should be possible to also
-- insert data reasonably fast without resource contention.
--
-- 4. For reasons in 2. and 3., the index should be as narrow as possible so it fits well in
-- memory and should it require maintenance, isn't resource intensive. For this
-- reason the index is narrow by design (ideally non-clustered). Currently the entity
-- is recognized in the storage by the grain type and its ID, which are unique in Orleans silo.
-- The ID is the grain ID bytes (if string type UTF-8 bytes) and possible extension key as UTF-8
-- bytes concatenated with the ID and then hashed.
--
-- Reason for hashing: Database engines usually limit the length of the column sizes, which
-- would artificially limit the length of IDs or types. Even when within limitations, the
-- index would be thick and consume more memory.
--
-- In the current setup the ID and the type are hashed into two INT type instances, which
-- are made a compound index. When there are no collisions, the index can quickly locate
-- the unique row. Along with the hashed index values, the NVARCHAR(nnn) values are also
-- stored and they are used to prune hash collisions down to only one result row.
--
-- 5. The design leads to duplication in the storage. It is reasonable to assume there will
-- a low number of services with a given service ID operational at any given time. Or that
-- compared to the number of grain IDs, there are a fairly low number of different types of
-- grain. The catch is that were these data separated to another table, it would make INSERT
-- and UPDATE operations complicated and would require joins, temporary variables and additional
-- indexes or some combinations of them to make it work. It looks like fitting strategy
-- could be to use table compression.
--
-- 6. For the aforementioned reasons, grain state DELETE will set NULL to the data fields
-- and updates the Version number normally. This should alleviate the need for index or
-- statistics maintenance with the loss of some bytes of storage space. The table can be scrubbed
-- in a separate maintenance operation.
--
-- 7. In the storage operations queries the columns need to be in the exact same order
-- since the storage table operations support optionally streaming.
CREATE TABLE OrleansStorage
(
    -- These are for the book keeping. Orleans calculates
    -- these hashes (see RelationalStorageProvide implementation),
    -- which are signed 32 bit integers mapped to the *Hash fields.
    -- The mapping is done in the code. The
    -- *String columns contain the corresponding clear name fields.
    --
    -- If there are duplicates, they are resolved by using GrainIdN0,
    -- GrainIdN1, GrainIdExtensionString and GrainTypeString fields.
    -- It is assumed these would be rarely needed.
    GrainIdHash                INT NOT NULL,
    GrainIdN0                BIGINT NOT NULL,
    GrainIdN1                BIGINT NOT NULL,
    GrainTypeHash            INT NOT NULL,
    GrainTypeString            NVARCHAR(512) NOT NULL,
    GrainIdExtensionString    NVARCHAR(512) NULL,
    ServiceId                NVARCHAR(150) NOT NULL,

    -- The usage of the Payload records is exclusive in that
    -- only one should be populated at any given time and two others
    -- are NULL. The types are separated to advantage on special
    -- processing capabilities present on database engines (not all might
    -- have both JSON and XML types.
    --
    -- One is free to alter the size of these fields.
    PayloadBinary    BLOB NULL,
    PayloadXml        LONGTEXT NULL,
    PayloadJson        LONGTEXT NULL,

    -- Informational field, no other use.
    ModifiedOn DATETIME NOT NULL,

    -- The version of the stored payload.
    Version INT NULL

    -- The following would in principle be the primary key, but it would be too thick
    -- to be indexed, so the values are hashed and only collisions will be solved
    -- by using the fields. That is, after the indexed queries have pinpointed the right
    -- rows down to [0, n] relevant ones, n being the number of collided value pairs.
) ROW_FORMAT = COMPRESSED KEY_BLOCK_SIZE = 16;
ALTER TABLE OrleansStorage ADD INDEX IX_OrleansStorage (GrainIdHash, GrainTypeHash);

-- The following alters the column to JSON format if MySQL is at least of version 5.7.8.
-- See more at https://dev.mysql.com/doc/refman/5.7/en/json.html for JSON and
-- http://dev.mysql.com/doc/refman/5.7/en/comments.html for the syntax.
/*!50708 ALTER TABLE OrleansStorage MODIFY COLUMN PayloadJson JSON */;

DELIMITER $$

CREATE PROCEDURE ClearStorage
(
    in _GrainIdHash INT,
    in _GrainIdN0 BIGINT,
    in _GrainIdN1 BIGINT,
    in _GrainTypeHash INT,
    in _GrainTypeString NVARCHAR(512),
    in _GrainIdExtensionString NVARCHAR(512),
    in _ServiceId NVARCHAR(150),
    in _GrainStateVersion INT
)
BEGIN
    DECLARE _newGrainStateVersion INT;
    DECLARE EXIT HANDLER FOR SQLEXCEPTION BEGIN ROLLBACK; RESIGNAL; END;
    DECLARE EXIT HANDLER FOR SQLWARNING BEGIN ROLLBACK; RESIGNAL; END;

    SET _newGrainStateVersion = _GrainStateVersion;

    -- Default level is REPEATABLE READ and may cause Gap Lock issues
    SET TRANSACTION ISOLATION LEVEL READ COMMITTED;
    START TRANSACTION;
    UPDATE OrleansStorage
    SET
        PayloadBinary = NULL,
        PayloadJson = NULL,
        PayloadXml = NULL,
        Version = Version + 1
    WHERE
        GrainIdHash = _GrainIdHash AND _GrainIdHash IS NOT NULL
        AND GrainTypeHash = _GrainTypeHash AND _GrainTypeHash IS NOT NULL
        AND GrainIdN0 = _GrainIdN0 AND _GrainIdN0 IS NOT NULL
        AND GrainIdN1 = _GrainIdN1 AND _GrainIdN1 IS NOT NULL
        AND GrainTypeString = _GrainTypeString AND _GrainTypeString IS NOT NULL
        AND ((_GrainIdExtensionString IS NOT NULL AND GrainIdExtensionString IS NOT NULL AND GrainIdExtensionString = _GrainIdExtensionString) OR _GrainIdExtensionString IS NULL AND GrainIdExtensionString IS NULL)
        AND ServiceId = _ServiceId AND _ServiceId IS NOT NULL
        AND Version IS NOT NULL AND Version = _GrainStateVersion AND _GrainStateVersion IS NOT NULL
        LIMIT 1;

    IF ROW_COUNT() > 0
    THEN
        SET _newGrainStateVersion = _GrainStateVersion + 1;
    END IF;

    SELECT _newGrainStateVersion AS NewGrainStateVersion;
    COMMIT;
END$$

DELIMITER $$
CREATE PROCEDURE WriteToStorage
(
    in _GrainIdHash INT,
    in _GrainIdN0 BIGINT,
    in _GrainIdN1 BIGINT,
    in _GrainTypeHash INT,
    in _GrainTypeString NVARCHAR(512),
    in _GrainIdExtensionString NVARCHAR(512),
    in _ServiceId NVARCHAR(150),
    in _GrainStateVersion INT,
    in _PayloadBinary BLOB,
    in _PayloadJson LONGTEXT,
    in _PayloadXml LONGTEXT
)
BEGIN
    DECLARE _newGrainStateVersion INT;
    DECLARE _rowCount INT;
    DECLARE EXIT HANDLER FOR SQLEXCEPTION BEGIN ROLLBACK; RESIGNAL; END;
    DECLARE EXIT HANDLER FOR SQLWARNING BEGIN ROLLBACK; RESIGNAL; END;

    SET _newGrainStateVersion = _GrainStateVersion;

    -- Default level is REPEATABLE READ and may cause Gap Lock issues
    SET TRANSACTION ISOLATION LEVEL READ COMMITTED;
    START TRANSACTION;

    -- Grain state is not null, so the state must have been read from the storage before.
    -- Let's try to update it.
    --
    -- When Orleans is running in normal, non-split state, there will
    -- be only one grain with the given ID and type combination only. This
    -- grain saves states mostly serially if Orleans guarantees are upheld. Even
    -- if not, the updates should work correctly due to version number.
    --
    -- In split brain situations there can be a situation where there are two or more
    -- grains with the given ID and type combination. When they try to INSERT
    -- concurrently, the table needs to be locked pessimistically before one of
    -- the grains gets @GrainStateVersion = 1 in return and the other grains will fail
    -- to update storage. The following arrangement is made to reduce locking in normal operation.
    --
    -- If the version number explicitly returned is still the same, Orleans interprets it so the update did not succeed
    -- and throws an InconsistentStateException.
    --
    -- See further information at https://docs.microsoft.com/dotnet/orleans/grains/grain-persistence.
    IF _GrainStateVersion IS NOT NULL
    THEN
        UPDATE OrleansStorage
        SET
            PayloadBinary = _PayloadBinary,
            PayloadJson = _PayloadJson,
            PayloadXml = _PayloadXml,
            ModifiedOn = UTC_TIMESTAMP(),
            Version = Version + 1
        WHERE
            GrainIdHash = _GrainIdHash AND _GrainIdHash IS NOT NULL
            AND GrainTypeHash = _GrainTypeHash AND _GrainTypeHash IS NOT NULL
            AND GrainIdN0 = _GrainIdN0 AND _GrainIdN0 IS NOT NULL
            AND GrainIdN1 = _GrainIdN1 AND _GrainIdN1 IS NOT NULL
            AND GrainTypeString = _GrainTypeString AND _GrainTypeString IS NOT NULL
            AND ((_GrainIdExtensionString IS NOT NULL AND GrainIdExtensionString IS NOT NULL AND GrainIdExtensionString = _GrainIdExtensionString) OR _GrainIdExtensionString IS NULL AND GrainIdExtensionString IS NULL)
            AND ServiceId = _ServiceId AND _ServiceId IS NOT NULL
            AND Version IS NOT NULL AND Version = _GrainStateVersion AND _GrainStateVersion IS NOT NULL
            LIMIT 1;

        IF ROW_COUNT() > 0
        THEN
            SET _newGrainStateVersion = _GrainStateVersion + 1;
            SET _GrainStateVersion = _newGrainStateVersion;
        END IF;
    END IF;

    -- The grain state has not been read. The following locks rather pessimistically
    -- to ensure only on INSERT succeeds.
    IF _GrainStateVersion IS NULL
    THEN
        INSERT INTO OrleansStorage
        (
            GrainIdHash,
            GrainIdN0,
            GrainIdN1,
            GrainTypeHash,
            GrainTypeString,
            GrainIdExtensionString,
            ServiceId,
            PayloadBinary,
            PayloadJson,
            PayloadXml,
            ModifiedOn,
            Version
        )
        SELECT * FROM ( SELECT
            _GrainIdHash,
            _GrainIdN0,
            _GrainIdN1,
            _GrainTypeHash,
            _GrainTypeString,
            _GrainIdExtensionString,
            _ServiceId,
            _PayloadBinary,
            _PayloadJson,
            _PayloadXml,
            UTC_TIMESTAMP(),
            1) AS TMP
        WHERE NOT EXISTS
        (
            -- There should not be any version of this grain state.
            SELECT 1
            FROM OrleansStorage
            WHERE
                GrainIdHash = _GrainIdHash AND _GrainIdHash IS NOT NULL
                AND GrainTypeHash = _GrainTypeHash AND _GrainTypeHash IS NOT NULL
                AND GrainIdN0 = _GrainIdN0 AND _GrainIdN0 IS NOT NULL
                AND GrainIdN1 = _GrainIdN1 AND _GrainIdN1 IS NOT NULL
                AND GrainTypeString = _GrainTypeString AND _GrainTypeString IS NOT NULL
                AND ((_GrainIdExtensionString IS NOT NULL AND GrainIdExtensionString IS NOT NULL AND GrainIdExtensionString = _GrainIdExtensionString) OR _GrainIdExtensionString IS NULL AND GrainIdExtensionString IS NULL)
                AND ServiceId = _ServiceId AND _ServiceId IS NOT NULL
        ) LIMIT 1;

        IF ROW_COUNT() > 0
        THEN
            SET _newGrainStateVersion = 1;
        END IF;
    END IF;

    SELECT _newGrainStateVersion AS NewGrainStateVersion;
    COMMIT;
END$$

DELIMITER ;

INSERT INTO OrleansQuery(QueryKey, QueryText)
VALUES
(
    'ReadFromStorageKey',
    'SELECT
        PayloadBinary,
        PayloadXml,
        PayloadJson,
        UTC_TIMESTAMP(),
        Version
    FROM
        OrleansStorage
    WHERE
        GrainIdHash = @GrainIdHash
        AND GrainTypeHash = @GrainTypeHash AND @GrainTypeHash IS NOT NULL
        AND GrainIdN0 = @GrainIdN0 AND @GrainIdN0 IS NOT NULL
        AND GrainIdN1 = @GrainIdN1 AND @GrainIdN1 IS NOT NULL
        AND GrainTypeString = @GrainTypeString AND GrainTypeString IS NOT NULL
        AND ((@GrainIdExtensionString IS NOT NULL AND GrainIdExtensionString IS NOT NULL AND GrainIdExtensionString = @GrainIdExtensionString) OR @GrainIdExtensionString IS NULL AND GrainIdExtensionString IS NULL)
        AND ServiceId = @ServiceId AND @ServiceId IS NOT NULL
        LIMIT 1;'
);

INSERT INTO OrleansQuery(QueryKey, QueryText)
VALUES
(
    'WriteToStorageKey','
    call WriteToStorage(@GrainIdHash, @GrainIdN0, @GrainIdN1, @GrainTypeHash, @GrainTypeString, @GrainIdExtensionString, @ServiceId, @GrainStateVersion, @PayloadBinary, @PayloadJson, @PayloadXml);'
);

INSERT INTO OrleansQuery(QueryKey, QueryText)
VALUES
(
    'ClearStorageKey','
    call ClearStorage(@GrainIdHash, @GrainIdN0, @GrainIdN1, @GrainTypeHash, @GrainTypeString, @GrainIdExtensionString, @ServiceId, @GrainStateVersion);'
);

将这两个脚本依次下载运行，既可完成Orleans数据库的初始化。

运行

接着分别启动运行Silo和Client项目，则可以在数据库OrleansStorage表中看到相应的Grain状态数据持久化效果。

事件溯源(Event Sourcing)

事件溯源是一种架构模式，是借鉴数据库事件日志的一种数据持久方式。

　　它存在以下几个特点：

整个系统以事件为驱动，所有业务都由事件驱动来完成。

系统的数据以事件为基础，事件要保存在某种存储上。

业务数据只是一些由事件产生的视图，不一定要保存到数据库中。

Event Sourcing遵循一个简单的思想,就是存储的时候只存储变化量,而不存储最终结果.需要最终结果的地方,就必须提取所有的变化量以及初始状态,让它们相加得到最终结果.

a.不保存对象的最新状态，而是保存对象产生的所有事件；

b.通过事件溯源（Event Sourcing）得到对象最新状态；

由于只存储变化量,意味着数据只增不减,意味着数据存储后就不会被更改,意味着高并发和高吞吐量.因为数据库的数据永远不变，所以多线程操作都不需要加锁。可是这里隐藏着另一个风险，就是读取的数据不一定是最新的。这个"非最新"的确是个难题，不过好消息是，如果一直读，最终能够读到最新的，这就是"最终一致性"。

日志一致性提供程序

Orleans实现了Event Sourcing机制，而且它的

StateStorage.LogConsistencyProvider（状态存储）

使用可单独配置的标准存储提供程序来存储 grain 状态快照。

保存在存储中的数据是一个对象，其中包含 grain 状态（由 JournaledGrain 的第一个类型参数指定）和一些元数据（版本号，以及用于避免在存储访问失败时事件重复的特殊标记）。

由于每次我们访问存储时都会读取/写入整个 grain 状态，因此该提供程序不适合用于 grain 状态很大的对象。

LogStorage.LogConsistencyProvider（日志存储）

使用可单独配置的标准存储提供程序将完整的事件序列存储为单个对象。

保存在存储中的数据是一个对象，其中包含此提供程序支持 RetrieveConfirmedEvents。所有事件始终可用并保存在内存中。

由于每次我们访问存储时都会读取/写入整个事件序列，因此该提供程序不适合在生产环境中使用，除非保证事件序列相当短。此提供程序的主要用途是演示事件溯源的语义以及示例/测试环境。

CustomStorage.LogConsistencyProvider（自定义存储）

允许开发人员插入其存储接口，然后一致性协议将在适当的时间调用该接口。此提供程序不会对存储的内容是状态快照还是事件做出具体的假设 – 由程序员控制这种选择（可以存储快照和/或事件）。

若要使用此提供程序，grain 必须如前所述派生自 JournaledGrain<TGrainState,TEventBase>，此外必须实现以下接口：

public interface ICustomStorageInterface<StateType, EventType>
{
    Task<KeyValuePair<int, StateType>> ReadStateFromStorage();

    Task<bool> ApplyUpdatesToStorage(
        IReadOnlyList<EventType> updates,
        int expectedVersion);
}

此提供程序不支持 RetrieveConfirmedEvents。当然，由于开发人员仍然控制着存储接口，因此他们不需要一开始就调用此方法，而可以实现事件检索。

基于Orleans的Event Sourcing持久化解决方案示例

项目环境：.Net Core3.1+MySql+Orleans Event Sourcing

示例项目源码地址：https://github.com/wswind/learn-orleans/tree/master/04.EventSourcing

此Event Sourcing示例解决方案采用的是LogStorage.LogConsistencyProvider日志一致性提供程序，但没有提供持久化存储，因此需要进行一定程度的改造。

Silo项目

作为Orleans服务端程序，单独启动，依赖Microsoft.Orleans.Server、Microsoft.Orleans.EventSourcing包，添加依赖MySql.Data包以支持Mysql持久化存储，Program.cs中相关代码配置如下：

private static async Task<ISiloHost> StartSilo()
{
    // define the cluster configuration
    var builder = new SiloHostBuilder()
        .UseLocalhostClustering()
        .AddMemoryGrainStorageAsDefault()
        .AddMemoryGrainStorage("DevStore")
        .AddAdoNetGrainStorage("OrleansStorage", options =>
        {
            options.Invariant = "MySql.Data.MySqlClient";
            //options.Invariant = "System.Data.SqlClient";
            options.ConnectionString = "Server=localhost;DataBase=eventsourcingtest;uid=root;pwd=12345678;pooling=true;port=3306;CharSet=utf8;sslMode=None;";
            //options.ConnectionString = "Server=.;Database=o3;Trusted_Connection=True;";
            options.UseJsonFormat = true;
        })
        .AddLogStorageBasedLogConsistencyProvider("LogStorage")
        .Configure<ClusterOptions>(options =>
        {
            options.ClusterId = "dev";
            options.ServiceId = "OrleansBasics";
        })
        .ConfigureApplicationParts(parts => parts.AddApplicationPart(typeof(HelloGrain).Assembly).WithReferences())
        .ConfigureLogging(logging => logging.AddConsole());

    var host = builder.Build();
    await host.StartAsync();
    return host;
}

GrainInterfaces项目

作为Orleans项目的公共接口，依赖Microsoft.Orleans.Core.Abstractions、Microsoft.Orleans.CodeGenerator.MSBuild包

定义Grain的行为接口：IHello : Orleans.IGrainWithIntegerKey

这里为了使示例看起来更简单明了一些，删掉了原示例接口程序中2个无用的接口。

public interface IHello : Orleans.IGrainWithIntegerKey
{
    Task<int> GetCount();//获取某个Grain状态属性接口
    Task NewEvent(EventData @event);//将新事件持久化存储接口
}

定义Grain相关的公共事件类，为了使最终示例效果更明显一些，加入了一个string类型的Who属性

[Serializable]
public class EventData
{
    public EventData()
    {
        When = DateTime.UtcNow;
    }

    public DateTime When;
    public string Who;
}

再定义2个基于此事件类的派生事件类

public class EventDataAdd : EventData
{
    public int AddCount;
}

public class EventDataMinus : EventData
{
    public int MinusCount;
}

Grains项目

作为Orleans的Grains的具体实现，依赖Microsoft.Orleans.Persistence.AdoNet、Microsoft.Orleans.Core.Abstractions、Microsoft.Orleans.CodeGenerator.MSBuild和Microsoft.Orleans.EventSourcing包

日志式 Grain 派生自 JournaledGrain<TGrainState,TEventBase>，具有以下类型参数：

表示 grain 状态的所有状态和事件对象都应该可序列化（因为日志一致性提供程序可能需要持久保存它们，和/或在通知消息中发送它们）。

简单说，支持Event Sourcing的Grain类需要派生自JournaledGrain<TGrainState,TEventBase>，它需要有两个泛型参数，一个是Grain的状态类，一个是与Grain相关的事件类

定义具体的Grain实体类：HelloGrain : JournaledGrain<HelloState, EventData>, IHello

同时需要指定持久化存储的配置名称[StorageProvider(ProviderName= "OrleansStorage")]

[StorageProvider(ProviderName = "OrleansStorage")]
[LogConsistencyProvider(ProviderName = "LogStorage")]
public class HelloGrain : JournaledGrain<HelloState, EventData>, IHello
{
    private readonly ILogger logger;

    public HelloGrain(ILogger<HelloGrain> logger)
    {
        this.logger = logger;
    }

    public Task<int> GetCount()
    {
        //读取Grain状态属性
        //为了读取当前 grain 状态并确定其版本号，JournaledGrain 提供了属性
        //GrainState State { get; }int Version { get; }
        //版本号始终等于已确认事件的总数，状态是将所有已确认事件应用于初始状态后的结果。
        return Task.FromResult(this.State.Count);
    }

    public async Task NewEvent(EventData @event)
    {
        //RaiseEvent 将事件写入存储，但不等待写入完成。
        RaiseEvent(@event);
        //对于许多应用程序而言，必须等待我们收到已持久保存事件的确认。 
        //在这种情况下，我们始终会通过等待 ConfirmEvents 来跟进
        await ConfirmEvents();
        //即使不显式调用 ConfirmEvents，事件最终也会得到确认 - 确认会在后台自动发生
    }

}

重要说明：应用程序永远不应直接修改 State 返回的对象。该对象仅供读取。相反，当应用程序想要修改状态时，它必须通过RaiseEvent(引发事件)来间接修改。

定义Grain的状态类：HelloState

public class HelloState
{
    public int Count { get; set; }//状态属性
    
    //更新状态以响应事件
    public void Apply(EventDataAdd addData)
    {
        Count += addData.AddCount;
    }
    public void Apply(EventDataMinus minusData)
    {
        Count -= minusData.MinusCount;
    }
}

每当RaiseEvent(引发事件)时，运行时都会自动更新 grain 状态。应用程序无需在引发事件后显式更新状态。但是，(a) GrainState 类可以在 (b) 也可以在Grain类重写 TransitionState 函数。

protected override void TransitionState(
    State state, EventType @event)
{
   // code that updates the state
}

Client项目

作为Orleans项目的客户端程序，单独启动，负责调用Orleans的Silo服务端，依赖Microsoft.Orleans.Client包，Program.cs中主要业务逻辑部分代码如下：

private static async Task DoClientWork(IClusterClient client)
{
    //获取Grain
    var client1 = client.GetGrain<IHello>(0);
    //制作事件对象
    EventDataAdd dataAdd = new EventDataAdd
    {
        Who = "syb",
        AddCount = 3
    };
    //将事件对象发给Grain，触发写入存储及Grain状态更新
    await client1.NewEvent(dataAdd);
    //获取Grain状态属性，检查是否更新成功
    var count = await client1.GetCount();
    Console.WriteLine(count);
}

数据库初始化

同样需要通过运行供应商特定的数据库创建脚本，初始化一个与代码兼容的供Orleans使用的数据库。

https://github.com/dotnet/orleans/tree/main/src/AdoNet/Shared/MySQL-Main.sql

https://github.com/dotnet/orleans/blob/main/src/AdoNet/Orleans.Persistence.AdoNet/MySQL-Persistence.sql

运行

最后，依次运行Silo和Client项目，也可以多次修改Client事件对象参数并启动运行，则可以在Mysql数据库看到Orleans的Event Sourcing效果。

其中事件类型序列存储在PayloadJson字段，大致内容如下：

{"$id": "1", "Log": {"$type": "System.Collections.Generic.List`1[[OrleansBasics.EventData, GrainInterfaces]], System.Private.CoreLib", "$values": [{"$id": "2", "Who": "syb", "When": "2022-08-18T09:01:51.011686Z", "$type": "OrleansBasics.EventDataAdd, GrainInterfaces", "AddCount": 3}, {"$id": "3", "Who": "shiyibo", "When": "2022-08-18T09:02:30.636478Z", "$type": "OrleansBasics.EventDataAdd, GrainInterfaces", "AddCount": 5}]}, "$type": "Orleans.EventSourcing.LogStorage.LogStateWithMetaData`1[[OrleansBasics.EventData, GrainInterfaces]], Orleans.EventSourcing", "WriteVector": "", "GlobalVersion": 2}

本文为作者月井石原创，转载请注明出处～

posted @ 2022-11-26 12:33 月井石阅读(860) 评论(0) 编辑收藏举报

会员力量，点亮园子希望

刷新页面返回顶部

厚积博发

————不找借口失败，只找理由成功！

Orleans基础知识以及使用示例

Orleans简介

Actor模型

Orleans框架的基本构成

Grains(颗粒)

Silos(筒仓)

Clients(客户端)

简单Orleans持久化解决方案示例

Silo项目

GrainInterfaces项目

Grains项目

Client项目

数据库初始化

运行

事件溯源(Event Sourcing)

日志一致性提供程序

基于Orleans的Event Sourcing持久化解决方案示例

Silo项目

GrainInterfaces项目

Grains项目

Client项目

数据库初始化

运行

公告