XML数据库的尝试

首先祝大家新年快乐.身体健康,平安就是福气.

对于一般的个人迷你项目,数据量不大的时候,完全没有必要使用数据库,管理数据使用XML就可以了.

自己尝试写了一个XML数据库,插入1w条小记录,大概3M大小,然后将一半数据进行更新,大约耗时3秒钟.

XML数据库其实就是一个内存数据库,数据都在内存里面,速度不慢.

然后由于是XML序列化的,其实ORM也不需要了.每个数据库文件保存一种格式的数据.

废话不说,上代码

先是数据模型:

/*
 * Created by SharpDevelop.
 * User: scs
 * Date: 2014/12/30
 * Time: 14:07
 * 
 * To change this template use Tools | Options | Coding | Edit Standard Headers.
 */
using System;
using DevKit.Common;
namespace DevKit.HtmlUtility
{
    /// <summary>
    /// Description of Class1.
    /// </summary>
    [Serializable]
    public class CodeSnap
    {
        /// <summary>
        /// 标题
        /// </summary>
        public string Title = string.Empty;
        /// <summary>
        /// 描述
        /// </summary>
        public string Descrpition = string.Empty;
        /// <summary>
        /// 类别
        /// </summary>
        public string Catalog = string.Empty;
        /// <summary>
        /// Tag
        /// </summary>
        public string Tag = string.Empty;
        /// <summary>
        /// 代码
        /// </summary>
        public string Code = string.Empty;
        /// <summary>
        /// 检索
        /// </summary>
        /// <param name="strKeyword">检索关键字</param>
        /// <returns></returns>
        Boolean Search(string strKeyword)
        {
            return false;    
        }

    }
}

数据模型是领域的数据,但是删除标志,更新时间这些数据库使用的字段也是需要的.由于需要序列化,必须打上[Serializable]

    /// <summary>
    /// 数据库记录
    /// </summary>
    [Serializable]
    public class Model<T>
    {
        /// <summary>
        /// 删除标志
        /// </summary>
        public Boolean IsDel;
        /// <summary>
        /// 统一编号
        /// </summary>
        public string DBId;
        /// <summary>
        /// 最后更新时间
        /// </summary>
        public DateTime LastUpdate;
        /// <summary>
        /// 数据
        /// </summary>
        public T DataRec;
    }

最后是数据库引擎的代码,这里用到了深拷贝

    /// <summary>
    /// 数据库引擎
    /// </summary>
    public class XmlDataBase<T>
    {
        /// <summary>
        /// 数据库状态
        /// </summary>
        public string Status = "Close";
        /// <summary>
        /// 数据表
        /// </summary>
        List<Model<T>> list = new List<Model<T>>();
        /// <summary>
        /// 数据库文件
        /// </summary>
        string DBfilename = string.Empty;
        /// <summary>
        /// 数据库记录数[Without IsDel]
        /// </summary>
        /// <returns></returns>
        public int getCount()
        {
            return list.Count(x => {
                return !x.IsDel;
            });
        }
        /// <summary>
        /// 数据库记录数[With IsDel]
        /// </summary>
        /// <returns></returns>
        public int getCountWithDel()
        {
            return list.Count;
        }
        /// <summary>
        /// 新建并且打开数据库
        /// </summary>
        /// <param name="xmlfilename"></param>
        public XmlDataBase(string xmlfilename)
        {
            DBfilename = xmlfilename;
            if (System.IO.File.Exists(xmlfilename)) {
                list = Utility.LoadObjFromXml<List<Model<T>>>(DBfilename);
            }
        }
        /// <summary>
        /// 压缩数据库
        /// </summary>
        public void Compress()
        {
            var Compresslist = new List<Model<T>>();
            Func<Model<T>,Boolean> inner = (x) => {
                return (!x.IsDel);
            };
            Compresslist = list.FindAll(new Predicate<Model<T>>(inner));            
            list = Compresslist;
            Commit();
        }
        /// <summary>
        /// 添加
        /// </summary>
        /// <param name="rec"></param>
        public void AppendRec(T rec)
        {
            var dbrec = new Model<T>();
            dbrec.DBId = Guid.NewGuid().ToString();
            dbrec.DataRec = Common.Utility.DeepCopy(rec);
            dbrec.LastUpdate = DateTime.Now;
            list.Add(dbrec);
        }
        /// <summary>
        /// 删除
        /// </summary>
        /// <param name="rec"></param>
        public void DelRec(Model<T> rec)
        {
            rec.IsDel = true;
            UpdateDB(Utility.DeepCopy(rec));
        }
        /// <summary>
        /// 更新
        /// </summary>
        /// <param name="rec"></param>
        public void UpdataRec(Model<T> rec)
        {
            UpdateDB(Utility.DeepCopy(rec));
        }
        /// <summary>
        /// 数据的修改
        /// </summary>
        /// <param name="rec">传递过来对象的深拷贝</param>
        void UpdateDB(Model<T> rec)
        {
            for (int i = 0; i < list.Count; i++) {
                if (rec.DBId == list[i].DBId) {
                    rec.LastUpdate = DateTime.Now;
                    //不允许内部数据使用外部数据的指针引用
                    //这里使用深拷贝                    
                    list[i] = rec;
                    break;
                }
            }
        }
        /// <summary>
        /// 提交更新
        /// </summary>
        public void Commit()
        {
            Utility.SaveObjAsXml(DBfilename, list);
        }
        /// <summary>
        /// 检索
        /// </summary>
        /// <param name="SearchMethod"></param>
        /// <returns>数据对象的深拷贝</returns>
        public List<Model<T>> Search(Func<T,Boolean> SearchMethod)
        {
            Func<Model<T>,Boolean> inner = (x) => {
                return (SearchMethod(x.DataRec) && !x.IsDel);
            };
            List<Model<T>> t = new List<Model<T>>();
            foreach (Model<T> element in list.FindAll(new Predicate<Model<T>>(inner))) {
                //这里也是将数据的副本给与外部
                t.Add(Utility.DeepCopy(element));
            }    
            return t;
        }
    }

数据库内部有一个列表,列表里面存放着数据记录,每个数据记录包括[业务数据]和[数据库信息]

数据的读取,给外部的是一个数据的深拷贝,这样的话,保证了外部对于数据的修改不会影响内部数据.

在传统的数据库中,一般都是通过TCP协议交换数据的,所以,数据其实也是一个深拷贝.

读取如此,保存数据也是将列表替换为一个外部对象的深拷贝.

每次保存数据的时候,其实是将所有的数据都写入数据XML文件中,当然,数据量少的前提下,这样做是可以的.

下面是一个使用的例子:数据库的New语句

            Common.XmlDataBase<CodeSnap> db= new Common.XmlDataBase<CodeSnap>(@"C:\中和软件\CodeSnap.xml");;

        void BtnAppendClick(object sender, EventArgs e)
        {
            Stopwatch x = new Stopwatch();
            x.Start();
            for (int i = 0; i < 9999; i++) {
                var r = new CodeSnap();
                r.Title = "Title" + i.ToString();
                r.Descrpition = "Descrpition";
                r.Tag = "Tag";
                r.Code = "Code";
                db.AppendRec(r);
            }
            db.Commit();
            var t = db.Search((m) => {
                return true;
            });
            for (int i = 0; i < t.Count; i++) {
                if (i % 2 == 1) {
                    t[i].DataRec.Title = "New Title";
                    db.UpdataRec(t[i]);
                }
            }
            db.Commit();
            x.Stop();
            MessageBox.Show(x.Elapsed.ToString());
        }

这个只是一个XML数据的雏形,原代码基本上都在这里了.

可以改进的地方大概如下:NameSpace这些XML特有的属性的去除.

<ArrayOfModelOfCodeSnap xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">

现在Key是用GUID的,这个东西也蛮大的,如果不考虑压缩数据库的问题,可以使用数字连番.

(如果使用数字连番的话,由于压缩数据库会改变数据记录数,可能出现主健重复的问题)

其他压缩,例如时间,现在使用标准的DateTime.Now,所以时间也很冗长.以后可以将时间格式化后,保存为文字列.

    <IsDel>false</IsDel>
    <DBId>ef65bff8-4951-464d-bd8f-432f1148b9f8</DBId>
    <LastUpdate>2014-12-31T11:02:43.5750566+08:00</LastUpdate>

当然,XML也可以换成JSON的,这样的话,数据可以更小,但是JSON操作还不是NET内置的功能,所以暂时不使用.

里面用到的XML操作和深拷贝代码如下

        }
        /// <summary>
        /// 保存对象
        /// </summary>
        public static void SaveObjAsXml<T>(string filename, T Obj)
        {
            var xml = new XmlSerializer(typeof(T));
            var writer = new StreamWriter(filename);
            xml.Serialize(writer, Obj);
            writer.Close();
        }
        /// <summary>
        /// 读取对象
        /// </summary>
        /// <typeparam name="T"></typeparam>
        /// <param name="filename"></param>
        /// <returns></returns>
        public static T LoadObjFromXml<T>(string filename)
        {
            var xml = new XmlSerializer(typeof(T));
            var reader = new StreamReader(filename);
            T obj = (T)xml.Deserialize(reader);
            reader.Close();
            return obj;
        }
        /// <summary>
        /// 深拷贝
        /// </summary>
        /// <param name="obj"></param>
        /// <returns></returns>
        public static T DeepCopy<T>(T obj){
            BinaryFormatter bFormatter = new BinaryFormatter();
            MemoryStream stream = new MemoryStream();
            bFormatter.Serialize(stream, obj);
            stream.Seek(0, SeekOrigin.Begin);
            return (T)bFormatter.Deserialize(stream);
        }

出处：https://www.cnblogs.com/TextEditor/p/4195361.html

完整代码参考：

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;

using System.Xml.Serialization;
using System.IO;
using System.Runtime.Serialization.Formatters.Binary;
using System.Xml;

namespace ConsoleApplication1.xml
{
    /// <summary>
    /// 数据库记录
    /// </summary>
    [Serializable]
    public class Model<T>
    {
        /// <summary>
        /// 删除标志
        /// </summary>
        public Boolean IsDel;
        /// <summary>
        /// 统一编号
        /// </summary>
        public string DBId;
        /// <summary>
        /// 最后更新时间
        /// </summary>
        public DateTime LastUpdate;
        /// <summary>
        /// 数据
        /// </summary>
        public T DataRec;
    }
    /// <summary>
    /// 数据库引擎
    /// </summary>
    public class XmlDataBase<T>
    {
        /// <summary>
        /// 数据表
        /// </summary>
        List<Model<T>> list = new List<Model<T>>();

        /// <summary>
        /// 数据库文件
        /// </summary>
        string DBfilename = string.Empty;

        /// <summary>
        /// 数据库状态
        /// </summary>
        public string Status = "Close";

        /// <summary>
        /// 数据库记录数[Without IsDel]
        /// </summary>
        /// <returns></returns>
        public int getCountWithoutDel()
        {
            Refresh();
            return list.Count(x => !x.IsDel);

        }

        /// <summary>
        /// 数据库记录数[With All]
        /// </summary>
        /// <returns></returns>
        public int getCountWithAll()
        {
            Refresh();
            return list.Count;
        }

        /// <summary>
        /// 新建并且打开数据库
        /// </summary>
        /// <param name="xmlfilename"></param>
        public XmlDataBase(string xmlfilename)
        {
            DBfilename = xmlfilename;
            if (System.IO.File.Exists(DBfilename))
            {
                list = LoadObjFromXml<List<Model<T>>>(DBfilename);
            }
        }

        /// <summary>
        /// 刷新数据
        /// </summary>
        public void Refresh()
        {
            //非静态，所以，可能在其他地方发生了数据更新
            if (System.IO.File.Exists(DBfilename))
            {
                list = LoadObjFromXml<List<Model<T>>>(DBfilename);
            }
        }

        /// <summary>
        /// 压缩数据库,清理已经删除的记录
        /// </summary>
        public void Compress()
        {
            var Compresslist = new List<Model<T>>();
            Func<Model<T>, Boolean> inner = (x) => (!x.IsDel);
            Compresslist = list.FindAll(new Predicate<Model<T>>(inner));
            list = Compresslist;
            Commit();
        }
        
        /// <summary>
        /// 添加
        /// </summary>
        /// <param name="rec"></param>
        public void AppendRec(T rec)
        {
            var dbrec = new Model<T>();
            dbrec.DBId = Guid.NewGuid().ToString();
            dbrec.DataRec = DeepCopy(rec);
            dbrec.LastUpdate = DateTime.Now;
            list.Add(dbrec);
        }

        /// <summary>
        /// 删除
        /// </summary>
        /// <param name="rec"></param>
        public void DelRecord(Model<T> rec)
        {
            rec.IsDel = true;
            UpdateDB(DeepCopy(rec));
        }

        /// <summary>
        /// 删除制定编号数据
        /// </summary>
        /// <param name="DBId"></param>
        public void DelRecordByDBID(string DBId)
        {
            for (int i = 0; i < list.Count; i++)
            {
                if (DBId == list[i].DBId)
                {
                    list[i].IsDel = true;
                    list[i].LastUpdate = DateTime.Now;
                    break;
                }
            }
        }

        /// <summary>
        /// 更新
        /// </summary>
        /// <param name="rec"></param>
        public void UpdataRec(Model<T> rec)
        {
            UpdateDB(DeepCopy(rec));
        }

        /// <summary>
        /// 数据的修改
        /// </summary>
        /// <param name="rec">传递过来对象的深拷贝</param>
        private void UpdateDB(Model<T> rec)
        {
            for (int i = 0; i < list.Count; i++)
            {
                if (rec.DBId == list[i].DBId)
                {
                    rec.LastUpdate = DateTime.Now;
                    //不允许内部数据使用外部数据的指针引用,这里使用深拷贝                    
                    list[i] = rec;
                    break;
                }
            }
        }

        /// <summary>
        /// 提交更新
        /// </summary>
        public void Commit()
        {
            SaveObjAsXml(DBfilename, list);
        }

        /// <summary>
        /// 检索
        /// </summary>
        /// <param name="SearchMethod"></param>
        /// <returns>数据对象的深拷贝</returns>
        public List<Model<T>> SearchAsDBRecords(Func<T, Boolean> SearchMethod)
        {
            Refresh();
            Func<Model<T>, Boolean> inner = (x) => (SearchMethod(x.DataRec) && !x.IsDel);
            var t = new List<Model<T>>();
            foreach (Model<T> element in list.FindAll(new Predicate<Model<T>>(inner)))
            {
                //这里也是将数据的副本给与外部
                t.Add(DeepCopy(element));
            }
            return t;
        }

        /// <summary>
        /// 检索（根据数据号）
        /// </summary>
        /// <param name="DBID">数据号</param>
        /// <returns></returns>
        public Model<T> SearchAsDBRecordByDBID(string DBID)
        {
            Refresh();
            Model<T> result = list.Find((x) => x.DBId == DBID && !x.IsDel);
            if (result != null)
                return DeepCopy(result);
            return null;
        }

        /// <summary>
        /// 检索,（如果只是获取T对象列表）
        /// </summary>
        /// <param name="SearchMethod"></param>
        /// <returns>数据对象的深拷贝</returns>
        public List<T> SearchAsObjRecords(Func<T, Boolean> SearchMethod)
        {
            Refresh();
            Func<Model<T>, Boolean> inner = (x) => (SearchMethod(x.DataRec) && !x.IsDel);
            var t = new List<T>();
            foreach (Model<T> element in list.FindAll(new Predicate<Model<T>>(inner)))
            {
                //这里也是将数据的副本给与外部
                t.Add(DeepCopy(element.DataRec));
            }
            return t;
        }

        /// <summary>
        /// 检索（根据数据号）（如果只是获取T对象）
        /// 调用前请使用IsRecordExist函数确认数据是否存在
        /// </summary>
        /// <param name="DBID">数据号</param>
        /// <returns></returns>
        public T SearchAsObjRecordByDBID(string DBID)
        {
            Refresh();
            T t = default(T);
            Model<T> result = list.Find((x) => x.DBId == DBID);
            t =DeepCopy(result.DataRec);
            return t;
        }

        /// <summary>
        /// 是否存在数据
        /// </summary>
        /// <param name="SearchMethod"></param>
        /// <returns></returns>
        public bool IsRecordExists(Func<T, Boolean> SearchMethod)
        {
            Func<Model<T>, Boolean> inner = (x) => (SearchMethod(x.DataRec) && !x.IsDel);
            return list.FindAll(new Predicate<Model<T>>(inner)).Count != 0;
        }

        /// <summary>
        /// 是否存在指定番号数据
        /// </summary>
        /// <param name="DBID"></param>
        /// <returns></returns>
        public bool IsRecordExistsByDBID(string DBID)
        {
            Func<Model<T>, Boolean> inner = (x) => (x.DBId == DBID && !x.IsDel);
            return list.FindAll(new Predicate<Model<T>>(inner)).Count != 0;
        }



        /// <summary>
        /// 保存对象
        /// </summary>
        /// <param name="filename"></param>
        /// <param name="Obj"></param>
        private static void SaveObjAsXml<T>(string filename, T Obj)
        {
            var settings = new XmlWriterSettings();
            settings.Indent = true;
            //NewLineChars对于String属性的东西无效
            //这是对于XML中换行有效，
            //String的换行会变成Console的NewLine /n
            settings.NewLineChars = System.Environment.NewLine;
            var xml = new XmlSerializer(typeof(T));
            var writer = XmlWriter.Create(filename, settings);
            var ns = new XmlSerializerNamespaces();
            ns.Add("", "");
            xml.Serialize(writer, Obj, ns);
            writer.Close();
        }

        /// <summary>
        /// 读取对象
        /// </summary>
        /// <typeparam name="T"></typeparam>
        /// <param name="filename"></param>
        /// <returns></returns>
        private static T LoadObjFromXml<T>(string filename)
        {
            var setting = new XmlReaderSettings();
            var xml = new XmlSerializer(typeof(T));
            var reader = XmlReader.Create(filename, setting);
            T obj = (T)xml.Deserialize(reader);
            reader.Close();
            return obj;
        }

        /// <summary>
        /// 深拷贝
        /// </summary>
        /// <param name="obj"></param>
        /// <returns></returns>
        private static T DeepCopy<T>(T obj)
        {
            var bFormatter = new BinaryFormatter();
            var stream = new MemoryStream();
            bFormatter.Serialize(stream, obj);
            stream.Seek(0, SeekOrigin.Begin);
            return (T)bFormatter.Deserialize(stream);
        }
    }
}

View Code

对象定义格式如下：

// 模式一

using System;
namespace ConsoleApplication1.xml
{
    /// <summary>
    /// Description of Class1.
    /// </summary>
    [Serializable]
    public class CodeSnap
    {
        /// <summary>
        /// 标题
        /// </summary>
        public string Title = string.Empty;
        /// <summary>
        /// 描述
        /// </summary>
        public string Descrpition = string.Empty;
        /// <summary>
        /// 类别
        /// </summary>
        public string Catalog = string.Empty;
        /// <summary>
        /// Tag
        /// </summary>
        public string Tag = string.Empty;
        /// <summary>
        /// 代码
        /// </summary>
        public string Code = string.Empty;
        /// <summary>
        /// 检索
        /// </summary>
        /// <param name="strKeyword">检索关键字</param>
        /// <returns></returns>
        public Boolean Search(string strKeyword)
        {
            return false;    
        }

    }
}

// 模式二
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;

namespace ConsoleApplication1.xml
{
    [Serializable]
    public class CodeBook
    {
        private string bName;

        public string BookName
        {
            get { return bName; }
            set { bName = value; }
        }

        private string bAnthor;

        public string Anthor
        {
            get { return bAnthor; }
            set { bAnthor = value; }
        }
        
        private int _price;

        public int Price
        {
            get { return _price; }
            set { _price = value; }
        }

    }
}

调用方式参考：

            xml.XmlDataBase<xml.CodeSnap> db = new xml.XmlDataBase<xml.CodeSnap>(@"C:\temp\CodeSnap.xml");
            for (int i = 0; i < 4; i++)
            {
                var r = new xml.CodeSnap();
                r.Title = "Title" + i.ToString();
                r.Descrpition = "Descrpition";
                r.Tag = "Tag";
                r.Code = "Code";
                db.AppendRec(r);
            }
            //db.Commit();

            xml.XmlDataBase<xml.CodeBook> db1 = new xml.XmlDataBase<xml.CodeBook>(@"C:\temp\book.xml");
            for (int i = 0; i < 5; i++)
            {
                var r = new xml.CodeBook();
                r.Anthor = "anth " + i;
                r.BookName = " book " + i;
                r.Price = i;
                db1.AppendRec(r);
            }
            //db1.Commit();

            var t = db1.SearchAsDBRecords((m) =>
            {
                return true;
            });

            var bbb = t.FindAll((ss) => {return ss.DataRec.Price > 10;});


            for (int i = 0; i < t.Count; i++)
            {
                if (i % 2 == 1)
                {
                    //xml.CodeBook b = t[i];
                    //t[i].Price += 10;
                    t[i].DataRec.Price += 10;
                    //db1.UpdataRec(t[i]);
                }
                if (i==3)
                {
                    //db1.DelRecordByDBID(t[i].DBId);
                    db1.DelRecord(t[i]);
                }
                Console.WriteLine(db1.Status);
            }
            db1.Commit();
            Console.WriteLine(db1.getCountWithAll());
            db1.Compress();

posted on 2017-03-31 11:51 jack_Meng 阅读(358) 评论(0) 编辑收藏举报

刷新页面返回顶部

Jack_孟

XML数据库的尝试

导航

公告