现在ORM已经是一门非常成熟的技术了,相信用的人不少,加上Linq to sql和Entity Framework的推波助澜,现在还用DataSet和DataTable的人已经越来越少了,不过,如果项目里面不用ORM工具,就不得不回归到DataSet时代吗?
也许,我们没法改变项目的决策,但是,我们可以自己制造工具。
这里先忽略掉那些麻烦的sql,调用那个存储过程之类的事情,假设我们已经通过各种手段(不管你是SqlHelper的拥护者还是enterprise library的支持者,或者是自己动手丰衣足食的DIY派),得到了一个IDataReader实例(什么,你不知道IDataReader接口?去msdn看看吧),并且把msdn上的说明:
提供一种方法来读取一个或多个通过在数据源执行命令所获得的只进结果集流
简化成:
提供一种方法来读取一个通过在数据源执行命令所获得的只进结果集流
也就是说,忽略掉一个存储过程返回多个结果集的情况下,我们可以直接把这个IDataReader实例中承载的数据转换为一个List<T>的形式。
读到这里你可能会觉得这样的功能用一个反射就可以搞定了,何必写这么一篇水文?
不过,这里用的不是常规反射。什么意思?反射还有常规和非常规之分吗?
常规的反射是直接使用GetProperty和SetValue就可以实现这样的功能了,但是常规反射的问题是反射的效率问题,这样的代码效率是手写代码的1/1000,也就是说,虽然你写的很轻松,但是对CLR来说,这个代码执行的非常累。
我这里要用的是反射中的Emit技术,动态拼装IL,获得一个高性能转换器,其执行效率至少是手写代码1/2,也就是说,使用这个代码的目的是你写的轻松,CLR执行的也轻松。
不过,在这个之前,需要先定义数据实体,这个又是每个人的习惯都不一样的东西,我个人是习惯于利用Attribute来表明哪个属性是哪一列,不过考虑到便捷性,会用一个类级的Attribute表明这个类默认为是数据列,或者表明不是默认为是数据列:
2 internal sealed class DbResultAttribute : Attribute
3 {
4
5 public DbResultAttribute()
6 : this(true) { }
7
8 public DbResultAttribute(bool defaultAsDbColumn)
9 {
10 DefaultAsDbColumn = defaultAsDbColumn;
11 }
12
13 public bool DefaultAsDbColumn { get; private set; }
14
15 }
16
而对于普通的属性,再给与一次修改列名和排除在数据列之外的机会:
2 internal sealed class DbColumnAttribute : Attribute
3 {
4
5 public DbColumnAttribute() { }
6
7 public string ColumnName { get; set; }
8
9 public bool Ignore { get; set; }
10
11 }
12
这样,我们就可以很简单的定义一个数据实体了:
2 public class MyClass
3 {
4 public int IntValue { get; set; }
5 public string StringValue { get; set; }
6 [DbColumn(ColumnName = "DecimalValue")]
7 public decimal? NullableDecimalAndDifferentName { get; set; }
8 public MyEnum EnumIsAlsoSupportted { get; set; }
9 public MyEnum? NullableEnumIsAlsoSupportted { get; set; }
10 [DbColumn(Ignore = true)]
11 public object NonDbValue { get; set; }
12 }
13
14 public enum MyEnum
15 {
16 X,
17 Y,
18 Z,
19 }
20
定义简单吧,不过,可以看出来实现部分也是异常的复杂,除了标准的int,string,decimal类型,还要支持可空类型,以及枚举类型,甚至是可空枚举类型。。。不过,如果数据库类型和定义的类型(枚举看它的基础类型)不匹配就会直接抛出异常,这点要注意一下。
千里之行始于足下,先做最简单的,搭建一个简单的环境,并且读取这个实体类的信息吧:
2 {
3
4 #region Public Static Methods
5
6 public static List<T> Select<T>(this IDataReader reader)
7 where T : class, new()
8 {
9 if (reader == null)
10 throw new ArgumentNullException("reader");
11 return EntityConverter<T>.Select(reader);
12 }
13
14 #endregion
15
16 #region Class: EntityConverter<T>
17
18 private class EntityConverter<T>
19 where T : class, new()
20 {
21
22 #region Struct: DbColumnInfo
23
24 private struct DbColumnInfo
25 {
26 public readonly string PropertyName;
27 public readonly string ColumnName;
28 public readonly Type Type;
29 public readonly MethodInfo SetMethod;
30
31 public DbColumnInfo(PropertyInfo prop, DbColumnAttribute attr)
32 {
33 PropertyName = prop.Name;
34 ColumnName = attr.ColumnName ?? prop.Name;
35 Type = prop.PropertyType;
36 SetMethod = prop.GetSetMethod(false);
37 }
38 }
39
40 #endregion
41
42 #region Fields
43 private static Converter<IDataReader, List<T>> batchDataLoader;
44 #endregion
45
46 #region Properties
47
48 private static Converter<IDataReader, List<T>> BatchDataLoader
49 {
50 get
51 {
52 if (batchDataLoader == null)
53 batchDataLoader = CreateBatchDataLoader(new List<DbColumnInfo>(GetProperties()));
54 return batchDataLoader;
55 }
56 }
57
58 #endregion
59
60 #region Init Methods
61
62 private static IEnumerable<DbColumnInfo> GetProperties()
63 {
64 DbResultAttribute dbResult = Attribute.GetCustomAttribute(typeof(T), typeof(DbResultAttribute), true) as DbResultAttribute;
65 foreach (var prop in typeof(T).GetProperties(BindingFlags.Instance | BindingFlags.Public))
66 {
67 if (prop.GetIndexParameters().Length > 0)
68 continue;
69 var setMethod = prop.GetSetMethod(false);
70 if (setMethod == null)
71 continue;
72 var attr = Attribute.GetCustomAttribute(prop, typeof(DbColumnAttribute), true) as DbColumnAttribute;
73 if (dbResult != null && dbResult.DefaultAsDbColumn)
74 if (attr != null && attr.Ignore)
75 continue;
76 else
77 attr = attr ?? new DbColumnAttribute();
78 else
79 if (attr == null || attr.Ignore)
80 continue;
81 yield return new DbColumnInfo(prop, attr);
82 }
83 }
84
85 #endregion
86
87 }
88
这里,我们先建立一个DataReaderExtensions类,这个类的功能只有一个,给IDataReader添加一个Select<T>的方法,然后把思想转嫁给一个内部类EntityConverter<T>的Select方法(这个方法在那里?还没贴出来哪,别急),为什么用这么一个内部泛型类那?建议参考老赵的这篇文章,主要是出于性能方面的考虑。
然后我们在通过GetProperties方法,获得关于这个实体类型的信息,放到一个叫DbColumnInfo的结构体里面(为什么用结构体哪?还是性能方面的考虑,减少垃圾对象,让GC过的轻松点)。这里的GetProperties方法就是一个彻头彻尾的普通反射,那么性能自然也就是手写代码的1/1000,不过别急,这个段代码对每个T而言仅仅跑1次,也就是说,某个类型Select过了,那么下一次,这个类型就不需要跑这个GetProperties方法了。
不过,有一点不要误解,即使缓存了PropertyInfo,那也仅仅是减少了发现成员的代价,PropertyInfo.SetValue的代价依然是反射的标准代价,执行效率是1/1000。
也就是说,之后的任务才是本文的重点内容,消除掉PropertyInfo.SetValue的代价,让我们的反射跑得飞起来。
不过在这个之前,先把我们需要的一些IDataReader里面的方法先反射出来,做好缓存(放在DataReaderExtensions类里面,避免为每个T反射一次):
2 private static readonly MethodInfo DataRecord_ItemGetter_Int =
3 typeof(IDataRecord).GetMethod("get_Item", new Type[] { typeof(int) });
4 private static readonly MethodInfo DataRecord_GetOrdinal =
5 typeof(IDataRecord).GetMethod("GetOrdinal");
6 private static readonly MethodInfo DataReader_Read =
7 typeof(IDataReader).GetMethod("Read");
8 private static readonly MethodInfo Convert_IsDBNull =
9 typeof(Convert).GetMethod("IsDBNull");
10 private static readonly MethodInfo DataRecord_GetDateTime =
11 typeof(IDataRecord).GetMethod("GetDateTime");
12 private static readonly MethodInfo DataRecord_GetDecimal =
13 typeof(IDataRecord).GetMethod("GetDecimal");
14 private static readonly MethodInfo DataRecord_GetDouble =
15 typeof(IDataRecord).GetMethod("GetDouble");
16 private static readonly MethodInfo DataRecord_GetInt32 =
17 typeof(IDataRecord).GetMethod("GetInt32");
18 private static readonly MethodInfo DataRecord_GetInt64 =
19 typeof(IDataRecord).GetMethod("GetInt64");
20 private static readonly MethodInfo DataRecord_GetString =
21 typeof(IDataRecord).GetMethod("GetString");
22 private static readonly MethodInfo DataRecord_IsDBNull =
23 typeof(IDataRecord).GetMethod("IsDBNull");
24 #endregion
25
接下来就是Emit了,但是鉴于其的难度比较高,就不仔细说明了,如果各位有兴趣,可以单独聊,这里就直接把这成堆的天书贴上来了(未处理short、byte、bool的情况,因为Oracle不支持这个类型,所以一直没写,同样枚举也只支持基础类型是int的或long的):
2
3 private static Converter<IDataReader, List<T>> CreateBatchDataLoader(List<DbColumnInfo> columnInfoes)
4 {
5 DynamicMethod dm = new DynamicMethod(string.Empty, typeof(List<T>),
6 new Type[] { typeof(IDataReader) }, typeof(EntityConverter<T>));
7 ILGenerator il = dm.GetILGenerator();
8 LocalBuilder list = il.DeclareLocal(typeof(List<T>));
9 LocalBuilder item = il.DeclareLocal(typeof(T));
10 Label exit = il.DefineLabel();
11 Label loop = il.DefineLabel();
12 // List<T> list = new List<T>();
13 il.Emit(OpCodes.Newobj, typeof(List<T>).GetConstructor(Type.EmptyTypes));
14 il.Emit(OpCodes.Stloc_S, list);
15 // [ int %index% = arg.GetOrdinal(%ColumnName%); ]
16 LocalBuilder[] colIndices = GetColumnIndices(il, columnInfoes);
17 // while (arg.Read()) {
18 il.MarkLabel(loop);
19 il.Emit(OpCodes.Ldarg_0);
20 il.Emit(OpCodes.Callvirt, DataReader_Read);
21 il.Emit(OpCodes.Brfalse, exit);
22 // T item = new T { %Property% = };
23 BuildItem(il, columnInfoes, item, colIndices);
24 // list.Add(item);
25 il.Emit(OpCodes.Ldloc_S, list);
26 il.Emit(OpCodes.Ldloc_S, item);
27 il.Emit(OpCodes.Callvirt, typeof(List<T>).GetMethod("Add"));
28 // }
29 il.Emit(OpCodes.Br, loop);
30 il.MarkLabel(exit);
31 // return list;
32 il.Emit(OpCodes.Ldloc_S, list);
33 il.Emit(OpCodes.Ret);
34 return (Converter<IDataReader, List<T>>)dm.CreateDelegate(typeof(Converter<IDataReader, List<T>>));
35 }
36
37 private static LocalBuilder[] GetColumnIndices(ILGenerator il, List<DbColumnInfo> columnInfoes)
38 {
39 LocalBuilder[] colIndices = new LocalBuilder[columnInfoes.Count];
40 for (int i = 0; i < colIndices.Length; i++)
41 {
42 // int %index% = arg.GetOrdinal(%ColumnName%);
43 colIndices[i] = il.DeclareLocal(typeof(int));
44 il.Emit(OpCodes.Ldarg_0);
45 il.Emit(OpCodes.Ldstr, columnInfoes[i].ColumnName);
46 il.Emit(OpCodes.Callvirt, DataRecord_GetOrdinal);
47 il.Emit(OpCodes.Stloc_S, colIndices[i]);
48 }
49 return colIndices;
50 }
51
52 private static void BuildItem(ILGenerator il, List<DbColumnInfo> columnInfoes,
53 LocalBuilder item, LocalBuilder[] colIndices)
54 {
55 // T item = new T();
56 il.Emit(OpCodes.Newobj, typeof(T).GetConstructor(Type.EmptyTypes));
57 il.Emit(OpCodes.Stloc_S, item);
58 for (int i = 0; i < colIndices.Length; i++)
59 {
60 if (IsCompatibleType(columnInfoes[i].Type, typeof(int)))
61 {
62 // item.%Property% = arg.GetInt32(%index%);
63 ReadInt32(il, item, columnInfoes, colIndices, i);
64 }
65 else if (IsCompatibleType(columnInfoes[i].Type, typeof(int?)))
66 {
67 // item.%Property% = arg.IsDBNull ? default(int?) : (int?)arg.GetInt32(%index%);
68 ReadNullableInt32(il, item, columnInfoes, colIndices, i);
69 }
70 else if (IsCompatibleType(columnInfoes[i].Type, typeof(long)))
71 {
72 // item.%Property% = arg.GetInt64(%index%);
73 ReadInt64(il, item, columnInfoes, colIndices, i);
74 }
75 else if (IsCompatibleType(columnInfoes[i].Type, typeof(long?)))
76 {
77 // item.%Property% = arg.IsDBNull ? default(long?) : (long?)arg.GetInt64(%index%);
78 ReadNullableInt64(il, item, columnInfoes, colIndices, i);
79 }
80 else if (IsCompatibleType(columnInfoes[i].Type, typeof(decimal)))
81 {
82 // item.%Property% = arg.GetDecimal(%index%);
83 ReadDecimal(il, item, columnInfoes[i].SetMethod, colIndices[i]);
84 }
85 else if (columnInfoes[i].Type == typeof(decimal?))
86 {
87 // item.%Property% = arg.IsDBNull ? default(decimal?) : (int?)arg.GetDecimal(%index%);
88 ReadNullableDecimal(il, item, columnInfoes[i].SetMethod, colIndices[i]);
89 }
90 else if (columnInfoes[i].Type == typeof(DateTime))
91 {
92 // item.%Property% = arg.GetDateTime(%index%);
93 ReadDateTime(il, item, columnInfoes[i].SetMethod, colIndices[i]);
94 }
95 else if (columnInfoes[i].Type == typeof(DateTime?))
96 {
97 // item.%Property% = arg.IsDBNull ? default(DateTime?) : (int?)arg.GetDateTime(%index%);
98 ReadNullableDateTime(il, item, columnInfoes[i].SetMethod, colIndices[i]);
99 }
100 else
101 {
102 // item.%Property% = (%PropertyType%)arg[%index%];
103 ReadObject(il, item, columnInfoes, colIndices, i);
104 }
105 }
106 }
107
108 private static bool IsCompatibleType(Type t1, Type t2)
109 {
110 if (t1 == t2)
111 return true;
112 if (t1.IsEnum && Enum.GetUnderlyingType(t1) == t2)
113 return true;
114 var u1 = Nullable.GetUnderlyingType(t1);
115 var u2 = Nullable.GetUnderlyingType(t2);
116 if (u1 != null && u2 != null)
117 return IsCompatibleType(u1, u2);
118 return false;
119 }
120
121 private static void ReadInt32(ILGenerator il, LocalBuilder item,
122 List<DbColumnInfo> columnInfoes, LocalBuilder[] colIndices, int i)
123 {
124 il.Emit(OpCodes.Ldloc_S, item);
125 il.Emit(OpCodes.Ldarg_0);
126 il.Emit(OpCodes.Ldloc_S, colIndices[i]);
127 il.Emit(OpCodes.Callvirt, DataRecord_GetInt32);
128 il.Emit(OpCodes.Callvirt, columnInfoes[i].SetMethod);
129 }
130
131 private static void ReadNullableInt32(ILGenerator il, LocalBuilder item,
132 List<DbColumnInfo> columnInfoes, LocalBuilder[] colIndices, int i)
133 {
134 var local = il.DeclareLocal(columnInfoes[i].Type);
135 Label intNull = il.DefineLabel();
136 Label intCommon = il.DefineLabel();
137 il.Emit(OpCodes.Ldloca, local);
138 il.Emit(OpCodes.Ldarg_0);
139 il.Emit(OpCodes.Ldloc_S, colIndices[i]);
140 il.Emit(OpCodes.Callvirt, DataRecord_IsDBNull);
141 il.Emit(OpCodes.Brtrue_S, intNull);
142 il.Emit(OpCodes.Ldarg_0);
143 il.Emit(OpCodes.Ldloc_S, colIndices[i]);
144 il.Emit(OpCodes.Callvirt, DataRecord_GetInt32);
145 il.Emit(OpCodes.Call, columnInfoes[i].Type.GetConstructor(
146 new Type[] { Nullable.GetUnderlyingType(columnInfoes[i].Type) }));
147 il.Emit(OpCodes.Br_S, intCommon);
148 il.MarkLabel(intNull);
149 il.Emit(OpCodes.Initobj, columnInfoes[i].Type);
150 il.MarkLabel(intCommon);
151 il.Emit(OpCodes.Ldloc_S, item);
152 il.Emit(OpCodes.Ldloc, local);
153 il.Emit(OpCodes.Callvirt, columnInfoes[i].SetMethod);
154 }
155
156 private static void ReadInt64(ILGenerator il, LocalBuilder item,
157 List<DbColumnInfo> columnInfoes, LocalBuilder[] colIndices, int i)
158 {
159 il.Emit(OpCodes.Ldloc_S, item);
160 il.Emit(OpCodes.Ldarg_0);
161 il.Emit(OpCodes.Ldloc_S, colIndices[i]);
162 il.Emit(OpCodes.Callvirt, DataRecord_GetInt64);
163 il.Emit(OpCodes.Callvirt, columnInfoes[i].SetMethod);
164 }
165
166 private static void ReadNullableInt64(ILGenerator il, LocalBuilder item,
167 List<DbColumnInfo> columnInfoes, LocalBuilder[] colIndices, int i)
168 {
169 var local = il.DeclareLocal(columnInfoes[i].Type);
170 Label intNull = il.DefineLabel();
171 Label intCommon = il.DefineLabel();
172 il.Emit(OpCodes.Ldloca, local);
173 il.Emit(OpCodes.Ldarg_0);
174 il.Emit(OpCodes.Ldloc_S, colIndices[i]);
175 il.Emit(OpCodes.Callvirt, DataRecord_IsDBNull);
176 il.Emit(OpCodes.Brtrue_S, intNull);
177 il.Emit(OpCodes.Ldarg_0);
178 il.Emit(OpCodes.Ldloc_S, colIndices[i]);
179 il.Emit(OpCodes.Callvirt, DataRecord_GetInt64);
180 il.Emit(OpCodes.Call, columnInfoes[i].Type.GetConstructor(
181 new Type[] { Nullable.GetUnderlyingType(columnInfoes[i].Type) }));
182 il.Emit(OpCodes.Br_S, intCommon);
183 il.MarkLabel(intNull);
184 il.Emit(OpCodes.Initobj, columnInfoes[i].Type);
185 il.MarkLabel(intCommon);
186 il.Emit(OpCodes.Ldloc_S, item);
187 il.Emit(OpCodes.Ldloc, local);
188 il.Emit(OpCodes.Callvirt, columnInfoes[i].SetMethod);
189 }
190
191 private static void ReadDecimal(ILGenerator il, LocalBuilder item,
192 MethodInfo setMethod, LocalBuilder colIndex)
193 {
194 il.Emit(OpCodes.Ldloc_S, item);
195 il.Emit(OpCodes.Ldarg_0);
196 il.Emit(OpCodes.Ldloc_S, colIndex);
197 il.Emit(OpCodes.Callvirt, DataRecord_GetDecimal);
198 il.Emit(OpCodes.Callvirt, setMethod);
199 }
200
201 private static void ReadNullableDecimal(ILGenerator il, LocalBuilder item,
202 MethodInfo setMethod, LocalBuilder colIndex)
203 {
204 var local = il.DeclareLocal(typeof(decimal?));
205 Label decimalNull = il.DefineLabel();
206 Label decimalCommon = il.DefineLabel();
207 il.Emit(OpCodes.Ldloca, local);
208 il.Emit(OpCodes.Ldarg_0);
209 il.Emit(OpCodes.Ldloc_S, colIndex);
210 il.Emit(OpCodes.Callvirt, DataRecord_IsDBNull);
211 il.Emit(OpCodes.Brtrue_S, decimalNull);
212 il.Emit(OpCodes.Ldarg_0);
213 il.Emit(OpCodes.Ldloc_S, colIndex);
214 il.Emit(OpCodes.Callvirt, DataRecord_GetDecimal);
215 il.Emit(OpCodes.Call, typeof(decimal?).GetConstructor(new Type[] { typeof(decimal) }));
216 il.Emit(OpCodes.Br_S, decimalCommon);
217 il.MarkLabel(decimalNull);
218 il.Emit(OpCodes.Initobj, typeof(decimal?));
219 il.MarkLabel(decimalCommon);
220 il.Emit(OpCodes.Ldloc_S, item);
221 il.Emit(OpCodes.Ldloc, local);
222 il.Emit(OpCodes.Callvirt, setMethod);
223 }
224
225 private static void ReadDateTime(ILGenerator il, LocalBuilder item,
226 MethodInfo setMethod, LocalBuilder colIndex)
227 {
228 il.Emit(OpCodes.Ldloc_S, item);
229 il.Emit(OpCodes.Ldarg_0);
230 il.Emit(OpCodes.Ldloc_S, colIndex);
231 il.Emit(OpCodes.Callvirt, DataRecord_GetDateTime);
232 il.Emit(OpCodes.Callvirt, setMethod);
233 }
234
235 private static void ReadNullableDateTime(ILGenerator il, LocalBuilder item,
236 MethodInfo setMethod, LocalBuilder colIndex)
237 {
238 var local = il.DeclareLocal(typeof(DateTime?));
239 Label dtNull = il.DefineLabel();
240 Label dtCommon = il.DefineLabel();
241 il.Emit(OpCodes.Ldloca, local);
242 il.Emit(OpCodes.Ldarg_0);
243 il.Emit(OpCodes.Ldloc_S, colIndex);
244 il.Emit(OpCodes.Callvirt, DataRecord_IsDBNull);
245 il.Emit(OpCodes.Brtrue_S, dtNull);
246 il.Emit(OpCodes.Ldarg_0);
247 il.Emit(OpCodes.Ldloc_S, colIndex);
248 il.Emit(OpCodes.Callvirt, DataRecord_GetDateTime);
249 il.Emit(OpCodes.Call, typeof(DateTime?).GetConstructor(new Type[] { typeof(DateTime) }));
250 il.Emit(OpCodes.Br_S, dtCommon);
251 il.MarkLabel(dtNull);
252 il.Emit(OpCodes.Initobj, typeof(DateTime?));
253 il.MarkLabel(dtCommon);
254 il.Emit(OpCodes.Ldloc_S, item);
255 il.Emit(OpCodes.Ldloc, local);
256 il.Emit(OpCodes.Callvirt, setMethod);
257 }
258
259 private static void ReadObject(ILGenerator il, LocalBuilder item,
260 List<DbColumnInfo> columnInfoes, LocalBuilder[] colIndices, int i)
261 {
262 Label common = il.DefineLabel();
263 il.Emit(OpCodes.Ldloc_S, item);
264 il.Emit(OpCodes.Ldarg_0);
265 il.Emit(OpCodes.Ldloc_S, colIndices[i]);
266 il.Emit(OpCodes.Callvirt, DataRecord_ItemGetter_Int);
267 il.Emit(OpCodes.Dup);
268 il.Emit(OpCodes.Call, Convert_IsDBNull);
269 il.Emit(OpCodes.Brfalse_S, common);
270 il.Emit(OpCodes.Pop);
271 il.Emit(OpCodes.Ldnull);
272 il.MarkLabel(common);
273 il.Emit(OpCodes.Unbox_Any, columnInfoes[i].Type);
274 il.Emit(OpCodes.Callvirt, columnInfoes[i].SetMethod);
275 }
276
277 #endregion
278
279 #region Internal Methods
280
281 internal static List<T> Select(IDataReader reader)
282 {
283 return BatchDataLoader(reader);
284 }
285
286 #endregion
287
好,到这里,这个扩展方法已经完成了,来试用一下吧:
2 {
3 var reader = GetReader();
4 return reader.Select<MyClass>();
5 }
6
7 public IDataReader GetReader()
8 {
9 // todo : GetReader
10 throw new NotImplementedException();
11 }
12
是不是很简单,大家不妨做个效率测试,看看是这段基于反射的代码效率高还是大家手写的效率高,以及填充到DataSet的效率,当然要注意数据的访问时间本身是不缺定的。(里面用了不少优化的访问方式,如果大家在处理IDataReader的手法不够娴熟的话,还真难说效率谁的高,呵呵)
(最后的示例代码里面忘记用using了。。。不改了,大家自己加上吧)