Richard__Lee

导航

BulkSqlCopy 批量导入数据(Ef支持)

Ado.net对批量数据的支持相信大家都已经非常熟悉。再此就不在多说,就当是给自己备个份,没办法,这个方法太好用了。

  public static void BulkCreate( string tableName,bool IsGuid=true)
       {
           string sql = string.Format("select *  from {0}",tableName);
           DataTable dt = new DataTable();
           using (SqlDataAdapter da = new SqlDataAdapter(sql, constr))
           {
               da.Fill(dt);
           }
           if (dt != null && dt.Rows.Count > 0)
           {
               using (var tran = db.Database.BeginTransaction())
               {
                   try
                   {
                       using (SqlBulkCopy sqlBulkCopy = new SqlBulkCopy(db.Database.Connection.ConnectionString))
                       { //每1W条一个事物
                           sqlBulkCopy.BatchSize = 10000;
                           sqlBulkCopy.BulkCopyTimeout = 12000;
                           sqlBulkCopy.DestinationTableName = string.Format("dbo.{0}", tableName);
                           sqlBulkCopy.BatchSize = dt.Rows.Count;
                           if (!IsGuid)
                           {//如果主键不是Guid,是Int自增类型,这个时候就需要注意了,直接忽略第一列,所以for循环从1开始。
                               for (int i = 1; i < dt.Columns.Count; i++)
                               {
                                   sqlBulkCopy.ColumnMappings.Add(i, i);
                               }
                           }
                           sqlBulkCopy.WriteToServer(dt);
                           tran.Commit();
                       }
                   }
                   catch
                   {
                       tran.Rollback();
                   }

               }
           }
       }
View Code

上面简单实现了下,如果是EntityFrameWork 怎么办,不想让代码层面出现不和谐的红色sql语句。很简单,只需要三个参数,一个是表名(可以根据映射的实体特性[Attribute]获得),一个是列名,最后是列值,后面的两个自然是根据反射获得了。下面简单实现下,只当做参考,请原谅:

        public void BulkCreate<T>(List<T> list) where T : new()
        {
            DataTable dt = new DataTable();
            Type type = typeof(T);
            string tableName = GetTableName(type);
            PropertyInfo[] propes = type.GetProperties();
            foreach (var prop in propes)
            {
                dt.Columns.Add(prop.Name);
            }
            //再此是假设数据库列与对象列一致 没有判断列属性,如果有列属性的话 ,就需要再多一次判断
            foreach (var entity in list)
            {
                DataRow row = dt.NewRow();
                foreach (DataColumn col in dt.Columns)
                {
                    foreach (var prop in propes)
                    {
                        if (!col.ColumnName.Equals("id", StringComparison.InvariantCultureIgnoreCase))
                        {
                            if (prop.Name.Equals(col.ColumnName, StringComparison.InvariantCultureIgnoreCase))
                                row[col.ColumnName] = prop.GetValue(entity);
                        }
                    }

                }
                dt.Rows.Add(row);
            }
            if (dt != null && dt.Rows.Count > 0)
            {
                using (var tran = db.BeginTransaction())
                {
                    try
                    {
                        BulkCopy(dt, tableName);
                    }
                    catch (Exception e)
                    {
                        tran.Rollback();
                    }
                    tran.Commit();
                }

            }
        }

        private bool BulkCopy(DataTable dt, string name)
        {
            //string constr = System.Configuration.ConfigurationManager.ConnectionStrings["BingStampAzureContext"].ToString();
            string constr = db.Database.Connection.ConnectionString;
            SqlBulkCopy sqlBulkCopy = null;
            try
            {
                using (SqlConnection conn = new SqlConnection(constr))
                {
                    sqlBulkCopy = new SqlBulkCopy(constr);
                    sqlBulkCopy.BulkCopyTimeout = 12000;
                    sqlBulkCopy.DestinationTableName = string.Format("dbo.{0}", name);
                    sqlBulkCopy.BatchSize = dt.Rows.Count;
                    sqlBulkCopy.WriteToServer(dt);
                }

                return true;
            }
            catch (Exception ex)
            {
                return false;
            }
            finally
            {
                if (sqlBulkCopy != null) sqlBulkCopy.Close();
            }
        }

        public string GetTableName(Type type)
        {
            object[] attributes = type.GetCustomAttributes(false);
            if (attributes.Length > 0)
            {
                var attribute = (System.ComponentModel.DataAnnotations.Schema.TableAttribute)attributes[0];
                return attribute.Name;
            }
            else
                return string.Empty;
        }
View Code

下面给个对象调用下 :

       [Table("Student")]
        public class Student
        {
            public int Id { get; set; }
            public string Name { get; set; }
            public int Age { get; set; }
            public string Address { get; set; }
            public string Remark { get; set; }
        }
View Code

具体测试略。。

 

 

posted on 2014-04-22 11:46  Richard__Lee  阅读(323)  评论(0编辑  收藏  举报