EF大数据批量添加性能问题 - 罗朝福
罗朝福发表于 2015-08-21 09:46:00
前几天做一个批量发消息的功能，因为要向消息表中批量写入数据，用的EF框架的插入方法；不用不知道，一用吓一跳；就10000条数据就耗时好几分钟，对应追求用户体验的我来说这是极不能容忍的，后来改为拼接SQL，性能提高了好几倍；现在来分享一下经验：
原始的方法类似这种：
1 public ActionResult Add(ItemDetails entity)
2 {
3 var sw = new Stopwatch();
4 sw.Start();
5 using (db)
6 {
7 for (var i = 0; i < 10000; i++)
8 {
9 db.ItemDetails.Add(entity);
10 db.SaveChanges();
11 }
12 }
13 sw.Stop();
14 var date = sw.Elapsed;
15 return Json(string.Format("总耗时：{0}", date));
16 }
2 {
3 var sw = new Stopwatch();
4 sw.Start();
5 using (db)
6 {
7 for (var i = 0; i < 10000; i++)
8 {
9 db.ItemDetails.Add(entity);
10 db.SaveChanges();
11 }
12 }
13 sw.Stop();
14 var date = sw.Elapsed;
15 return Json(string.Format("总耗时：{0}", date));
16 }
来看看添加10000条数据耗时：
就10000条数据就耗时这么久，要是上百万的数据量，那简直就不能想象，再来看看优化后的：
生成SQL的方法：
1 public class ItemDetailBatch
2 {
3 public static string BatchAdd(ItemDetails entity)
4 {
5 SqlParameter [] paras=
6 {
7 new SqlParameter("@Item_Name",SqlDbType.VarChar,100),
8 new SqlParameter("@Item_Price",SqlDbType.Int),
9 new SqlParameter("@Image_Name",SqlDbType.VarChar,100),
10 new SqlParameter("@Description",SqlDbType.VarChar,100),
11 new SqlParameter("@AddedBy",SqlDbType.VarChar,100)
12 };
13 paras[0] .Value= entity.Item_Name;
14 paras[1].Value = entity.Item_Price;
15 paras[2].Value = entity.Image_Name;
16 paras[3].Value = entity.Description;
17 paras[4].Value = entity.AddedBy;
18 var sb=new StringBuilder();
19 sb.Append("insert into ItemDetails (Item_Name,Item_Price,Image_Name,Description,AddedBy) ");
20 sb.AppendFormat("values ('{0}',{1},'{2}','{3}','{4}')", paras[0].Value, paras[1].Value, paras[2].Value,paras[3].Value, paras[4].Value);
21 return sb.ToString();
22 }
23 }
2 {
3 public static string BatchAdd(ItemDetails entity)
4 {
5 SqlParameter [] paras=
6 {
7 new SqlParameter("@Item_Name",SqlDbType.VarChar,100),
8 new SqlParameter("@Item_Price",SqlDbType.Int),
9 new SqlParameter("@Image_Name",SqlDbType.VarChar,100),
10 new SqlParameter("@Description",SqlDbType.VarChar,100),
11 new SqlParameter("@AddedBy",SqlDbType.VarChar,100)
12 };
13 paras[0] .Value= entity.Item_Name;
14 paras[1].Value = entity.Item_Price;
15 paras[2].Value = entity.Image_Name;
16 paras[3].Value = entity.Description;
17 paras[4].Value = entity.AddedBy;
18 var sb=new StringBuilder();
19 sb.Append("insert into ItemDetails (Item_Name,Item_Price,Image_Name,Description,AddedBy) ");
20 sb.AppendFormat("values ('{0}',{1},'{2}','{3}','{4}')", paras[0].Value, paras[1].Value, paras[2].Value,paras[3].Value, paras[4].Value);
21 return sb.ToString();
22 }
23 }
Controller层调用：
1 public ActionResult Add(ItemDetails entity)
2 {
3 var sw = new Stopwatch();
4 sw.Start();
5 using (var db = new ShoppingDBConn())
6 {
7 var sql = new StringBuilder();
8 for (int i = 0; i < 10000; i++)
9 {
10 //生成SQL
11 sql.Append(ItemDetailBatch.BatchAdd(entity));
12 }
13 //一次性执行SQL
14 db.Database.ExecuteSqlCommand(sql.ToString());
15 }
16 sw.Stop();
17 var date = sw.Elapsed;
18 return Json(string.Format("总耗时：{0}", date));
19 }
2 {
3 var sw = new Stopwatch();
4 sw.Start();
5 using (var db = new ShoppingDBConn())
6 {
7 var sql = new StringBuilder();
8 for (int i = 0; i < 10000; i++)
9 {
10 //生成SQL
11 sql.Append(ItemDetailBatch.BatchAdd(entity));
12 }
13 //一次性执行SQL
14 db.Database.ExecuteSqlCommand(sql.ToString());
15 }
16 sw.Stop();
17 var date = sw.Elapsed;
18 return Json(string.Format("总耗时：{0}", date));
19 }
界面数据：
同样10000条总耗时：
EF没添加一次都要向数据库提交一次，而直接拼接SQL的方法就是减少与数据库的交互次数，一次性提交执行所有数据；
本文链接：EF大数据批量添加性能问题，转载请注明。