How to bulk insert / update in linq in sql?
How can I accomplish these 2 scenarios.
I am currently doing something like this
public class Repository
{
private LinqtoSqlContext dbcontext = new LinqtoSqlContext();
public void Update()
{
// find record
// update record
// save record ( dbcontext.submitChanges()
}
public void Insert()
{
// make a database table object ( ie ProductTable t = new ProductTable() { productname
="something"}
// insert record ( dbcontext.ProductTable.insertOnSubmit())
// dbcontext.submitChanges();
}
}
So now I am trying to load an XML file which has a lot of records. First, I check the entries one at a time. Then I want to insert them into the database, but instead of doing submitChanges () after each record, I want to make a massive gift at the end.
So I have something like this
public class Repository
{
private LinqtoSqlContext dbcontext = new LinqtoSqlContext();
public void Update()
{
// find record
// update record
}
public void Insert()
{
// make a database table object ( ie ProductTable t = new ProductTable() { productname
="something"}
// insert record ( dbcontext.ProductTable.insertOnSubmit())
}
public void SaveToDb()
{
dbcontext.submitChanges();
}
}
Then in my service layer I would like
for(int i = 0; i < 100; i++)
{
validate();
if(valid == true)
{
update();
insert()
}
}
SaveToDb();
So, pretend that my for loop has a count for the entire record found in the XML file. I check it first. If it is valid, I must update the table before inserting the record. Then I insert the entry.
After that, I want to save everything in one go.
I'm not sure if I can bulk save on refresh if it should be after every time or what.
But I thought it probably works for insert.
Nothing seems to crash and I'm not sure how to check if entries are being added to the dbcontext.
a source to share
Simple answer: you don't. Linq2Sql is a lot of stuff - it is not a replacement for bulk load / bulk copy. You will be much more efficient using the ETL route:
- Generate flat file (csv, etc.) with new data
- Load it into the database using bulk loading mechanisms
- If the data is updated, etc. - load them into temporary tables and use the MERGE command to merge it into the main table.
Linq2Sql by design will always suck in bulk insert scripts. ORMs are simply not ETL tools.
a source to share
Even when you add multiple records to the DataContext before calling SubmitChanges, LINQ2SQL will loop through and insert them one at a time. You can check this by executing one of the partial methods of the entity class ("InsertMyObject (MyObject instance)"). It will be called for each pending line separately.
I don't see anything wrong with your plan - you say it works, but you just don't know how to test it? Can't you just look in the database to check if records have been added?
Another way to see which records are expected in the DataContext and have not yet been added is to call GetChangeSet () in the data context and then access the Inserts property of the returned object to get a list of rows that will be inserted when SubmitChanges is called.
a source to share
I haven't "released" this project yet, but it's a T4 based repository system that extends Linq To SQL and implements a bunch of batch operations (delete, update, csv creation, etc.): http://code.google.com / p / grim-repo / . You can check the source code and implement it however you see fit.
Also, this link has great source code for batch operations: http://www.aneyfamily.com/terryandann/post/2008/04/Batch-Updates-and-Deletes-with-LINQ-to-SQL.aspx
And, also, I know it's tempting, but not crap on older people. Try batch operations with DataAdapters / ADO.net: http://davidhayden.com/blog/dave/archive/2006/01/05/2665.aspx . It is faster but inevitably fades.Finally, once you have an XML file, you can create a stored procedure that uses the embedded sproc SQL file, sp_xml_preparedocument. See how to use it here: http://msdn.microsoft.com/en-us/library/ms187367.aspx
a source to share