sofia, bulgaria | 9-10 october advanced data access patterns with ado.net 2.0 julie lerman the data...
TRANSCRIPT
Sofia, Bulgaria | 9-10 OctoberSofia, Bulgaria | 9-10 October
Advanced Data Access Patterns
Advanced Data Access Patterns
with ADO.NET 2.0with ADO.NET 2.0
Julie Lerman
The Data Farm
Julie Lerman
The Data Farm
Sofia, Bulgaria | 9-10 October
About MeAbout Me● .NET Consultant and Mentor
● 20+ years developing
● Microsoft .NET MVP
● ASPInsider
● INETA Speaker
● Various publications & conferences● Blogs: thedatafarm.com/blog, blog.ziffdavis.com/devlife
● Founder and leader of Vermont .NET
● INETA Speaker Committee, VTSDA Board
● .NET Consultant and Mentor
● 20+ years developing
● Microsoft .NET MVP
● ASPInsider
● INETA Speaker
● Various publications & conferences● Blogs: thedatafarm.com/blog, blog.ziffdavis.com/devlife
● Founder and leader of Vermont .NET
● INETA Speaker Committee, VTSDA Board
Sofia, Bulgaria | 9-10 October
AgendaAgenda
● ADO.NET 2.0 DataSet performance
● Pushing huge data back to SQL Server
● Caching huge amounts of data
● Querying huge amounts of data in memory
● ADO.NET 2.0 DataSet performance
● Pushing huge data back to SQL Server
● Caching huge amounts of data
● Querying huge amounts of data in memory
Sofia, Bulgaria | 9-10 October
New in ADO.NET 2.0New in ADO.NET 2.0
● New features used in this session
● Batch Updating with DataAdapter
● SqlBulkCopy Class
● SqlDependency Class
● New features used in this session
● Batch Updating with DataAdapter
● SqlBulkCopy Class
● SqlDependency Class
Sofia, Bulgaria | 9-10 October
ADO.NET 2.0 PerformanceADO.NET 2.0 Performance
● DataSet Scalability across the board
● Can handle lots of rows
● Indexing Engine completely re-written
● Incremental indexing updates
● much more
● Low overhead for using Data Views
● Serialization: Faster, more compact
● Supports Binary & Schema-less serialization
● DataSet Scalability across the board
● Can handle lots of rows
● Indexing Engine completely re-written
● Incremental indexing updates
● much more
● Low overhead for using Data Views
● Serialization: Faster, more compact
● Supports Binary & Schema-less serialization
Sofia, Bulgaria | 9-10 OctoberSofia, Bulgaria | 9-10 October
DemonstrationDemonstration
Indexing Engine PerformanceIndexing Engine Performance
Sofia, Bulgaria | 9-10 October
Moving BIG DataMoving BIG Data
● Getting data is easy and fast
● Uploading is a different story
● 1.1 Update Options
● DataAdapter.Update – one row at a time
● Controlled stored procedure update – one row at a time
● Getting data is easy and fast
● Uploading is a different story
● 1.1 Update Options
● DataAdapter.Update – one row at a time
● Controlled stored procedure update – one row at a time
Sofia, Bulgaria | 9-10 October
Updating in 2.0Updating in 2.0
● DataAdapter Batch Update
● Pushes groups of rows up to SQL Server
● SQL Server still updates one at a time
● Find sweet spot for UpdateBatchSize value
● SqlDataAdapter & OracleDataAdapter
● Bulk Copy
● Streamed Insert using SQL Server Bulk Copy
● DataAdapter Batch Update
● Pushes groups of rows up to SQL Server
● SQL Server still updates one at a time
● Find sweet spot for UpdateBatchSize value
● SqlDataAdapter & OracleDataAdapter
● Bulk Copy
● Streamed Insert using SQL Server Bulk Copy
*slide slightly modified from original printed version
Sofia, Bulgaria | 9-10 OctoberSofia, Bulgaria | 9-10 October
DemonstrationDemonstration
Batch Update & Bulk Copy ClassBatch Update & Bulk Copy Class
Sofia, Bulgaria | 9-10 October
Fast Updates w/BCPFast Updates w/BCP
● Use BCP to stream Inserts & Updates and Deletes to temptables
● Create & run a DML TSQL command to process inserts, updates and deletes from the temptables into the real tables
● Wrap it all in a transaction
● End to End streaming with DataReader
● Use BCP to stream Inserts & Updates and Deletes to temptables
● Create & run a DML TSQL command to process inserts, updates and deletes from the temptables into the real tables
● Wrap it all in a transaction
● End to End streaming with DataReader
DML: Database Manipulation Language
Sofia, Bulgaria | 9-10 OctoberSofia, Bulgaria | 9-10 October
DemonstrationDemonstration
Using BCP & DML to process huge amounts of data
Using BCP & DML to process huge amounts of data
Sofia, Bulgaria | 9-10 October
DataSet CachingDataSet Caching
● DataSet improvements enable storing lots of data into cache
● SqlDependency enables automated notification of changes to .NET
● Combine SqlDependency and Caching for serious resource reduction
● DataSet improvements enable storing lots of data into cache
● SqlDependency enables automated notification of changes to .NET
● Combine SqlDependency and Caching for serious resource reduction
Sofia, Bulgaria | 9-10 OctoberSofia, Bulgaria | 9-10 October
DemonstrationDemonstration
SqlDependency & DataSet CachePart 1
SqlDependency & DataSet CachePart 1
Sofia, Bulgaria | 9-10 October
Break up Big cacheBreak up Big cache
● SqlDependency is all or nothing
● Create separate dependencies for logical sections of your cache
● SqlDependency is all or nothing
● Create separate dependencies for logical sections of your cache
Sofia, Bulgaria | 9-10 OctoberSofia, Bulgaria | 9-10 October
DemonstrationDemonstration
SqlDependency & DataSet CachePart 2: Achieve granularity
SqlDependency & DataSet CachePart 2: Achieve granularity
Sofia, Bulgaria | 9-10 October
Query BIG DataSetQuery BIG DataSet● Query a huge DataSet in cache● In-Memory Querying is coming!
● LiNQ● VB9/C#3
● What about today?● ADO.NET 2.0 has everything you need
to build a simple Query Processor● Great example of QP from ADO.NET
Team Technical Lead Pablo Castro (“for demo only”)
● Query a huge DataSet in cache● In-Memory Querying is coming!
● LiNQ● VB9/C#3
● What about today?● ADO.NET 2.0 has everything you need
to build a simple Query Processor● Great example of QP from ADO.NET
Team Technical Lead Pablo Castro (“for demo only”)
Sofia, Bulgaria | 9-10 October
Sample Query ProcessorSample Query Processor
● Takes advantage of new ADO.NET 2.0 tools
● Ability to query huge datasets
● Uses Caching
● Super Fast indexing
● DataTable.Select
● DataView
● Very simplistic compared to T-SQL
● *NOT* for production use
● Takes advantage of new ADO.NET 2.0 tools
● Ability to query huge datasets
● Uses Caching
● Super Fast indexing
● DataTable.Select
● DataView
● Very simplistic compared to T-SQL
● *NOT* for production use
Sofia, Bulgaria | 9-10 OctoberSofia, Bulgaria | 9-10 October
DemonstrationDemonstration
A look at a sample Query Processor written with ADO.NET 2.0
A look at a sample Query Processor written with ADO.NET 2.0
Sofia, Bulgaria | 9-10 October
SummarySummary● ADO.NET 2.0’s performance and
functional improvements put new power in our hands
● You can work with huge amounts of data efficiently
● Functionality like BatchUpdate gives you easy access to some of this power
● With a little more code, you can do wonders!
● ADO.NET 2.0’s performance and functional improvements put new power in our hands
● You can work with huge amounts of data efficiently
● Functionality like BatchUpdate gives you easy access to some of this power
● With a little more code, you can do wonders!
Sofia, Bulgaria | 9-10 October
ResourcesResources
● PDC05 Session DAT408, Pablo Castro, ADO.NET Team Technical Lead
● ADO.NET Team Blog: blogs.msdn.com/dataaccess
● MSDN Data Access Dev Center
● msdn.microsoft.com/data
● What’s New in ADO.NET 2.0, Julia Lerman, MSDN Magazine April 2005
● PDC05 Session DAT408, Pablo Castro, ADO.NET Team Technical Lead
● ADO.NET Team Blog: blogs.msdn.com/dataaccess
● MSDN Data Access Dev Center
● msdn.microsoft.com/data
● What’s New in ADO.NET 2.0, Julia Lerman, MSDN Magazine April 2005
Sofia, Bulgaria | 9-10 October
ADO.NET 2.0 BookshelfADO.NET 2.0 Bookshelf
● ADO.NET and System.Xml v 2.0 – The Beta Version: Alex Homer, Dave Sussman & Mark Fussell, Addison-Wesley
● PRO ADO.NET 2.0, Sahil Malik, APress
● Professional ADO.NET 2: Programming with SQL Server 2005, Oracle, and MySQL, Wallace B. McClure, Gregory A. Beamer, IV John J. Croft, J. Ambrose Little, Bill Ryan, Phil Winstanley, David Yack, Jeremy Zongker, WROX Press
● Programming Microsoft ADO.NET 2.0 Applications:
Advanced Topics, Glenn Johnson, Microsoft Press
● Microsoft ADO.NET 2.0 Step by Step, Rebecca Riordan, Microsoft Press
● ADO.NET and System.Xml v 2.0 – The Beta Version: Alex Homer, Dave Sussman & Mark Fussell, Addison-Wesley
● PRO ADO.NET 2.0, Sahil Malik, APress
● Professional ADO.NET 2: Programming with SQL Server 2005, Oracle, and MySQL, Wallace B. McClure, Gregory A. Beamer, IV John J. Croft, J. Ambrose Little, Bill Ryan, Phil Winstanley, David Yack, Jeremy Zongker, WROX Press
● Programming Microsoft ADO.NET 2.0 Applications:
Advanced Topics, Glenn Johnson, Microsoft Press
● Microsoft ADO.NET 2.0 Step by Step, Rebecca Riordan, Microsoft Press
Sofia, Bulgaria | 9-10 October
Contact InfoContact Info
Julie Lerman
www.thedatafarm.com
Blogs
www.thedatafarm.com/blog
blogs.ziffdavis.com/devlife
Julie Lerman
www.thedatafarm.com
Blogs
www.thedatafarm.com/blog
blogs.ziffdavis.com/devlife
Sofia, Bulgaria | 9-10 OctoberSofia, Bulgaria | 9-10 October
Please fill out the survey forms!
They are the key to amazing prizes that you can get at the end of each day
Please fill out the survey forms!
They are the key to amazing prizes that you can get at the end of each day
Thank you!Thank you!