In memory database in .net

Question

I have a .net application where i want to use In-Memory data structure. Please advice me how it works in compare to the physical database.

Solution 1

In-Memory Database (IMDB) is a memory-resident relational database that eliminates disk access by storing and manipulating data in main memory. An IMDB usually features a strict memory-based architecture and direct data manipulation.

A bit related stuff's :

• Good in-memory database for .net/PowerShell?
• SQLite in-memory database backup in .NET
• Alternative to the TimesTen in memory database in .NET

Solution 2

There are two myths that should be corrected when you describe memory databases.

1) "A memory database is less persistent that a disk database". While this is true for simpler memory databases, enterprise level memory databases secures data to disk when they commit transactions. Disks are only slow when the disk arms move. If you think about it, you can write a gigabyte in seconds on a fast disk. And if your database changes by that much, you can secure terabytes per day in real time. This makes ram databases such as HANA and Starcounter as safe as disk databases while super fast. You can turn of the power at any time and checkpoints and recoveries works the same as for disk based databases.

2) "Memory databases are much faster." The reason why memory databases are faster is simply because they operate in memory. If you put a traditional database on a RAM drive, nothing much happens. In fact, as caches these days typically exceeds your database size, they already reside in memory. The reason the memory database is so much more efficient is that the database image is treated as primary memory and not secondary memory. This means that a modern RAM database does not copy pages from disk image to RAM when it reads data. In modern servers, the memory wall quickly becomes a bottleneck. This is avoided in RAM databases. The second reason is that when you develop something for a medium that is thousands of times faster than disk, you tend not to add overhead in microseconds and milliseconds as things that consumes nanoseconds are immediately visible. At the scale of disks, there is no reason to optimize code at this level. When suddenly RAM prices drop (98% since 2000), you cannot just rewrite your whole database engine.

Solution 3

An in memory database works just like an ordinary database, but the content is stored in memory instead of on disk. This has the effect that all data is lost when the application is shut down. They have to be rebuild and populated with data on each startup.

An example of a database that can run in in-memory mode is SQLite. Note that SQLite is an in process database, you host it within your application.

Solution 4

You can use the MemCached that it is a distributed memory object caching system. Memcached is an in-memory key-value store for small chunks of arbitrary data (strings, objects) from results of database calls, API calls, or page rendering.

With memcached, you can see that all of the servers are looking into the same virtual pool of memory. This means that a given item is always stored and always retrieved from the same location in your entire web cluster.

Solution 5

Here's a tutorial to access an in-memory database using Data Provider for .NET. You can use SQL statements for predictive analytics, geospatial, text analytics and fuzzy search. You can download this DB for free (HANA, express edition) and use it in a Virtual Machine in a computer with more than 8GB RAM or install it into a VM in MS Azure.

View more solutions

Comments

[anothershrubery]

You'll need to clarify a bit further your actual objectives here.

[MUG4N]

sounds like you think about datasets: msdn.microsoft.com/en-us/library/ss7fbaez.aspx

[Mike de Klerk]

The abbreviation IMDB might be correct but is pretty useless. As google quickly yield urls pointing to the movie rating website. No offense though.

[user1496062]

in Addition to 2) Compared to say Sql server Express you don't have the IPC costs which are significant , the serialization to sql formats nor SQL query interpretation.