CMemMapFile v1.67

Welcome to CMemMapFile, A freeware C++ based class to encapsulate memory mapped files.

Features

Memory mapping is a powerful mechanism Win32 provides to implement shared memory and also to access files though a simple memory pointer without having to implement your own home brewed buffering mechanisms. As an example its as simple as calling

void* lpData = mmf.Open();
CharUpperBuff(static_cast<LPSTR>(lpData),dwFileLength);

to convert a file to upper case.

Areas where you might find this of interest include very large database files with fixed records, audio processing, string operations and image processing.

The other side of memory mapped files is to implement shared memory. As you will probably know, Win32 puts each process into its own address space, thus making it impossible to pass ordinary pointers across process boundaries. With memory mapped files you get back this very useful mechanism.

This class provides a simple MFC class interface and a simple dialog based application which demonstrates all the functionality of the class.

Usage

• To use CMemMapFile in your project simply #include memmap.h from the test application in whichever files you want to use the class in.
• As of v1.60, the class is only supported on VC 2017 or later.
• Included in the download is a simple dialog based application which demonstrates the 2 main features of memory mapped files, namely mapping a file system file to a pointer and implementation of shared memory. For further details about the example program have a look at the BOOL CTestmemmapApp::InitInstance() function and the CDialog1 member functions both in testmemmap.cpp

Copyright

• You are allowed to include the source code in any product (commercial, shareware, freeware or otherwise) when your product is released in binary form.
• You are allowed to modify the source code in any way you want except you cannot modify the copyright details at the top of each module.
• If you want to distribute source code with your application, then you are only allowed to distribute versions released by the author. This is to maintain a single distribution point for the source code.

History

v1.67 (15 February 2022)

• Updated copyright details.
• Updated the code to use C++ uniform initialization for all variable declarations

v1.66 (26 March 2020)

• Fixed a compile issue in UnMapEx method where parameter was declared incorrectly. Thanks to Steve Arnold for reporting this issue.

v1.65 (16 March 2020)

• Updated copyright details.
• Fixed more Clang-Tidy static code analysis warnings in the code.

v1.64 (15 September 2019)

• Fixed a number of compiler warnings when the code is compiled with VS 2019 Preview

v1.63 (21 April 2019)

• Updated copyright details
• Updated the code to clean compile on VC 2019

v1.62 (1 September 2018)

• Fixed a number of compiler warnings when using VS 2017 15.8.2

v1.61 (8 June 2018)

• Removed dependency on std::invoke from the code

v1.60 (3 June 2018)

• Updated copyright details
• Fixed a number of C++ core guidelines compiler warnings. These changes mean that the code will now only compile on VC 2017 or later
• Replaced NULL throughout the code with nullptr
• Added support for MapViewOfFileEx API
• Added support for UnmapViewOfFileEx API
• Added support for OpenFileMappingFromApp API
• Added support for MapViewOfFileFromApp API

v1.59 (18 December 2015)

• Updated copyright details
• Added SAL annotations to all the code.
• Updated the code to clean compile on VC 2015

v1.58 (26 January 2014)

• Updated copyright details.
• Updated the code to clean compile on VC 2013
• Updated the code to clean compile using /analyze
• Reverted the code changes which made the class completely thread-safe. The issue was that you ended up with a class which had two different synchronisation primitives running in parallel throughout the class codebase. This design pattern is always bad and in this case could lead to thread deadlock situations in some situations. Similar to the MFC thread synchronisation primitives design patterns, an instance of the CMemMapFile class should only be used from the one thread. If you want to use the same logical memory mapped file from multiple threads in your application, then create a second instance and simple call the MapFile, MapMemory or MapExistingMemory methods to get access to the mapping. See the "MoniterSM" worker thread in testmemmap.cpp in the sample app on how to do this correctly. Thanks to "jianhongsong" for reporting this issue.

v1.57 (16 March 2012)

• Thread protected the GetFileHandle and GetFileMappingHandle methods.

v1.56 (15 March 2012)

• Updated copyright details
• Made the class completely thread-safe meaning that you can share instances of CMemMapFile across threads without worrying about corruption of its member variables. Thanks to HaeRim Lee for prompting this update. To achieve this I used a nested class called CMemMapCriticalSection which is derived from the ATL::CCriticalSection class. This CMemMapCriticalSection class provides for critical sections with spin locks and normal critical sections as well as stack based release semantics for critical sections through the use of the ATL::CComCritSecLock class. It would be nice if the built in ATL CriticalSection class supported this but this is still not present as of VC 2010 SP1.

v1.55 (25 November 2011)

• Updated class to be completely MFC independent. You can now use CMemMapFile in an ATL only project. Thanks to HaeRim Lee for prompting this update.

v1.54 (20 November 2011)

• Updated copyright details
• Updated code to clean compile on VC 2010
• Replaced ASSERT calls with ATLASSERT
• Updated MapMemory and MapFile methods to include a new BOOL bNoCache parameter. Thanks to HaeRim Lee for providing this nice update
• Removed m_bOpen member variable and all code which uses it. This can avoid thread safety issues when a class instance of CMemMapFile is shared across threads. Thanks to HaeRim Lee for providing this nice update
• Reworked the code in CMemMapFile::UnMap to call ReleaseMutex directly instead of calling Close. This method now shares the same consistent ordering of object creation / destruction as the main Map... methods. This also fixes an issue where the mutex previously got released twice. Thanks to HaeRim Lee for providing this nice update
• The whole class is now implemented in Memmap.h and the Memmap.cpp module is now defunct and redundant. You now just need to #include the Memmap.h in client code to use the class.

v1.53 (6 July 2009)

• Updated copyright details
• Updated the sample apps project settings to more modern default values.
• All the failure paths in the methods now preserve the Win32 last error value
• Removed unnecessary code to get the length of the file in MapFile. Also simplified the logic in this method.
• All parameters which specify a mapping size now use a SIZE_T parameter which is consistent with their equivalent API values.
• All calls to MapViewOfFile are now checked for failure

v1.52 (16 August 2008)

• Updated copyright details
• Code now compiles cleanly using Code Analysis (/analyze)
• Updated code to compile correctly using _ATL_CSTRING_EXPLICIT_CONSTRUCTORS define
• The code now only supports VC 2005 or later.
• Removed VC 6 style AppWizard comments from the code.
• Updated the code to correctly handle "Global\" and "Local\" prefixes which can be included with any Win32 Named object. Thanks to Gert Rijs for reporting this bug. To achieve this the parameters to the MapFile method has been reworked. This function now takes a pszMappingName which allows client code to explicitly specify the mapping name
• The mutex name use to serialize access to the contents of the memory mapped file is now explicitly provided as a external parameter to the class.
• MapExistingMemory now has a LPSECURITY_ATTRIBUTES parameter

v1.51 (7 July 2006)

• Code now uses newer C++ style casts instead of C style casts.
• The code now requires the Platform SDK if compiled using VC 6.
• Updated code to compile cleanly using VC 2005.

v1.50 (2 May 2006)

• Updated the copyright details in the modules.
• Updated the documentation to use the same style as the web site.
• Addition of a CMEMMAPFILE_EXT_CLASS macro to allow the class to be easily incorporated into an extension dll.
• The sample app now uses filenames without "(" or ")" characters in their filenames. Thanks to Andrew MacGinitie for reporting this issue.
• Fixed a bug in the sample app where it reports the wrong filename when doing the lowercase conversion. Again thanks to Andrew MacGinitie for reporting this issue.
• Fixed an issue in the download where the sample file called "input.txt" is now provided out of the box. Again thanks to Andrew MacGinitie for reporting this issue.
• Fixed an issue in the sample app when the code is compiled with /Wp64

v1.49 (30 April 2005)

• Removed derivation from CObject MFC class. In fact the class can now operate entirely independently of MFC.
• Destructor is now virtual.
• Fixed a bug where the mutex name which is used to synchronize access to the MMF's data could by default have the same name for multiple MMF's.
• CreateMappingName and CreateMutexName methods are now virtual

v1.48 (23 December 2004)

• Removed unnecessary include of winioctl.h header file.
• Fixed a problem with the declaration of the FSCTL_SET_SPARSE macro which is used in the support of growable MMF. With the VC6 WinIOCTL.h, the macro generates the value 0x000980C4, but the correct value for the DeviceIoControl function to enable the sparse file option is 0x000900C4. Thanks to a posting on CodeProject for pointing out this problem.
• Optimized CMemMapFile::CreateMappingName by now using CString::Replace.
• Addition of a bInheritHandle parameter to MapExistingMemory.
• Fixed a bug in the handling of calls to GetFileSize()
• Removed unnecessary check to verify that mapping size is greater than 4GB since CreateFileMapping will do this for us.
• Updated sample app to use a sample "input.txt" file for demonstration purposes instead of using "c:\config.sys"
• Code now uses unsigned __int64 for specifying indexes and lengths for memory mapping. This should allow the code to be easily used to map > 4GB on 64 bit versions of Windows.
• Tidied up some of the ASSERT code at the top of each key function in the class.

v1.47 (6 June 2004)

• Fixed an issue in MapHandle where the wrong value was sent to CreateFileMapping. This issue only occurs when you are not mapping the whole of a file, but instead decide to perform the mapping a chunk at a time. Thanks to Nicolas Stohler for reporting this problem.
• Removed the AppendNull option as it is incompatible with general use of memory mapped files.
• Reviewed all the TRACE statements throughout the class
• Added ASSERT validation at the top of functions which modify member variables
• Failing to create mutex in MapHandle and MapExistingMemory not fails the function

v1.46 (17 November 2003)

• Fixed a memory leak in UnMap as reported by Bart Duijndam using "Memory Validator". The memory leak may in fact not be real, but the code change avoids the reported problem.

v1.45 (5 October 2003)

• Updated copyright details.
• Fixed a small typo in the description of the FSCTL_SET_SPARSE IOCTL. Thanks to amores perros for reporting this.
• Fixed a minor tab indentation problem at the start of the MapFile method. Again thanks to amores perros for reporting this.
• Removed the unnecessary AssertValid function. Again thanks to amores perros for reporting this.

v1.44 (4 April 2001)

• Now supports passing in a security descriptor to MapFile and MapMemory
• Now supports opening a specified portion of a file, rather than always mapping all of the file

v1.43 (7 March 2001)

• Updated copyright information.
• Fixed problem where mutex was not being locked when read only access to the memory mapped file was desired. Access to the MMF should be synchronised irrespective of the access mode.

v1.42 (24 March 2000)

• Fixed a simple typo problem in a TRACE statement when compiled for UNICODE

v1.41 (21 April 1999)

• Added code to work around a Windows bug where you try to memory map a zero length file on Windows 95 or 98.

v1.4 (30 March 1999)

• Minor updates to the style of the help file.
• Code is now UNICODE compliant and build configurations are provided.
• Code now supports growable memory mapped files as provided with Windows 2000 and NTFS 5 volumes.
• Addition of accessor functions for file handle and file mapping handle

v1.3 (22 October 1998)

• Fixed a bug in a number of calls to CreateMappingName when the classes were being used to share memory.
• New documentation in the form of this HTML file.
• Sample now ships as standard with VC 5 workspace files
• Tidy up of the demo app including:

  Made the amount of text being shared a constant of MAX_EDIT_TEXT instead of hardcoding it to 20 everywhere in the sample.

  Changed where the timer is being created to OnInitDialog

  Tidied up the initialisation sequence in OnInitDialog

  Now using _tcscpy instead of _tcsncpy to ensure array is null terminated

  Fixed resource.h which was causing the resources to fail to compile

  Removed unnecessary symbols from resource.h

  Optimised the way the OnTimer code works to only update the text when it has changed in the MMF. This means that you can type continuously into the edit control.

v1.2 (29 May 1998)

• Mapping a file now has the option of making it named or not.

v1.1 (20 April 1998)

• Now uses GetFileSize SDK call instead of GetFileInformationByHandle as a more "reliable" way to determine file length.
• Included TRACE statements to call GetLastError in all places where SDK functions fail

v1.0 (31 March 1998)

• Class now avoids trying to lock the mutex if only read access is required
• User now has the option of specifying whether a file should be mapped with A Null terminator at the end. Can prove helpful when you want to use some of the "C" runtime functions on the pointer returned.

CMemMapFile Class Members

CMemMapFile
~CMemMapFile
MapFile
MapMemory
MapExistingMemory
Open
Close
UnMap
UnMapEx
Flush
GetFileHandle
GetFileMappingHandle

CMemMapFile::CMemMapFile

CMemMapFile();

Remarks

Standard Constructor for the class.

CMemMapFile::~CMemMapFile

~CMemMapFile();

Remarks

Standard destructor for the class. Calls the UnMap function

CMemMapFile::MapFile

BOOL MapFile(LPCTSTR pszFilename, BOOL bReadOnly = FALSE, DWORD dwShareMode = 0, LPCTSTR pszMappingName = NULL, LPCTSTR pszMutexName = NULL, BOOL bGrowable = FALSE, const unsigned __int64& dwStartOffset = 0, const SIZE_T& nNumberOfBytesToMap = 0, LPSECURITY_ATTRIBUTES lpSecurityAttributes = NULL, BOOL bNoCache = FALSE, LPVOID pBaseAddress = nullptr);

Return Value

TRUE if the file was successfully mapped otherwise FALSE

Parameters

sFilename is the filename to memory map

bReadOnly can be set to true if you are not interested in modifying the file

dwShareMode is the share flags to use when calling CreateFile. This may be prove useful if you want to map a file using MMF techniques but allow other processes access to it as read only

pszMappingName is the Win32 name of the memory mapping you want to create. Normally when you map a file there is no need to do this as the memory mapping is not being used to share memory and you can leave this value as NULL

pszMutexName is the Win32 name of the mutex used to serialize access to the memory mapping

bGrowable If you set this value to TRUE, then the underlying file will be set to be a "sparse" file. In the context of memory mapped files, this means that you will not get access violations when you try to write past the end of the file, instead the OS will silently grow the file for you. Please note that this is only supported on NTFS 5 volumes on Windows 2000 or later. This could be useful where you want to use memory mapped files to implement features such as direct to disk audio recording where you do not want to pre-allocate a certain size disk file up front

dwStartOffset The offset into the file at which you want the file mapping to start

nNumberOfBytesToMap The number of bytes you would like to map. The default value of 0 means that all the file will be mapped

lpSecurityAttributes Allows NT security attributes to be associated with this file mapping

bNoCache If set to TRUE, then the SEC_NOCACHE flag is passed to the CreateFileMapping function

pBaseAddress The base memory address at which to perform the memory mapping

Remarks

Maps a file system file. Note that an attempt to modify a file mapping when it is in read only mode will generate an access violation

CMemMapFile::MapMemory

BOOL MapMemory(LPCTSTR pszMappingName, LPCTSTR pszMutexName, const SIZE_T& nBytes, BOOL bReadOnly = FALSE, LPSECURITY_ATTRIBUTES lpSecurityAttributes = NULL, BOOL bNoCache = FALSE, const unsigned __int64& dwStartOffset = 0, LPVOID pBaseAddress = nullptr);

Return Value

TRUE if the memory was successfully mapped otherwise FALSE

Parameters

pszMappingName is the Win32 name of the memory mapping you want to create

pszMutexName is the Win32 name of the mutex used to serialize access to the memory mapping

nBytes is the size to map

bReadOnly is as in MapFile

lpSecurityAttributes Allows NT security attributes to be associated with this file mapping

bNoCache If set to TRUE, then the SEC_NOCACHE flag is passed to the CreateFileMapping function

dwStartOffset The offset into the memory at which you want the file mapping to start

pBaseAddress The base memory address at which to perform the memory mapping

Remarks

This creates a piece of shared memory

CMemMapFile::MapExistingMemory

BOOL MapExistingMemory(LPCTSTR pszMappingName, LPCTSTR pszMutexName, const SIZE_T& nBytes, BOOL bReadOnly = FALSE, BOOL bInheritHandle = FALSE, LPSECURITY_ATTRIBUTES lpSecurityAttributes = NULL, _In_ const unsigned __int64& dwStartOffset = 0, LPVOID pBaseAddress = nullptr);

Return Value

TRUE if the memory was successfully mapped otherwise FALSE

Parameters

pszMappingName is the Win32 name of the memory mapping you want to create

pszMutexName is the Win32 name of the mutex used to serialize access to the memory mapping

nBytes is the size to map

bReadOnly is as in MapFile

bInheritHandle Specifies if the handle is to be inherited by a new process during process creation

lpSecurityAttributes Allows NT security attributes to be associated with this file mapping

dwStartOffset The offset into the memory at which you want the file mapping to start

pBaseAddress The base memory address at which to perform the memory mapping

Remarks

This function is quite similar to MapMemory except that it fails it the mapping is already created. This function can be used to connect to existing shared memory or where you want to detect if this is the first instance of your program to run

CMemMapFile::Open

LPVOID Open(DWORD dwTimeout = INFINITE);

Return Value

Pointer to the data as stored in the memory mapped file

Parameters

dwTimeout The dwTimeout value is the time in milliseconds to wait if the mapping is already opened by another thread or process

Remarks

Once you have called MapFile, MapMemory or MapExisitingMemory you can call this function to actually get a pointer to the data. Internally the class uses a mutex to synchronise access to the memory so that you do not have to worry about thread synchronisation problems when using the class. If you use the default value then your thread will be suspended indefinitely if another thread is using the object.

CMemMapFile::Close

BOOL Close();

Remarks

This is the corollary of the Open function and there should be a matching call to Close for each call to Open. After calling close you are free to call Open again to get back the pointer

CMemMapFile::UnMap

void UnMap();

Remarks

This unmaps the object and releases all the file handles and synchronisation objects. You can consider this function to be the corollary to the Map...() functions. This function is also called in the destructor if you forget to do it yourself

CMemMapFile::UnMapEx

void UnMapEx(ULONG UnmapFlags);

Parameters

UnmapFlags The parameter value which is passed to the UnampViewOfFileEx API

Remarks

This unmaps the object and releases all the file handles and synchronisation objects using UnmapViewOfFileEx rather than UnmapViewOfFile.

CMemMapFile::Flush

BOOL Flush();

Return Value

Returns TRUE if the flush was successful otherwise FALSE

Remarks

Flushes the mapping object to disk.

CMemMapFile::GetFileHandle

HANDLE GetFileHandle()

Return Value

Returns the file handle which this memory mapping instance encapsulates. This value returned will be INVALID_HANDLE_VALUE if it is called on a memory mapping which is being used to share memory rather than a file on the file system.

CMemMapFile::GetFileMappingHandle

HANDLE GetFileMappingHandle();

Return Value

Returns the file mapping handle which this memory mapping instance encapsulates.

Contacting the Author

PJ Naughter
Email: pjna@naughter.com
Web: http://www.naughter.com
15 February 2022