For files in the classic format, the NetCDF-Library uses a private cache. It works by dividing the NetCDF-file into pages (the size depends on the system prefered IO block size or is set to 8 kiB by default), that can be paged in/out separately. This has a number of effects:
After we understood the impact of the cache on large input/output operations, we decided to change the NetCDF source code to deactivate the cache completely and prepared a patch for it. This patch will be handed back to the NetCDF-developers, so we hope that there will be an official version soon which allows the user to switch off the cache.
Applying the patch:
Changing the caching behavior:
2012-07-30: Eliminated a bug in the handling of incomplete reads/writes.