mirror/netcdf-c
2
0
Fork 0
mirror of https://github.com/Unidata/netcdf-c.git synced 2024-12-27 08:49:16 +08:00
Code Issues Packages Projects Releases Wiki Activity
bab5ef7549
netcdf-c/nc_test/Make0

38 lines
895 B
Plaintext
Raw Normal View History

diskless: make read and write loop to read/write whole file when persisting
2012-04-10 06:03:02 +08:00
# Test c output
Fix https://github.com/Unidata/netcdf-c/issues/963 Fix https://github.com/Unidata/netcdf-c/issues/962 1. remove the --disable-diskless option since it is no longer needed. Similarly for CMakeLists.txt. 2. Fixed nc4files.c where BAIL and return were mixed leading to situation where cleanup code was not being invoked. This probably occurs elsewhere, but I did not find any specifically.
2018-05-12 05:30:19 +08:00
T=tst_diskless
re e-support UBS-599337 re pull request https://github.com/Unidata/netcdf-c/pull/405 re pull request https://github.com/Unidata/netcdf-c/pull/446 Notes: 1. This branch is a cleanup of the magic.dmh branch. 2. magic.dmh was originally merged, but caused problems with parallel IO. It was re-issued as pull request https://github.com/Unidata/netcdf-c/pull/446. 3. This branch + pull request replace any previous pull requests and magic.dmh branch. Given an otherwise valid netCDF file that has a corrupted header, the netcdf library currently crashes. Instead, it should return NC_ENOTNC. Additionally, the NC_check_file_type code does not do the forward search required by hdf5 files. It currently only looks at file position 0 instead of 512, 1024, 2048,... Also, it turns out that the HDF4 magic number is assumed to always be at the beginning of the file (unlike HDF5). The change is localized to libdispatch/dfile.c See https://support.hdfgroup.org/release4/doc/DSpec_html/DS.pdf Also, it turns out that the code in NC_check_file_type is duplicated (mostly) in the function libsrc4/nc4file.c#nc_check_for_hdf. This branch does the following. 1. Make NC_check_file_type return NC_ENOTNC instead of crashing. 2. Remove nc_check_for_hdf and centralize all file format checking NC_check_file_type. 3. Add proper forward search for HDF5 files (but not HDF4 files) to look for the magic number at offsets of 0, 512, 1024... 4. Add test tst_hdf5_offset.sh. This tests that hdf5 files with an offset are properly recognized. It does so by prefixing a legal file with some number of zero bytes: 512, 1024, etc. 5. Off-topic: Added -N flag to ncdump to force a specific output dataset name.
2017-10-25 06:25:09 +08:00
re: esupport MQO-415619 and https://github.com/Unidata/netcdf-c/issues/708 Expand the NC_INMEMORY capabilities to support writing and accessing the final modified memory. Three new functions have been added: nc_open_memio, nc_create_mem, and nc_close_memio. The following new capabilities were added. 1. nc_open_memio() allows the NC_WRITE mode flag so a chunk of memory can be passed in and be modified 2. nc_create_mem() allows the NC_INMEMORY flag to be set to cause the created file to be kept in memory. 3. nc_close_mem() allows the final in-memory contents to be retrieved at the time the file is closed. 4. A special flag, NC_MEMIO_LOCK, is provided to ensure that the provided memory will not be freed or reallocated. Note the following. 1. If nc_open_memio() is called with NC_WRITE, and NC_MEMIO_LOCK is not set, then the netcdf-c library will take control of the incoming memory. This means that the original memory block should not be freed but the block returned by nc_close_mem() must be freed. 2. If nc_open_memio() is called with NC_WRITE, and NC_MEMIO_LOCK is set, then modifications to the original memory may fail if the space available is insufficient. Documentation is provided in the file docs/inmemory.md. A test case is provided: nc_test/tst_inmemory.c driven by nc_test/run_inmemory.sh WARNING: changes were made to the dispatch table for the close entry. From int (*close)(int) to int (*close)(int,void*).
2018-02-26 12:45:31 +08:00
#ARGS=t
re e-support UBS-599337 re pull request https://github.com/Unidata/netcdf-c/pull/405 re pull request https://github.com/Unidata/netcdf-c/pull/446 Notes: 1. This branch is a cleanup of the magic.dmh branch. 2. magic.dmh was originally merged, but caused problems with parallel IO. It was re-issued as pull request https://github.com/Unidata/netcdf-c/pull/446. 3. This branch + pull request replace any previous pull requests and magic.dmh branch. Given an otherwise valid netCDF file that has a corrupted header, the netcdf library currently crashes. Instead, it should return NC_ENOTNC. Additionally, the NC_check_file_type code does not do the forward search required by hdf5 files. It currently only looks at file position 0 instead of 512, 1024, 2048,... Also, it turns out that the HDF4 magic number is assumed to always be at the beginning of the file (unlike HDF5). The change is localized to libdispatch/dfile.c See https://support.hdfgroup.org/release4/doc/DSpec_html/DS.pdf Also, it turns out that the code in NC_check_file_type is duplicated (mostly) in the function libsrc4/nc4file.c#nc_check_for_hdf. This branch does the following. 1. Make NC_check_file_type return NC_ENOTNC instead of crashing. 2. Remove nc_check_for_hdf and centralize all file format checking NC_check_file_type. 3. Add proper forward search for HDF5 files (but not HDF4 files) to look for the magic number at offsets of 0, 512, 1024... 4. Add test tst_hdf5_offset.sh. This tests that hdf5 files with an offset are properly recognized. It does so by prefixing a legal file with some number of zero bytes: 512, 1024, etc. 5. Off-topic: Added -N flag to ncdump to force a specific output dataset name.
2017-10-25 06:25:09 +08:00
merge-squash
2015-08-16 06:26:35 +08:00
#CMD=valgrind --leak-check=full
This completes (for now) the refactoring of libsrc4. The file docs/indexing.dox tries to provide design information for the refactoring. The primary change is to replace all walking of linked lists with the use of the NCindex data structure. Ncindex is a combination of a hash table (for name-based lookup) and a vector (for walking the elements in the index). Additionally, global vectors are added to NC_HDF5_FILE_INFO_T to support direct mapping of an e.g. dimid to the NC_DIM_INFO_T object. These global vectors exist for dimensions, types, and groups because they have globally unique id numbers. WARNING: 1. since libsrc4 and libsrchdf4 share code, there are also changes in libsrchdf4. 2. Any outstanding pull requests that change libsrc4 or libhdf4 are likely to cause conflicts with this code. 3. The original reason for doing this was for performance improvements, but as noted elsewhere, this may not be significant because the meta-data read performance apparently is being dominated by the hdf5 library because we do bulk meta-data reading rather than lazy reading.
2018-03-17 01:46:18 +08:00
#CMD=export NETCDF_LOG_LEVEL=5 ;gdb --args
Add provenance info for netcdf-4 files. This consists of a persistent attribute named _NCProperties plus two computed attributes _IsNetcdf4 and _SuperblockVersion. See the 'Provenance Attributes' section of docs/attribute_conventions.md for details.
2016-05-04 11:17:06 +08:00
CMD=gdb --args
Merge branch 'master' into cdf-5, in preparation for merging the CDF-5 functionality into the master branch. This will be the key new feature for netcdf 4.4.0.
2015-11-06 04:40:35 +08:00
re: esupport MQO-415619 and https://github.com/Unidata/netcdf-c/issues/708 Expand the NC_INMEMORY capabilities to support writing and accessing the final modified memory. Three new functions have been added: nc_open_memio, nc_create_mem, and nc_close_memio. The following new capabilities were added. 1. nc_open_memio() allows the NC_WRITE mode flag so a chunk of memory can be passed in and be modified 2. nc_create_mem() allows the NC_INMEMORY flag to be set to cause the created file to be kept in memory. 3. nc_close_mem() allows the final in-memory contents to be retrieved at the time the file is closed. 4. A special flag, NC_MEMIO_LOCK, is provided to ensure that the provided memory will not be freed or reallocated. Note the following. 1. If nc_open_memio() is called with NC_WRITE, and NC_MEMIO_LOCK is not set, then the netcdf-c library will take control of the incoming memory. This means that the original memory block should not be freed but the block returned by nc_close_mem() must be freed. 2. If nc_open_memio() is called with NC_WRITE, and NC_MEMIO_LOCK is set, then modifications to the original memory may fail if the space available is insufficient. Documentation is provided in the file docs/inmemory.md. A test case is provided: nc_test/tst_inmemory.c driven by nc_test/run_inmemory.sh WARNING: changes were made to the dispatch table for the close entry. From int (*close)(int) to int (*close)(int,void*).
2018-02-26 12:45:31 +08:00
#PAR=1
merge-squash
2015-08-16 06:26:35 +08:00
re: esupport MQO-415619 and https://github.com/Unidata/netcdf-c/issues/708 Expand the NC_INMEMORY capabilities to support writing and accessing the final modified memory. Three new functions have been added: nc_open_memio, nc_create_mem, and nc_close_memio. The following new capabilities were added. 1. nc_open_memio() allows the NC_WRITE mode flag so a chunk of memory can be passed in and be modified 2. nc_create_mem() allows the NC_INMEMORY flag to be set to cause the created file to be kept in memory. 3. nc_close_mem() allows the final in-memory contents to be retrieved at the time the file is closed. 4. A special flag, NC_MEMIO_LOCK, is provided to ensure that the provided memory will not be freed or reallocated. Note the following. 1. If nc_open_memio() is called with NC_WRITE, and NC_MEMIO_LOCK is not set, then the netcdf-c library will take control of the incoming memory. This means that the original memory block should not be freed but the block returned by nc_close_mem() must be freed. 2. If nc_open_memio() is called with NC_WRITE, and NC_MEMIO_LOCK is set, then modifications to the original memory may fail if the space available is insufficient. Documentation is provided in the file docs/inmemory.md. A test case is provided: nc_test/tst_inmemory.c driven by nc_test/run_inmemory.sh WARNING: changes were made to the dispatch table for the close entry. From int (*close)(int) to int (*close)(int,void*).
2018-02-26 12:45:31 +08:00
CFLAGS=-Wall -Wno-unused-variable -Wno-unused-function -g -O0 -I.. -I../include
diskless: make read and write loop to read/write whole file when persisting
2012-04-10 06:03:02 +08:00
Add provenance info for netcdf-4 files. This consists of a persistent attribute named _NCProperties plus two computed attributes _IsNetcdf4 and _SuperblockVersion. See the 'Provenance Attributes' section of docs/attribute_conventions.md for details.
2016-05-04 11:17:06 +08:00
ifdef PAR
CC=mpicc
#CC=/usr/local/bin/mpicc
LDFLAGS=../liblib/.libs/libnetcdf.a -L/usr/local/lib -lhdf5_hl -lhdf5 -lz -ldl -lcurl -lpnetcdf -lmpich -lm
else
CC=gcc
merge-squash
2015-08-16 06:26:35 +08:00
#LDFLAGS=../liblib/.libs/libnetcdf.a -L/usr/local/lib -lhdf5_hl -lhdf5 -lz -lm -lcurl
Add provenance info for netcdf-4 files. This consists of a persistent attribute named _NCProperties plus two computed attributes _IsNetcdf4 and _SuperblockVersion. See the 'Provenance Attributes' section of docs/attribute_conventions.md for details.
2016-05-04 11:17:06 +08:00
LDFLAGS=../liblib/.libs/libnetcdf.a -L/usr/local/lib -lhdf5_hl -lhdf5 -lz -ldl -lm -lcurl
endif
diskless: make read and write loop to read/write whole file when persisting
2012-04-10 06:03:02 +08:00
# cd .. ; ${MAKE} all
merge-squash
2015-08-16 06:26:35 +08:00
LLP=/usr/local/lib:${LD_LIBRARY_PATH}
Add provenance info for netcdf-4 files. This consists of a persistent attribute named _NCProperties plus two computed attributes _IsNetcdf4 and _SuperblockVersion. See the 'Provenance Attributes' section of docs/attribute_conventions.md for details.
2016-05-04 11:17:06 +08:00
all:: cmp
merge-squash
2015-08-16 06:26:35 +08:00
export LD_LIBRARY_PATH=${LLP}; export CFLAGS; export LDFLAGS; \
re e-support UBS-599337 re pull request https://github.com/Unidata/netcdf-c/pull/405 re pull request https://github.com/Unidata/netcdf-c/pull/446 Notes: 1. This branch is a cleanup of the magic.dmh branch. 2. magic.dmh was originally merged, but caused problems with parallel IO. It was re-issued as pull request https://github.com/Unidata/netcdf-c/pull/446. 3. This branch + pull request replace any previous pull requests and magic.dmh branch. Given an otherwise valid netCDF file that has a corrupted header, the netcdf library currently crashes. Instead, it should return NC_ENOTNC. Additionally, the NC_check_file_type code does not do the forward search required by hdf5 files. It currently only looks at file position 0 instead of 512, 1024, 2048,... Also, it turns out that the HDF4 magic number is assumed to always be at the beginning of the file (unlike HDF5). The change is localized to libdispatch/dfile.c See https://support.hdfgroup.org/release4/doc/DSpec_html/DS.pdf Also, it turns out that the code in NC_check_file_type is duplicated (mostly) in the function libsrc4/nc4file.c#nc_check_for_hdf. This branch does the following. 1. Make NC_check_file_type return NC_ENOTNC instead of crashing. 2. Remove nc_check_for_hdf and centralize all file format checking NC_check_file_type. 3. Add proper forward search for HDF5 files (but not HDF4 files) to look for the magic number at offsets of 0, 512, 1024... 4. Add test tst_hdf5_offset.sh. This tests that hdf5 files with an offset are properly recognized. It does so by prefixing a legal file with some number of zero bytes: 512, 1024, etc. 5. Off-topic: Added -N flag to ncdump to force a specific output dataset name.
2017-10-25 06:25:09 +08:00
${CMD} ./t ${ARGS}
merge-squash
2015-08-16 06:26:35 +08:00
Add provenance info for netcdf-4 files. This consists of a persistent attribute named _NCProperties plus two computed attributes _IsNetcdf4 and _SuperblockVersion. See the 'Provenance Attributes' section of docs/attribute_conventions.md for details.
2016-05-04 11:17:06 +08:00
cmp::
export LD_LIBRARY_PATH=${LLP}; export CFLAGS; export LDFLAGS; \
${CC} -o t ${CFLAGS} ${T}.c ${SRC} ${LDFLAGS}; \
merge-squash
2015-08-16 06:26:35 +08:00
cpp::
${CC} -E ${CFLAGS} ${T}.c > ${T}.txt
Reference in New Issue Copy Permalink