mirror/netcdf-c
2
0
Fork 0
mirror of https://github.com/Unidata/netcdf-c.git synced 2024-12-27 08:49:16 +08:00
Code Issues Packages Projects Releases Wiki Activity
793ecc8e60
netcdf-c/nc_test/run_inmemory.sh

49 lines
1.1 KiB
Bash
Raw Normal View History

re: esupport MQO-415619 and https://github.com/Unidata/netcdf-c/issues/708 Expand the NC_INMEMORY capabilities to support writing and accessing the final modified memory. Three new functions have been added: nc_open_memio, nc_create_mem, and nc_close_memio. The following new capabilities were added. 1. nc_open_memio() allows the NC_WRITE mode flag so a chunk of memory can be passed in and be modified 2. nc_create_mem() allows the NC_INMEMORY flag to be set to cause the created file to be kept in memory. 3. nc_close_mem() allows the final in-memory contents to be retrieved at the time the file is closed. 4. A special flag, NC_MEMIO_LOCK, is provided to ensure that the provided memory will not be freed or reallocated. Note the following. 1. If nc_open_memio() is called with NC_WRITE, and NC_MEMIO_LOCK is not set, then the netcdf-c library will take control of the incoming memory. This means that the original memory block should not be freed but the block returned by nc_close_mem() must be freed. 2. If nc_open_memio() is called with NC_WRITE, and NC_MEMIO_LOCK is set, then modifications to the original memory may fail if the space available is insufficient. Documentation is provided in the file docs/inmemory.md. A test case is provided: nc_test/tst_inmemory.c driven by nc_test/run_inmemory.sh WARNING: changes were made to the dispatch table for the close entry. From int (*close)(int) to int (*close)(int,void*).
2018-02-26 12:45:31 +08:00
#!/bin/sh
if test "x$srcdir" = x ; then srcdir=`pwd`; fi
. ../test_common.sh
set -e
# Get the target OS and CPU
CPU=`uname -p`
OS=`uname`
#Constants
FILE3=tst_inmemory3
CREATE3=tst_inmemory3_create
FILE4=tst_inmemory4
CREATE4=tst_inmemory4_create
Fix failings of nc_test/run_inmemory.sh re: issue https://github.com/Unidata/netcdf-c/issues/1233 Changes: 1. remove exit that was there for testing. 2. the program tst_open_mem must be netcdf-4 only. 3. fix some diff problems - Change dataset name for tst_inmemory4_create to tst_inmemory4 - Modify tst_inmemory.c to reorder the variables (somewhat major rewrite) Minor Unrelated Fixes: 1. fix comment problem in nc_provenance.h 2. Fix memory leak in tst_open_mem.c 3. fix ncdump/bindata.c to properly compile if netcdf4 is disabled. 4. minor changes to ncgen.l
2018-12-05 10:22:02 +08:00
# For tst_open_mem NETCDF4 only
Fix use of 'int' to represent 'hid_t' that caused HDF5 1.10 to fail. re: github issue https://github.com/Unidata/netcdf-fortran/issues/82 This was originally discovered in the Fortran tests, but is a problem in the C library. The problem only occurred when using HDF5-1.10.x. The reason it failed is that starting with 1.10, the hid_t type was changed from 32 bits to 64 bits. The function libsrc4/nc4memcb.c#NC4_image_init was using type int (doh!) to return the hdf fileid instead of hid_t type. This, of course, caused the id to be truncated and in turn later use of the id caused hdf5 to fail. Fix is trivial: replace int with hid_t. This also requires a related change in nc4mem.c. Also added the test case derived from the original Fortran code. You would think I would learn...
2018-05-31 04:47:37 +08:00
OMEMFILE=f03tst_open_mem.nc
re: esupport MQO-415619 and https://github.com/Unidata/netcdf-c/issues/708 Expand the NC_INMEMORY capabilities to support writing and accessing the final modified memory. Three new functions have been added: nc_open_memio, nc_create_mem, and nc_close_memio. The following new capabilities were added. 1. nc_open_memio() allows the NC_WRITE mode flag so a chunk of memory can be passed in and be modified 2. nc_create_mem() allows the NC_INMEMORY flag to be set to cause the created file to be kept in memory. 3. nc_close_mem() allows the final in-memory contents to be retrieved at the time the file is closed. 4. A special flag, NC_MEMIO_LOCK, is provided to ensure that the provided memory will not be freed or reallocated. Note the following. 1. If nc_open_memio() is called with NC_WRITE, and NC_MEMIO_LOCK is not set, then the netcdf-c library will take control of the incoming memory. This means that the original memory block should not be freed but the block returned by nc_close_mem() must be freed. 2. If nc_open_memio() is called with NC_WRITE, and NC_MEMIO_LOCK is set, then modifications to the original memory may fail if the space available is insufficient. Documentation is provided in the file docs/inmemory.md. A test case is provided: nc_test/tst_inmemory.c driven by nc_test/run_inmemory.sh WARNING: changes were made to the dispatch table for the close entry. From int (*close)(int) to int (*close)(int,void*).
2018-02-26 12:45:31 +08:00
echo ""
echo "*** Testing in-memory operations"
# Execute the core of the inmemory tests
Fix the NC_INMEMORY code to work in all cases with HDF5 1.10. re: github issue https://github.com/Unidata/netcdf-c/issues/1111 One of the less common use cases for the in-memory feature is apparently failing with HDF5-1.10.x. The fix is complicated and requires significant changes to libhdf5/nc4memcb.c. The current setup is detailed in the file docs/inmeminternal.dox. Additionally, it was discovered that the program nc_test/tst_inmemory.c, which is invoked by nc_test/run_inmemory.sh, actually was failing because of the above problem. But the failure is not detected since the script does not return non-zero value. Other Changes: 1. Fix nc_test_tst_inmemory to return errors correctly. 2. Make ncdap_tests/findtestserver.c and dap4_tests/findtestserver4.c be generated from ncdap_test/findtestserver.c.in. 3. Make LOG() print output to stderr instead of stdout to avoid contaminating e.g. ncdump output. 4. Modify the handling of NC_INMEMORY and NC_DISKLESS flags to properly handle that NC_DISKLESS => NC_INMEMORY. This affects a number of code pieces, especially memio.c.
2018-09-05 01:27:47 +08:00
${execdir}/tst_inmemory
Revert "Revert "Fix nczarr-experimental: improve build support, disengage hdf5 vs netcdf4 flags, and find AWS libraries""
2020-08-18 09:15:47 +08:00
if test "x$FEATURE_HDF5" = "xyes" ; then
Fix the NC_INMEMORY code to work in all cases with HDF5 1.10. re: github issue https://github.com/Unidata/netcdf-c/issues/1111 One of the less common use cases for the in-memory feature is apparently failing with HDF5-1.10.x. The fix is complicated and requires significant changes to libhdf5/nc4memcb.c. The current setup is detailed in the file docs/inmeminternal.dox. Additionally, it was discovered that the program nc_test/tst_inmemory.c, which is invoked by nc_test/run_inmemory.sh, actually was failing because of the above problem. But the failure is not detected since the script does not return non-zero value. Other Changes: 1. Fix nc_test_tst_inmemory to return errors correctly. 2. Make ncdap_tests/findtestserver.c and dap4_tests/findtestserver4.c be generated from ncdap_test/findtestserver.c.in. 3. Make LOG() print output to stderr instead of stdout to avoid contaminating e.g. ncdump output. 4. Modify the handling of NC_INMEMORY and NC_DISKLESS flags to properly handle that NC_DISKLESS => NC_INMEMORY. This affects a number of code pieces, especially memio.c.
2018-09-05 01:27:47 +08:00
${execdir}/tst_open_mem ${srcdir}/${OMEMFILE}
Fix failings of nc_test/run_inmemory.sh re: issue https://github.com/Unidata/netcdf-c/issues/1233 Changes: 1. remove exit that was there for testing. 2. the program tst_open_mem must be netcdf-4 only. 3. fix some diff problems - Change dataset name for tst_inmemory4_create to tst_inmemory4 - Modify tst_inmemory.c to reorder the variables (somewhat major rewrite) Minor Unrelated Fixes: 1. fix comment problem in nc_provenance.h 2. Fix memory leak in tst_open_mem.c 3. fix ncdump/bindata.c to properly compile if netcdf4 is disabled. 4. minor changes to ncgen.l
2018-12-05 10:22:02 +08:00
fi
re: esupport MQO-415619 and https://github.com/Unidata/netcdf-c/issues/708 Expand the NC_INMEMORY capabilities to support writing and accessing the final modified memory. Three new functions have been added: nc_open_memio, nc_create_mem, and nc_close_memio. The following new capabilities were added. 1. nc_open_memio() allows the NC_WRITE mode flag so a chunk of memory can be passed in and be modified 2. nc_create_mem() allows the NC_INMEMORY flag to be set to cause the created file to be kept in memory. 3. nc_close_mem() allows the final in-memory contents to be retrieved at the time the file is closed. 4. A special flag, NC_MEMIO_LOCK, is provided to ensure that the provided memory will not be freed or reallocated. Note the following. 1. If nc_open_memio() is called with NC_WRITE, and NC_MEMIO_LOCK is not set, then the netcdf-c library will take control of the incoming memory. This means that the original memory block should not be freed but the block returned by nc_close_mem() must be freed. 2. If nc_open_memio() is called with NC_WRITE, and NC_MEMIO_LOCK is set, then modifications to the original memory may fail if the space available is insufficient. Documentation is provided in the file docs/inmemory.md. A test case is provided: nc_test/tst_inmemory.c driven by nc_test/run_inmemory.sh WARNING: changes were made to the dispatch table for the close entry. From int (*close)(int) to int (*close)(int,void*).
2018-02-26 12:45:31 +08:00
Revert "Revert "Fix nczarr-experimental: improve build support, disengage hdf5 vs netcdf4 flags, and find AWS libraries""
2020-08-18 09:15:47 +08:00
echo "**** Test ncdump of the resulting inmemory data: netcdf-3"
re: esupport MQO-415619 and https://github.com/Unidata/netcdf-c/issues/708 Expand the NC_INMEMORY capabilities to support writing and accessing the final modified memory. Three new functions have been added: nc_open_memio, nc_create_mem, and nc_close_memio. The following new capabilities were added. 1. nc_open_memio() allows the NC_WRITE mode flag so a chunk of memory can be passed in and be modified 2. nc_create_mem() allows the NC_INMEMORY flag to be set to cause the created file to be kept in memory. 3. nc_close_mem() allows the final in-memory contents to be retrieved at the time the file is closed. 4. A special flag, NC_MEMIO_LOCK, is provided to ensure that the provided memory will not be freed or reallocated. Note the following. 1. If nc_open_memio() is called with NC_WRITE, and NC_MEMIO_LOCK is not set, then the netcdf-c library will take control of the incoming memory. This means that the original memory block should not be freed but the block returned by nc_close_mem() must be freed. 2. If nc_open_memio() is called with NC_WRITE, and NC_MEMIO_LOCK is set, then modifications to the original memory may fail if the space available is insufficient. Documentation is provided in the file docs/inmemory.md. A test case is provided: nc_test/tst_inmemory.c driven by nc_test/run_inmemory.sh WARNING: changes were made to the dispatch table for the close entry. From int (*close)(int) to int (*close)(int,void*).
2018-02-26 12:45:31 +08:00
${NCDUMP} -n "${FILE3}" ${FILE3}.nc > ${FILE3}.cdl
${NCDUMP} -n "${FILE3}" ${CREATE3}.nc > ${CREATE3}.cdl
Fix the NC_INMEMORY code to work in all cases with HDF5 1.10. re: github issue https://github.com/Unidata/netcdf-c/issues/1111 One of the less common use cases for the in-memory feature is apparently failing with HDF5-1.10.x. The fix is complicated and requires significant changes to libhdf5/nc4memcb.c. The current setup is detailed in the file docs/inmeminternal.dox. Additionally, it was discovered that the program nc_test/tst_inmemory.c, which is invoked by nc_test/run_inmemory.sh, actually was failing because of the above problem. But the failure is not detected since the script does not return non-zero value. Other Changes: 1. Fix nc_test_tst_inmemory to return errors correctly. 2. Make ncdap_tests/findtestserver.c and dap4_tests/findtestserver4.c be generated from ncdap_test/findtestserver.c.in. 3. Make LOG() print output to stderr instead of stdout to avoid contaminating e.g. ncdump output. 4. Modify the handling of NC_INMEMORY and NC_DISKLESS flags to properly handle that NC_DISKLESS => NC_INMEMORY. This affects a number of code pieces, especially memio.c.
2018-09-05 01:27:47 +08:00
diff -wb ${FILE3}.cdl ${CREATE3}.cdl
re: esupport MQO-415619 and https://github.com/Unidata/netcdf-c/issues/708 Expand the NC_INMEMORY capabilities to support writing and accessing the final modified memory. Three new functions have been added: nc_open_memio, nc_create_mem, and nc_close_memio. The following new capabilities were added. 1. nc_open_memio() allows the NC_WRITE mode flag so a chunk of memory can be passed in and be modified 2. nc_create_mem() allows the NC_INMEMORY flag to be set to cause the created file to be kept in memory. 3. nc_close_mem() allows the final in-memory contents to be retrieved at the time the file is closed. 4. A special flag, NC_MEMIO_LOCK, is provided to ensure that the provided memory will not be freed or reallocated. Note the following. 1. If nc_open_memio() is called with NC_WRITE, and NC_MEMIO_LOCK is not set, then the netcdf-c library will take control of the incoming memory. This means that the original memory block should not be freed but the block returned by nc_close_mem() must be freed. 2. If nc_open_memio() is called with NC_WRITE, and NC_MEMIO_LOCK is set, then modifications to the original memory may fail if the space available is insufficient. Documentation is provided in the file docs/inmemory.md. A test case is provided: nc_test/tst_inmemory.c driven by nc_test/run_inmemory.sh WARNING: changes were made to the dispatch table for the close entry. From int (*close)(int) to int (*close)(int,void*).
2018-02-26 12:45:31 +08:00
Revert "Revert "Fix nczarr-experimental: improve build support, disengage hdf5 vs netcdf4 flags, and find AWS libraries""
2020-08-18 09:15:47 +08:00
if test "x$FEATURE_HDF5" = "xyes" ; then
echo "**** Test ncdump of the resulting inmemory data: netcdf-4"
re: esupport MQO-415619 and https://github.com/Unidata/netcdf-c/issues/708 Expand the NC_INMEMORY capabilities to support writing and accessing the final modified memory. Three new functions have been added: nc_open_memio, nc_create_mem, and nc_close_memio. The following new capabilities were added. 1. nc_open_memio() allows the NC_WRITE mode flag so a chunk of memory can be passed in and be modified 2. nc_create_mem() allows the NC_INMEMORY flag to be set to cause the created file to be kept in memory. 3. nc_close_mem() allows the final in-memory contents to be retrieved at the time the file is closed. 4. A special flag, NC_MEMIO_LOCK, is provided to ensure that the provided memory will not be freed or reallocated. Note the following. 1. If nc_open_memio() is called with NC_WRITE, and NC_MEMIO_LOCK is not set, then the netcdf-c library will take control of the incoming memory. This means that the original memory block should not be freed but the block returned by nc_close_mem() must be freed. 2. If nc_open_memio() is called with NC_WRITE, and NC_MEMIO_LOCK is set, then modifications to the original memory may fail if the space available is insufficient. Documentation is provided in the file docs/inmemory.md. A test case is provided: nc_test/tst_inmemory.c driven by nc_test/run_inmemory.sh WARNING: changes were made to the dispatch table for the close entry. From int (*close)(int) to int (*close)(int,void*).
2018-02-26 12:45:31 +08:00
${NCDUMP} ${FILE4}.nc > ${FILE4}.cdl
Fix failings of nc_test/run_inmemory.sh re: issue https://github.com/Unidata/netcdf-c/issues/1233 Changes: 1. remove exit that was there for testing. 2. the program tst_open_mem must be netcdf-4 only. 3. fix some diff problems - Change dataset name for tst_inmemory4_create to tst_inmemory4 - Modify tst_inmemory.c to reorder the variables (somewhat major rewrite) Minor Unrelated Fixes: 1. fix comment problem in nc_provenance.h 2. Fix memory leak in tst_open_mem.c 3. fix ncdump/bindata.c to properly compile if netcdf4 is disabled. 4. minor changes to ncgen.l
2018-12-05 10:22:02 +08:00
${NCDUMP} -n ${FILE4} ${CREATE4}.nc > ${CREATE4}.cdl
Fix the NC_INMEMORY code to work in all cases with HDF5 1.10. re: github issue https://github.com/Unidata/netcdf-c/issues/1111 One of the less common use cases for the in-memory feature is apparently failing with HDF5-1.10.x. The fix is complicated and requires significant changes to libhdf5/nc4memcb.c. The current setup is detailed in the file docs/inmeminternal.dox. Additionally, it was discovered that the program nc_test/tst_inmemory.c, which is invoked by nc_test/run_inmemory.sh, actually was failing because of the above problem. But the failure is not detected since the script does not return non-zero value. Other Changes: 1. Fix nc_test_tst_inmemory to return errors correctly. 2. Make ncdap_tests/findtestserver.c and dap4_tests/findtestserver4.c be generated from ncdap_test/findtestserver.c.in. 3. Make LOG() print output to stderr instead of stdout to avoid contaminating e.g. ncdump output. 4. Modify the handling of NC_INMEMORY and NC_DISKLESS flags to properly handle that NC_DISKLESS => NC_INMEMORY. This affects a number of code pieces, especially memio.c.
2018-09-05 01:27:47 +08:00
diff -wb ${FILE4}.cdl ${CREATE4}.cdl
Fix failings of nc_test/run_inmemory.sh re: issue https://github.com/Unidata/netcdf-c/issues/1233 Changes: 1. remove exit that was there for testing. 2. the program tst_open_mem must be netcdf-4 only. 3. fix some diff problems - Change dataset name for tst_inmemory4_create to tst_inmemory4 - Modify tst_inmemory.c to reorder the variables (somewhat major rewrite) Minor Unrelated Fixes: 1. fix comment problem in nc_provenance.h 2. Fix memory leak in tst_open_mem.c 3. fix ncdump/bindata.c to properly compile if netcdf4 is disabled. 4. minor changes to ncgen.l
2018-12-05 10:22:02 +08:00
fi
re: esupport MQO-415619 and https://github.com/Unidata/netcdf-c/issues/708 Expand the NC_INMEMORY capabilities to support writing and accessing the final modified memory. Three new functions have been added: nc_open_memio, nc_create_mem, and nc_close_memio. The following new capabilities were added. 1. nc_open_memio() allows the NC_WRITE mode flag so a chunk of memory can be passed in and be modified 2. nc_create_mem() allows the NC_INMEMORY flag to be set to cause the created file to be kept in memory. 3. nc_close_mem() allows the final in-memory contents to be retrieved at the time the file is closed. 4. A special flag, NC_MEMIO_LOCK, is provided to ensure that the provided memory will not be freed or reallocated. Note the following. 1. If nc_open_memio() is called with NC_WRITE, and NC_MEMIO_LOCK is not set, then the netcdf-c library will take control of the incoming memory. This means that the original memory block should not be freed but the block returned by nc_close_mem() must be freed. 2. If nc_open_memio() is called with NC_WRITE, and NC_MEMIO_LOCK is set, then modifications to the original memory may fail if the space available is insufficient. Documentation is provided in the file docs/inmemory.md. A test case is provided: nc_test/tst_inmemory.c driven by nc_test/run_inmemory.sh WARNING: changes were made to the dispatch table for the close entry. From int (*close)(int) to int (*close)(int,void*).
2018-02-26 12:45:31 +08:00
# cleanup
rm -f ${FILE3}.nc ${FILE4}.nc ${CREATE3}.nc ${CREATE4}.nc
rm -f ${FILE3}.cdl ${FILE4}.cdl ${CREATE3}.cdl ${CREATE4}.cdl
echo "PASS: all inmemory tests"
exit 0
Reference in New Issue Copy Permalink