Removal of bad code
Description:
Albert came up with a better way of determining which performance
tests to run. I removed the broken way I did them...
Platforms tested:
Linux-pp
Purpose:
Bug Fixing
Description:
In H5FD_family_write and H5FD_family_read, size_t is checked against
hsize_t for overflow, which fails on IA32 architecture machine supporting
large files.
Solution:
Use temporary variable which won't pass the limit of size_t.
Platforms tested:
Linux 2.4(platinum) and IRIX64 6.5(modi4)
bug fix
Description:
pio_info_g has been moved to the test/libh5test.a but the
pio_* source files still have the old definition and declaration.
Some compilers hated the duplcation.
Solution:
Replaced the duplication with #include h5test.h and added test/ to
the include search path too.
Platforms tested:
eirene, modi4.
Code cleanup
Description:
Broke the FUNC_ENTER macro into several macros, with more specialized
uses (which followup mail will describe). This was designed to move
most/all of the checks which could be done at compile time to that point,
instead of needlessly performing them (over & over :-) at run-time.
This reduces the library's size (and thus staticly linked binaries) and
has a minor speedup effect also.
Platforms tested:
IRIX64 6.5 (modi4) with parallel & FORTRAN enabled, and additional testing
on FreeBSD and Solaris immediately after the checkin.
Update
Description:
Only print out the parameters for a run if we're in debug mode one
and the 0'th process or if we're in debug mode > 0.
Platforms tested:
Linux
Feature Addition
Description:
Added feature which prints out the parameters and the MPI_Info object
for a run for each process.
Added some functions for the GPFS system. They are ifdef'ed out right
now (well, they need to ahve the non-existant H5_HAVE_GPFS macro set)
and aren't in use just yet. But the stub functions are there.
Platforms tested:
Linux
Code cleanup.
Description:
Took Robb's recent ideas for improving the FUNC_ENTER/FUNC_LEAVE macros
equivalents in the SAF library and adapted them to our library. I added
an additional macro which is equivalent to FUNC_ENTER:
FUNC_ENTER_NOINIT - Has the API tracing code, etc. from FUNC_ENTER but
none of the library or interface initialization code. This is to
be used _only_ for static functions and those which explicitly
cannot have the library or interface initialization code enabled
(like the API termination routines, etc.).
This allowed many more of the functions in the library [but not all yet :-(]
to be wrapped with FUNC_ENTER[_NOINIT]/FUNC_LEAVE pairs.
It also reduced the size of the library and executables (by cutting out a
bunch of code which was never executed), I'll e-mail the exact results when
I've finished editing it.
Platforms tested:
IRIX64 6.5 (modi4)
Folded the changes made in v1.4 into v1.5.
print library version information in help page too.
Specify the nofill feature supported in v1.5 only.
Print the values of the KB, MB and GB in case a user wants to see them.
Setup a macro, H5_HAVE_NOFILL, to indicate if Dataset no fill feature
is supported. If not, --no-fill is an invalid option.
Platforms tested:
modi4
Purpose:
update c++ test project to avoid linking errors
Description:
since gethostname etc. window-version functions may not be linked without modifying project setting in link part; we have to modify project settings for every project affected. dsets_cpp and dsets_cppdll were ignored last time. Now update it.
Solution:
Platforms tested:
windows 2000
Update
Description:
Updating the source for the PIO performance tests with the current
code that's in the V1.4 branch. It includes the newest command-line
parameters.
Platforms tested:
Linux-pp
Purpose:
update, remove hdf4-related stuff.
Description:
hdf4 related tools have been moved out of HDF5 CVS tree, The install doc should reflect this.
Solution:
Platforms tested:
Purpose:
eliminating the compiler warnings in Windows
Solution:
I am eliminating the compiler warnings in Windows.
the last 2 were:
1)
if((oid_list = H5MM_malloc(oid_count*sizeof(hid_t)))==NULL)
HGOTO_ERROR(H5E_RESOURCE, H5E_NOSPACE, NULL,
"memory allocation failed");
D:\disk_w\hdf5\src\H5F.c(2590) : warning C4047: '=' : 'int ' differs in levels
of indirection from 'void *'
on the HGOTO_ERROR call , I replaced the NULL with FAIL
2)
*((float*)d) = *((double*)s);
D:\disk_w\hdf5\src\H5Tconv.c(6426) : warning C4244: '=' : conversion from 'double ' to 'float ', possible loss of data
I added the type cast
*((float*)d) = (float) *((double*)s);
we have now 0 errors 0 warnings on Windows !
Platforms tested:
w2000, linux
Code cleanup
Description:
Guard against getting into metadata broadcast in write routine and clean
up some error handling.
Platforms tested:
IRIX64 6.5 (modi4)
Feature
Description:
moved the routines of setting up and dumping MPI-info object to
test library so that it is avaiable for all tests too.
Platforms tested:
modi4(pp), eirene (serial)
Code cleanup
Description:
Change variable casts to use H5_ASSIGN_OVERFLOW, or check for overflow
before the cast with H5_CHECK_OVERFLOW.
Platforms tested:
FreeBSD 4.5 (sleipnir)
Purpose:
eliminate warnings on win32
Description:
some conversion warnings were being issued
Solution:
added some type casts
Platforms tested:
w2000, linux
changed the HDmemset to include a cast in win32; this was just to eliminate a compiler
warning. probably the macro can also be used in unix
#ifdef WIN32
#define HDmemset(X,C,Z) memset((void*)(X),C,Z)
#else /* WIN32 */
#define HDmemset(X,C,Z) memset(X,C,Z)
#endif /* WIN32 */
the list of previous warnings was
D:\disk_w\hdf5\src\H5FDstdio.c(659) : warning C4244: 'initializing' : conversion from 'unsigned __int64 ' to 'unsigned int ', possible loss of data
D:\disk_w\hdf5\src\H5Fcontig.c(435) : warning C4244: '=' : conversion from 'unsigned __int64 ' to 'unsigned int ', possible loss of data
D:\disk_w\hdf5\src\H5Fcontig.c(497) : warning C4244: '=' : conversion from 'unsigned __int64 ' to 'unsigned int ', possible loss of data
D:\disk_w\hdf5\src\H5Fcontig.c(915) : warning C4244: '=' : conversion from 'unsigned __int64 ' to 'unsigned int ', possible loss of data
D:\disk_w\hdf5\src\H5Fcontig.c(982) : warning C4244: '=' : conversion from 'unsigned __int64 ' to 'unsigned int ', possible loss of data
D:\disk_w\hdf5\src\H5Shyper.c(912) : warning C4244: '=' : conversion from 'unsigned __int64 ' to 'unsigned int ', possible loss of data
D:\disk_w\hdf5\src\H5Shyper.c(995) : warning C4244: '=' : conversion from 'unsigned __int64 ' to 'unsigned int ', possible loss of data
D:\disk_w\hdf5\src\H5Shyper.c(1936) : warning C4244: '=' : conversion from 'unsigned __int64 ' to 'unsigned int ', possible loss of data
D:\disk_w\hdf5\src\H5Shyper.c(2019) : warning C4244: '=' : conversion from 'unsigned __int64 ' to 'unsigned int ', possible loss of data
D:\disk_w\hdf5\src\H5Shyper.c(2862) : warning C4244: '=' : conversion from 'unsigned __int64 ' to 'unsigned int ', possible loss of data
D:\disk_w\hdf5\src\H5Shyper.c(2864) : warning C4244: '=' : conversion from 'unsigned __int64 ' to 'unsigned int ', possible loss of data
D:\disk_w\hdf5\src\H5Shyper.c(2948) : warning C4244: '=' : conversion from 'unsigned __int64 ' to 'unsigned int ', possible loss of data
D:\disk_w\hdf5\src\H5Shyper.c(3690) : warning C4244: '=' : conversion from 'unsigned __int64 ' to 'unsigned int ', possible loss of data
D:\disk_w\hdf5\src\H5Shyper.c(3692) : warning C4244: '=' : conversion from 'unsigned __int64 ' to 'unsigned int ', possible loss of data
D:\disk_w\hdf5\src\H5Shyper.c(3776) : warning C4244: '=' : conversion from 'unsigned __int64 ' to 'unsigned int ', possible loss of data
D:\disk_w\hdf5\src\H5Shyper.c(5167) : warning C4244: '+=' : conversion from '__int64 ' to 'unsigned int ', possible loss of data
D:\disk_w\hdf5\src\H5Tvlen.c(371) : warning C4244: '=' : conversion from 'unsigned __int64 ' to 'unsigned int ', possible loss of data
Document Code improvement below:
Description:
Propagated the "fill time" property into the parallel chunk allocation
routine, allowing it to avoid writing fill values to each new chunk
allocated. This improves the performance of chunked datasets in parallel
I/O to be on par with contiguous datasets again (on modi4).
Code improvement.
Description:
Propagated the "fill time" property into the parallel chunk allocation
routine, allowing it to avoid writing fill values to each new chunk
allocated. This improves the performance of chunked datasets in parallel
I/O to be on par with contiguous datasets again (on modi4).
Platforms tested:
IRIX64 6.5 (modi4)
Document Bug fix/Code improvement below:
Description:
Currently, the chunk data allocation routine invoked to allocate space for
the entire dataset is inefficient. It writes out each chunk in the dataset,
whether it is already allocated or not. Additionally, this happens not
only when it is created, but also anytime it is opened for writing, or the
dataset is extended. Worse, there's too much parallel I/O syncronization,
which slows things down even more.
Solution:
Only attempt to write out chunks that don't already exist. Additionally,
share the I/O writing between all the nodes, instead of writing everything
with process 0. Then, only block with MPI_Barrier if chunks were actually
created.
Bug fix/Code improvement.
Description:
Currently, the chunk data allocation routine invoked to allocate space for
the entire dataset is inefficient. It writes out each chunk in the dataset,
whether it is already allocated or not. Additionally, this happens not
only when it is created, but also anytime it is opened for writing, or the
dataset is extended. Worse, there's too much parallel I/O syncronization,
which slows things down even more.
Solution:
Only attempt to write out chunks that don't already exist. Additionally,
share the I/O writing between all the nodes, instead of writing everything
with process 0. Then, only block with MPI_Barrier if chunks were actually
created.
Platforms tested:
IRIX64 6.5 (modi4)
Purpose:
New Feature - per library change
Description:
Added the new member function getMemberIndex to classes EnumType and
CompType to match the new C API H5Tget_member_index. Given the name
of a member of an enumeration or compound datatype, this new function
queries the index of the member.
Platforms:
SunOS 5.7 (arabica)
Linux 6.2 (eirene)
New Feature - per library change
Description:
Added new member function DataSet::fillMemBuf for the new
C API H5Dfill. Quincey's description of H5Dfill is as followed:
Added new H5Dfill() routine to fill the elements in a selection for a
memory buffer with a fill value. This is a user API wrapper around some
internal routines which were needed for the fill-value modifications
from Raymond as well as Pedro's code for reducing the size of a chunked
dataset.
Platforms:
SunOS 5.7 (arabica)
Linux 6.2 (eirene)
Code cleanup/Performance enhancement
Description:
The code to extend the file size in H5FD_mpio_flush is getting run even when
the file size doesn't change. Also, it's sort of sidestepping MPI-I/O when
extending the file, instead of using MPI-I/O features to set the file's size.
Solution:
Only extend the file's size when the allocated size has increased. Also
use MPI_File_set_size() to change the file's size.
Platforms tested:
IRIX64 6.5 (modi4)
Bug Fix
Description:
When H5Freopen is called, the file to reopen's 'intent' (read/write
permissions, etc) is not being copied to the new file's 'intent'
information.
Solution:
Copy it. :-)
Platforms tested:
IRIX64 6.5 (modi4)
Bug fix
Description:
Calling MPI_Get_count needs to be done with the same MPI type as was used
for the transfer and we are always using MPI_BYTE, even when a different
MPI type was used for the transfer.
Solution:
Only query MPI_Get_count with MPI_BYTE when we really used MPI_BYTE for the
transfer. Wait for later to query MPI_Get_count with other MPI types.
Platforms tested:
IRIX64 6.5 (modi4)
Performance enhancement
Description:
Doing an MPI_File_sync() just before a file is closed causing a large
performance loss.
Solution:
Add flag to MPI file driver to avoid performance the MPI_File_sync() when
the flag is set before a call to H5F_flush().
Platforms tested:
IRIX64 6.5 (modi4)
Bug fix
Description:
Was not able to handle data size (file size) larger than 32bits.
Was using long, which is only 4 bytes big in SP, thus overflowing
into negative when trying to address 2GB or larger.
Solution:
Changed those variables involved in file size/offset calculation
to type off_t. (If a certain system/compiler has off_t defined
as 4 bytes, it can't write to file size larger than 2GB anyway.)
Note that the lseek of SP with -D_LARGE_FILE still fails for
offset larger than 2GB (works for 2GB). That has to be fixed
soon.
Platforms tested:
burrwhite (linux 2.4) and modi4 parallel.
