Code cleanup
Description:
Change variable casts to use H5_ASSIGN_OVERFLOW, or check for overflow
before the cast with H5_CHECK_OVERFLOW.
Platforms tested:
FreeBSD 4.5 (sleipnir)
Purpose:
eliminate warnings on win32
Description:
some conversion warnings were being issued
Solution:
added some type casts
Platforms tested:
w2000, linux
changed the HDmemset to include a cast in win32; this was just to eliminate a compiler
warning. probably the macro can also be used in unix
#ifdef WIN32
#define HDmemset(X,C,Z) memset((void*)(X),C,Z)
#else /* WIN32 */
#define HDmemset(X,C,Z) memset(X,C,Z)
#endif /* WIN32 */
the list of previous warnings was
D:\disk_w\hdf5\src\H5FDstdio.c(659) : warning C4244: 'initializing' : conversion from 'unsigned __int64 ' to 'unsigned int ', possible loss of data
D:\disk_w\hdf5\src\H5Fcontig.c(435) : warning C4244: '=' : conversion from 'unsigned __int64 ' to 'unsigned int ', possible loss of data
D:\disk_w\hdf5\src\H5Fcontig.c(497) : warning C4244: '=' : conversion from 'unsigned __int64 ' to 'unsigned int ', possible loss of data
D:\disk_w\hdf5\src\H5Fcontig.c(915) : warning C4244: '=' : conversion from 'unsigned __int64 ' to 'unsigned int ', possible loss of data
D:\disk_w\hdf5\src\H5Fcontig.c(982) : warning C4244: '=' : conversion from 'unsigned __int64 ' to 'unsigned int ', possible loss of data
D:\disk_w\hdf5\src\H5Shyper.c(912) : warning C4244: '=' : conversion from 'unsigned __int64 ' to 'unsigned int ', possible loss of data
D:\disk_w\hdf5\src\H5Shyper.c(995) : warning C4244: '=' : conversion from 'unsigned __int64 ' to 'unsigned int ', possible loss of data
D:\disk_w\hdf5\src\H5Shyper.c(1936) : warning C4244: '=' : conversion from 'unsigned __int64 ' to 'unsigned int ', possible loss of data
D:\disk_w\hdf5\src\H5Shyper.c(2019) : warning C4244: '=' : conversion from 'unsigned __int64 ' to 'unsigned int ', possible loss of data
D:\disk_w\hdf5\src\H5Shyper.c(2862) : warning C4244: '=' : conversion from 'unsigned __int64 ' to 'unsigned int ', possible loss of data
D:\disk_w\hdf5\src\H5Shyper.c(2864) : warning C4244: '=' : conversion from 'unsigned __int64 ' to 'unsigned int ', possible loss of data
D:\disk_w\hdf5\src\H5Shyper.c(2948) : warning C4244: '=' : conversion from 'unsigned __int64 ' to 'unsigned int ', possible loss of data
D:\disk_w\hdf5\src\H5Shyper.c(3690) : warning C4244: '=' : conversion from 'unsigned __int64 ' to 'unsigned int ', possible loss of data
D:\disk_w\hdf5\src\H5Shyper.c(3692) : warning C4244: '=' : conversion from 'unsigned __int64 ' to 'unsigned int ', possible loss of data
D:\disk_w\hdf5\src\H5Shyper.c(3776) : warning C4244: '=' : conversion from 'unsigned __int64 ' to 'unsigned int ', possible loss of data
D:\disk_w\hdf5\src\H5Shyper.c(5167) : warning C4244: '+=' : conversion from '__int64 ' to 'unsigned int ', possible loss of data
D:\disk_w\hdf5\src\H5Tvlen.c(371) : warning C4244: '=' : conversion from 'unsigned __int64 ' to 'unsigned int ', possible loss of data
Document Code improvement below:
Description:
Propagated the "fill time" property into the parallel chunk allocation
routine, allowing it to avoid writing fill values to each new chunk
allocated. This improves the performance of chunked datasets in parallel
I/O to be on par with contiguous datasets again (on modi4).
Code improvement.
Description:
Propagated the "fill time" property into the parallel chunk allocation
routine, allowing it to avoid writing fill values to each new chunk
allocated. This improves the performance of chunked datasets in parallel
I/O to be on par with contiguous datasets again (on modi4).
Platforms tested:
IRIX64 6.5 (modi4)
Document Bug fix/Code improvement below:
Description:
Currently, the chunk data allocation routine invoked to allocate space for
the entire dataset is inefficient. It writes out each chunk in the dataset,
whether it is already allocated or not. Additionally, this happens not
only when it is created, but also anytime it is opened for writing, or the
dataset is extended. Worse, there's too much parallel I/O syncronization,
which slows things down even more.
Solution:
Only attempt to write out chunks that don't already exist. Additionally,
share the I/O writing between all the nodes, instead of writing everything
with process 0. Then, only block with MPI_Barrier if chunks were actually
created.
Bug fix/Code improvement.
Description:
Currently, the chunk data allocation routine invoked to allocate space for
the entire dataset is inefficient. It writes out each chunk in the dataset,
whether it is already allocated or not. Additionally, this happens not
only when it is created, but also anytime it is opened for writing, or the
dataset is extended. Worse, there's too much parallel I/O syncronization,
which slows things down even more.
Solution:
Only attempt to write out chunks that don't already exist. Additionally,
share the I/O writing between all the nodes, instead of writing everything
with process 0. Then, only block with MPI_Barrier if chunks were actually
created.
Platforms tested:
IRIX64 6.5 (modi4)
Purpose:
New Feature - per library change
Description:
Added the new member function getMemberIndex to classes EnumType and
CompType to match the new C API H5Tget_member_index. Given the name
of a member of an enumeration or compound datatype, this new function
queries the index of the member.
Platforms:
SunOS 5.7 (arabica)
Linux 6.2 (eirene)
New Feature - per library change
Description:
Added new member function DataSet::fillMemBuf for the new
C API H5Dfill. Quincey's description of H5Dfill is as followed:
Added new H5Dfill() routine to fill the elements in a selection for a
memory buffer with a fill value. This is a user API wrapper around some
internal routines which were needed for the fill-value modifications
from Raymond as well as Pedro's code for reducing the size of a chunked
dataset.
Platforms:
SunOS 5.7 (arabica)
Linux 6.2 (eirene)
Code cleanup/Performance enhancement
Description:
The code to extend the file size in H5FD_mpio_flush is getting run even when
the file size doesn't change. Also, it's sort of sidestepping MPI-I/O when
extending the file, instead of using MPI-I/O features to set the file's size.
Solution:
Only extend the file's size when the allocated size has increased. Also
use MPI_File_set_size() to change the file's size.
Platforms tested:
IRIX64 6.5 (modi4)
Bug Fix
Description:
When H5Freopen is called, the file to reopen's 'intent' (read/write
permissions, etc) is not being copied to the new file's 'intent'
information.
Solution:
Copy it. :-)
Platforms tested:
IRIX64 6.5 (modi4)
Bug fix
Description:
Calling MPI_Get_count needs to be done with the same MPI type as was used
for the transfer and we are always using MPI_BYTE, even when a different
MPI type was used for the transfer.
Solution:
Only query MPI_Get_count with MPI_BYTE when we really used MPI_BYTE for the
transfer. Wait for later to query MPI_Get_count with other MPI types.
Platforms tested:
IRIX64 6.5 (modi4)
Performance enhancement
Description:
Doing an MPI_File_sync() just before a file is closed causing a large
performance loss.
Solution:
Add flag to MPI file driver to avoid performance the MPI_File_sync() when
the flag is set before a call to H5F_flush().
Platforms tested:
IRIX64 6.5 (modi4)
Bug fix
Description:
Was not able to handle data size (file size) larger than 32bits.
Was using long, which is only 4 bytes big in SP, thus overflowing
into negative when trying to address 2GB or larger.
Solution:
Changed those variables involved in file size/offset calculation
to type off_t. (If a certain system/compiler has off_t defined
as 4 bytes, it can't write to file size larger than 2GB anyway.)
Note that the lseek of SP with -D_LARGE_FILE still fails for
offset larger than 2GB (works for 2GB). That has to be fixed
soon.
Platforms tested:
burrwhite (linux 2.4) and modi4 parallel.
Bug Fix
Description:
Put quotes around some variables so that the shell script doesn't
complain to us about "too many arguements" for a test.
Platforms tested:
Linux
Bug Fix
Description:
Placed quotes around strings which could be empty so that the shell
script doesn't complain to us about "too many arguments for test"
problems.
Platforms tested:
Linux
New Feature
Description:
The VFL flush function is called immediately before a file is closed.
This can cause duplicate syncronization actions to occur, if the VFL
close function also performs them.
Solution:
Added 'closing' parameter to VFL 'flush' operation. This allows the VFL
flush function to bypass operations that will be duplicated within the VFL
close function.
Additionally, use the 'closing' parameter to bypass calls to MPI_File_sync()
when set. Since MPI_File_close() also syncronizes the file, this avoids
the terrible performance hit taken when calling MPI_File_sync() as the file
is closing.
Platforms tested:
IRIX64 6.5 (modi4)
Code cleanup
Description:
The parallel I/O file driver is optimized to only write metadata with one
process (and broadcast the results to the other processes). This is
currently enabled by a separate call to H5FD_mpio_tas_allsame() before
each metadata write to the file. This can easily lead to problems where
the prelude function call is omitted before the actual write code or, in
a threaded environment, lead to race condititions where the value set is
reset before being used.
Solution:
Since we only want to write metadata from one process, key off of the 'type'
parameter (which has information about whether the data being written it
metadata or raw data) to H5FD_mpio_write() as the method for determining
whether to only write from one process or not.
Platforms tested:
IRIX64 6.5 (modi4)
Bug Fix
Description:
Currently, only process 0 is writing attribute data to a file. This is
incorrect, because the raw data for attributes is cached in memory until
the object header is written and other processes are not able to read the
correct attribute information.
Solution:
Have all processes participate in writing the attribute data.
Platforms tested:
IRIX64 6.5 (modi4)
New Feature
Description:
Currently, only process 0 writes metadata to disk, leading to a potential
performance bottleneck as the other processors wait for it to catch up.
Solution:
Rotate the metadata responsibilities among all processes, speading out the
work.
Platforms tested:
IRIX64 6.5 (modi4)
Small code cleanup/improvement
Description:
Query the MPI rank only once and store it in the file structure for each
process.
Platforms tested:
IRIX64 6.5 (modi4)
Purpose:
Update h5dread/write_f parameters
Description:
fortran/h5d_FORTRAN.html
h5dread/write_f -- in the 'reference' versions, which yesterday's
revisions missed):
Added new overloading to dims parameter description.
Corrected prior dims type and noted that it is deprecated and
will be removed in 1.6.
fortran/h5a_FORTRAN.html
Updated "Last modified" tag to reflect yesterday's revisions.
Platforms tested:
IE 5
Name Change
Description:
Changed the macro's name from "MILLISECOND" to "MICROSECOND" since,
as Quincey pointed out, that's what it is ;).
Platforms tested:
Linux
Purpose:
Bugfix -- Closes#702
Description:
Commented out references to and descriptions of unimplemented
H5S functions. Particularly in the UG document, Dataspaces.html.
H5Sopen H5Slock H5Ssubspace
H5Scommit H5Sselect_op H5Ssubspace_name
H5Sis_subspace H5Sselect_order H5Ssubspace_location
Added explanatory "Note to developers" in comments near top of
Dataspaces.html.
Platforms tested:
IE 5
Bug fix, feature
Description:
Added code to time pure MPI_File_write_xxx and read_xxx routines.
Moved the setting of the timer_g to when timer is created (new)
and destroyed.
Platforms tested:
modi4pp
Modification
Description:
Put some of the variables into the global space so that random
functions can access them. (The output and timer variables which
point to the output file and timer resp.)
Placed the debug level 4 prints into the timer module so that it's
easier to automatically put timer start/stop printfs for new timers
which are created.
Added timers for MPI Time calls (called HDF5_MPI_WRITE and
HDF5_MPI_READ).
Platforms tested:
Linux
Purpose:
Update description of h5dread/write_f and h5aread/write_f dims parameter.
Description:
h5dread/write_f, h5aread/write_f
Added new overloading to dims parameter description.
Noted that prior dims type is deprecated and will be removed in 1.6.
Platforms tested:
IE 5
Purpose:
Maintenance
Description:
Added information about Parallel Fortran Support for HP-UX 11.00 SysV
and write/read overloaded subroutines (bug #670)
Purpose:
bug fix
Description:
In H5TB.c, previously use an unsigned variable to represent address difference that is maybe a negative value.
This will cause a potential bug in the future when this section of code is run.
Solution:
use H5F_address_cmp macro to compare two addresses.
Platforms tested:
linux 2.2.18
Bug Fix
Description:
There was a duplicate declaration of print_indent() in the header and
in the source files.
Solution:
Removed the public declaration and made the function static again.
Platforms tested:
Arabica
Purpose:
Bugfixes -- Fixes#657.
Addresses #720 (but does not complete it).
Description:
H5Pset_fapl_split: Added note re checking on use of this driver,
i.e., that H5Pget_driver returns H5FD_MULTI.
Fixes Bug #657.
Minor copy edits, including some addressed in Bug #720.
Platforms tested:
IE 5
