Commit Graph

387 Commits

Author SHA1 Message Date
Scot Breitenfeld
5773fd34bc [svn-r14902] Merged fortran_1_8 branch changes r14505:14901 into the trunk. New fortran wrappers added. 2008-04-30 14:23:26 -05:00
Pedro Vicente Nunes
6b41367ac6 [svn-r14892] RFC display compression ratio in h5dump
Here's the current behavior of h5dump regarding the printing of the dataset creation property list

For example

./h5dump -H -p -d filters

HDF5 "tfilters.h5" {
DATASET "deflate" {
   DATATYPE  H5T_STD_I32LE
   DATASPACE  SIMPLE { ( 20, 10 ) / ( 20, 10 ) }
   STORAGE_LAYOUT {
      CHUNKED ( 10, 5 )
      SIZE 385 
    }
   FILTERS {
      COMPRESSION DEFLATE { LEVEL 9 }
   }
   FILLVALUE {
      FILL_TIME H5D_FILL_TIME_IFSET
      VALUE  0
   }
   ALLOCATION_TIME {
      H5D_ALLOC_TIME_INCR
   }
}
}


The proposed behavior is to add this information after SIZE

SIZE 385 (51.9%COMPRESSION)

That percentage is obtained trough

Per = (b-a) / a

Where a = theoretical size obtained by multiplying datum size times number of elements
b =  size obtained with H5Dget_storage_size

The final print would look like

HDF5 "tfilters.h5" {
DATASET "deflate" {
   DATATYPE  H5T_STD_I32LE
   DATASPACE  SIMPLE { ( 20, 10 ) / ( 20, 10 ) }
   STORAGE_LAYOUT {
      CHUNKED ( 10, 5 )
      SIZE 385 (51.9%COMPRESSION)
    }
   FILTERS {
      COMPRESSION DEFLATE { LEVEL 9 }
   }
   FILLVALUE {
      FILL_TIME H5D_FILL_TIME_IFSET
      VALUE  0
   }
   ALLOCATION_TIME {
      H5D_ALLOC_TIME_INCR
   }
}
}

tested: windows, linux, solaris
2008-04-29 13:02:05 -05:00
Pedro Vicente Nunes
c83dbd9c18 [svn-r14792] bug fix: #1106 h5dump -b will dump ASCII values for datasets after the first one. One variable that controls the binary output was incorrectly reset to zero after a binary output was done a first time. The effect was that on cases of several datasets, the ones after the first were not binary written. Eliminated the resetting of that variable and tested a file with several datasets. Modified the test file so that it is easier to test with the tool binread, that reads the binary output of h5dump.
tested: windows, linux
2008-04-03 16:07:27 -05:00
Quincey Koziol
66fa79b423 [svn-r14690] Description:
Handle comparing datasets & attributes w/variable-length strings properly.

Tested on:
    Linux/64 2.6.9 (chicago)
2008-02-28 09:49:48 -05:00
Pedro Vicente Nunes
8005df6417 [svn-r14437] bug fix for h5dump subsetting (deal with blocks)and new test runs
add a check for block overlap after the command line parsing

* Algorithm
 *
 * In a inner loop, the parameters from SSET are translated into temporary 
 * variables so that 1 row is printed at a time (getting the coordinate indices 
 * at each row).
 * We define the stride, count and block to be 1 in the row dimension to achieve 
 * this and advance until all points are printed. 
 * An outer loop for cases where dimensionality is greater than 2D is made. 
 * In each iteration, the 2D block is displayed in the inner loop. The remaining 
 * slower dimensions above the first 2 are incremented one at a time in the outer loop
 *
 * The element position is obtained from the matrix according to:
 *       Given an index I(z,y,x) its position from the beginning of an array 
 *       of sizes A(size_z, size_y,size_x) is given by
 *       Position of I(z,y,x) = index_z * size_y * size_x 
 *                             + index_y * size_x
 *                             + index_x
 * 

tested: windows, linux
2008-01-16 11:12:08 -05:00
Pedro Vicente Nunes
be4ca1f67e [svn-r14414] revert the subsetting calls of h5dump to 1 size block, still a bug with size > 1
tested: linux
2008-01-14 19:18:29 -05:00
Pedro Vicente Nunes
a6605796b3 [svn-r14412] more progress on the block hyperslab bug, clean code
modified the 3D test case for subsetting with block and stride factors

tested: windows, linux
2008-01-14 16:57:19 -05:00
Pedro Vicente Nunes
1c3e5859d1 [svn-r14411] more progress on the block hyperslab bug, clean code
modified the 2D test case for subsetting with block and stride factors


tested: windows, linux
2008-01-14 16:29:09 -05:00
Pedro Vicente Nunes
b8b215b34a [svn-r14409] more progress on the block hyperslab bug, clean code
modified the 1D test case for subsetting with block and stride factors

tested: windows, linux
2008-01-14 16:08:55 -05:00
Pedro Vicente Nunes
7e80ad2af2 [svn-r14384] new file with output regarding the new binary file 2008-01-08 16:09:19 -05:00
Pedro Vicente Nunes
91b0b68167 [svn-r14383] new binary file: with double dataset with NaN values 2008-01-08 16:01:51 -05:00
Quincey Koziol
6878261a43 [svn-r14353] Description:
- Add hash value for skip list string types, to reduce # of string
        comparisons.
    - Fixed bug with metadata/small data block aggregator adding size == 0
        block into file free space list.
    - Refactored metadata/small data block aggregator code into single set of
        common routines.
    - Changed block aggregator code to be smarter about releasing space in the
        'other' block when the 'other' block has aggregated enough data.

Tested on:
        FreeBSD/32 6.2 (duty) in debug mode
        FreeBSD/64 6.2 (liberty) w/C++ & FORTRAN, in debug mode
        Linux/32 2.6 (kagiso) w/PGI compilers, w/C++ & FORTRAN, w/threadsafe,
                                in debug mode
        Linux/64-amd64 2.6 (smirom) w/default API=1.6.x, w/C++ & FORTRAN,
                                in production mode
        Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN,
                                in production mode
        Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN,
                                w/szip filter, in production mode
        Mac OS X/32 10.4.10 (amazon) in debug mode
        Linux/64-ia64 2.4 (tg-login3) w/parallel, w/FORTRAN, in production mode
2007-12-20 15:36:08 -05:00
Pedro Vicente Nunes
d3c3b4138a [svn-r14339] bug fix: aligned a line in usage that was not aligned
tested: linux
2007-12-12 13:27:26 -05:00
Pedro Vicente Nunes
c15315539b [svn-r14324] bug fix: #786 (E3) h5dump refers to the user block as "boot block"
replaced usage message  with "super block"

tested: linux
2007-12-06 11:25:51 -05:00
Quincey Koziol
2f36ea99d4 [svn-r14284] Description:
Add H5Lvisit_by_name() API routine to library.

	Eliminated all (five!) other group traversal routines and changed them
all to use the new API routine.

	Cleaned up output of h5ls & h5stat:
		- Issue error when requesting recursive traversal of a file
			with the "group info" flag, but no group given
		- Print info about root group in all(?) appropriate situations
		- Don't print "verbose" information about root group until the
			root group is in the list of objects to display
	(mostly because h5ls & h5stat had a different twist on traversing the
groups in a file that the other utilities)

Tested on:
        FreeBSD/32 6.2 (duty) in debug mode
        FreeBSD/64 6.2 (liberty) w/C++ & FORTRAN, in debug mode
        Linux/32 2.6 (kagiso) w/PGI compilers, w/C++ & FORTRAN, w/threadsafe,
                                in debug mode
        Linux/64-amd64 2.6 (smirom) w/default API=1.6.x, w/C++ & FORTRAN,
                                in production mode
        Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN,
                                in production mode
        Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN,
                                w/szip filter, in production mode
        Mac OS X/32 10.4.10 (amazon) in debug mode
        Linux/64-ia64 2.4 (tg-login3) w/parallel, w/FORTRAN, in production mode
2007-11-24 11:49:36 -05:00
Pedro Vicente Nunes
39f23efa17 [svn-r14273] new feature: added some long options to the tests
tested: linux
2007-11-20 11:07:40 -05:00
Pedro Vicente Nunes
78898b38c0 [svn-r14267] enhancement: formatted h5diff usage
tested: linux
2007-11-19 14:38:34 -05:00
Pedro Vicente Nunes
f0e99ad5d0 [svn-r14266] enhancement: format usage message
tested: linux
2007-11-19 13:46:59 -05:00
Pedro Vicente Nunes
d3507c4a5a [svn-r14264] new features: 1) new usage for h5diff " A la" h5dump format
2) new usage for h5repack and new command line parsing using the tools library parsing code

tested: windows, linux, solaris
2007-11-16 15:15:19 -05:00
Pedro Vicente Nunes
5c16c09f91 [svn-r14259] New feature: removed one test and file from the h5diff shell script that prints an error
message that has an absolute path (invalid option), thus not suitable for the multi platform test
2007-11-13 16:49:07 -05:00
Pedro Vicente Nunes
1a3c5f6aeb [svn-r14258] new feature: make h5diff use the same command line parsing code and syntax than h5dump
usage is now 
h5diff  [OPTIONS]  file1 file2  [obj1[obj2]]

tested: windows, linux, solaris
2007-11-13 16:40:23 -05:00
Pedro Vicente Nunes
aaddd739b4 [svn-r14216] add a -V version information for h5diff
tested: windows, linux
2007-10-22 16:02:40 -05:00
Pedro Vicente Nunes
8e4bcb5fee [svn-r14201] attribute ordering: added objects that do not have creation order tracked
current behavior is
if DCPL has creation order tracked for attributes  then sort the attributes  by creation order otherwise by name
regarding sort order (ascending or descending) it is done in whatever is requested

tested: linux
2007-10-11 14:42:30 -05:00
Pedro Vicente Nunes
e387120f94 [svn-r14200] added tests for XML output of attribute iteration orders (same calls as DDL )
tested: linux
2007-10-11 13:47:55 -05:00
Pedro Vicente Nunes
29fef95eb5 [svn-r14190]
Bug fixes
Avoid passing iteration flags on the dump_group function parameters but instead inspect the groups's property list just before calling H5Literate and H5Aiterate, in this later case checking the creation order flags with H5Pget_attr_creation_order

Tested: windows, linux
2007-10-06 09:54:02 -05:00
Pedro Vicente Nunes
59b7b09846 [svn-r14188] added a named datatype with attributes to the h5 file that shows attributes with several iteration orders
tested: windows, linux
2007-10-05 09:53:24 -05:00
Pedro Vicente Nunes
041de441c5 [svn-r14186] Added support for displaying several iteration orders on group and named datatype attributes
tested: windows, linux
2007-10-04 15:44:07 -05:00
Pedro Vicente Nunes
3026ace6ae [svn-r14182]
Added support for displaying several iteration orders on dataset attributes, 4 new tests in test script (name ascending, name descending, creation_order ascending, creation_order descending)
New h5 file is made on the generator program

Tested: windows, linux
2007-10-04 11:21:30 -05:00
Quincey Koziol
59b51a2ea7 [svn-r14180] Description:
Make H5Aiterate() versioned and change all internal use to H5Aiterate2()
	
	Leave some regression tests that exercise H5Aiterate1()

	Fix attribute display in h5dump & h5ls to be "by name" by default

Tested on:
        FreeBSD/32 6.2 (duty) in debug mode
        FreeBSD/64 6.2 (liberty) w/C++ & FORTRAN, in debug mode
        Linux/32 2.6 (kagiso) w/PGI compilers, w/C++ & FORTRAN, w/threadsafe,
                                in debug mode
        Linux/64 2.6 (smirom) w/default API=1.6.x, w/C++ & FORTRAN,
                                in production mode
        Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN,
                                w/szip filter, in production mode
        Mac OS X/32 10.4.10 (amazon) in debug mode
2007-10-04 10:06:28 -05:00
Pedro Vicente Nunes
fa94f16ad8 [svn-r14174] new h5dump usage to include -q and -z
tested: windows, linux, solaris 5.10

usage: h5dump [OPTIONS] file
  OPTIONS
     -h, --help           Print a usage message and exit
     -n, --contents       Print a list of the file contents and exit
     -B, --bootblock      Print the content of the boot block
     -H, --header         Print the header only; no data is displayed
     -A, --onlyattr       Print the header and value of attributes
     -i, --object-ids     Print the object ids
     -r, --string         Print 1-byte integer datasets as ASCII
     -e, --escape         Escape non printing characters
     -V, --version        Print version number and exit
     -a P, --attribute=P  Print the specified attribute
     -d P, --dataset=P    Print the specified dataset
     -y, --noindex        Do not print array indices with the data
     -p,   --properties   Print dataset filters, storage layout and fill value
     -f D, --filedriver=D Specify which driver to open the file with
     -g P, --group=P      Print the specified group and all members
     -l P, --soft-link=P  Print the value(s) of the specified soft link
     -o F, --output=F     Output raw data into file F
     -b B, --binary=B     Binary file output, of form B
     -t P, --datatype=P   Print the specified named datatype
     -w N, --width=N      Set the number of columns of output
     -q Q, --sort_by=Q    Sort groups and attributes by index Q
     -z Z, --sort_order=Z Sort groups and attributes by order Z
     -x, --xml            Output in XML using Schema
     -u, --use-dtd        Output in XML using DTD
     -D U, --xml-dtd=U    Use the DTD or schema at U
     -X S, --xml-ns=S      (XML Schema) Use qualified names n the XML
                          ":": no namespace, default: "hdf5:"
                          E.g., to dump a file called `-f', use h5dump -- -f

 Subsetting is available by using the following options with a dataset
 attribute. Subsetting is done by selecting a hyperslab from the data.
 Thus, the options mirror those for performing a hyperslab selection.
 The START and COUNT parameters are mandatory if you do subsetting.
 The STRIDE and BLOCK parameters are optional and will default to 1 in
 each dimension.

      -s L, --start=L     Offset of start of subsetting selection
      -S L, --stride=L    Hyperslab stride
      -c L, --count=L     Number of blocks to include in selection
      -k L, --block=L     Size of block in hyperslab

  D - is the file driver to use in opening the file. Acceptable values
        are "sec2", "family", "split", "multi", "direct", and "stream". Without
        the file driver flag, the file will be opened with each driver in
        turn and in the order specified above until one driver succeeds
        in opening the file.
  F - is a filename.
  P - is the full path from the root group to the object.
  N - is an integer greater than 1.
  L - is a list of integers the number of which are equal to the
        number of dimensions in the dataspace being queried
  U - is a URI reference (as defined in [IETF RFC 2396],
        updated by [IETF RFC 2732])
  B - is the form of binary output: MEMORY for a memory type, FILE for the
        file type, LE or BE for pre-existing little or big endian types.
        Must be used with -o (output file) and it is recommended that
        -d (dataset) is used
  Q - is the sort index type. It can be "creation_order" or "name" (default)
  Z - is the sort order type. It can be "descending" or "ascending" (default)

  Examples:

  1) Attribute foo of the group /bar_none in file quux.h5

        h5dump -a /bar_none/foo quux.h5

  2) Selecting a subset from dataset /foo in file quux.h5

      h5dump -d /foo -s "0,1" -S "1,1" -c "2,3" -k "2,2" quux.h5

  3) Saving dataset 'dset' in file quux.h5 to binary file 'out.bin' using a litt
le-endian type

      h5dump -d /dset -b LE -o out.bin quux.h5
2007-10-02 14:54:36 -05:00
Pedro Vicente Nunes
b2b63b3c75 [svn-r14173]
bug fix

the binary option expects a full path in -o
TOOLTEST   tbin1.ddl -d integer -o $TESTDIR/out1.bin -b LE    tbinary.h5

and it prints it in the expected output , making it absolutely not portable

Solution: made a special macro function TOOLTEST1 identical to TOOLTEST  except that it does not print the Expected output header

#############################
Expected output for 'h5dump -d integer -o /home/pvn/kagiso/build_hdf5/tools/h5dump/../testfiles/out1.bin -b LE tbinary.h5'
#############################



Tested : linux
2007-10-02 13:34:30 -05:00
Pedro Vicente Nunes
d0bcb06e7a [svn-r14172]
1)	added 5 new tests for the group creation order
2)	modified the h5dump test script to automatically generated non existing (new) output files
3)	cleaning of unused DDL files
4)	new modified DDL files include tcomp-3.ddl ( new form of named datatype) and the binary output files

tested : linux
2007-10-02 11:45:46 -05:00
Pedro Vicente Nunes
1bafc18730 [svn-r14164]
first batch for displaying groups in creation order

1) Added extra parameter to dump_group of type H5_index_t , to be passed to H5Literate.

When a group is tried to be displayed, an inquiry of its creation properties is made. If H5P_CRT_ORDER_TRACKED is present on the group property list then the display is made by creation order, otherwise it is made by name

2) Added a new file to h5dumpgentest that generates a file with a hierarchy of groups with creation order on and off in alternately  


Note : XML code was not modified

Tested : windows, linux
2007-10-01 09:49:23 -05:00
Quincey Koziol
579284f422 [svn-r14144] Description:
Move H5Gget_objinfo() to deprecated symbols section and retarget
internal usage to H5Lget_info()/H5Oget_info().

	Misc. other code cleanups...

Tested on:
	FreeBSD/32 6.2 (duty)
	FreeBSD/64 6.2 (liberty)
	Linux/32 2.6 (kagiso)
	Linux/64 2.6 (smirom)
	AIX/32 5.3 (copper)
	Solaris/32 2.10 (linew)
	Mac OS X/32 10.4.10 (amazon)
2007-09-13 10:44:56 -05:00
Quincey Koziol
3bed870363 [svn-r14136] Description:
Move H5Gget_num_objs() and several minor macros, etc. to deprecated
symbols section, replacing it with H5Gget_info().

Tested on:
	FreeBSD/32 6.2 (duty)
	FreeBSD/64 6.2 (liberty)
	Linux/32 2.6 (kagiso)
	Linux/64 2.6 (smirom)
	AIX/32 5.3 (copper)
	Solaris/32 5.10 (linew)
	Mac OS X/32 10.4.10 (amazon)
2007-09-06 10:54:22 -05:00
Quincey Koziol
5c35e64c8a [svn-r14132] Description:
Regenerate the h5diff_types.h5 file, which had somehow gotten mangled
in some earlier checkin (prior to ~2-3 days ago, at least).

	Clean up formatting a bit also..

Tested on:
	Mac OS X/32 10.4.10 (amazon)
2007-08-30 15:34:13 -05:00
Pedro Vicente Nunes
a1fcb56d78 [svn-r14051]
Make the compare to itself file test be another due to the recently introduced file with Nans

Tested: linux, solaris
2007-08-07 16:47:29 -05:00
Pedro Vicente Nunes
021ed070b2 [svn-r14048]
Added a dataset with NaN values to the h5diff file generator program and new binary file

Tested: linux
2007-08-07 16:08:39 -05:00
Scott Wegner
4965da5a72 [svn-r13894] Description:
On Windows, Mingw interprets all parameters starting with '/' as paths, and replaces the '/' with its home directory, "C:\Windows\msys\".  This was a problem in h5diff tests such as:
        h5diff h5diff_101.txt $FILE1 $FILE1 /g1/d1  g1/d2  -v

I've removed the leading '/', as h5diff will interpret it the same either way.

Tested:
kagiso, linew, and smirom, via h5committest
mingw on Windows XP
2007-06-20 13:34:11 -05:00
Pedro Vicente Nunes
65de8add1f [svn-r13805]
Early detection of no arguments, print usage and exit

Tested: linux
2007-05-24 07:49:24 -05:00
Pedro Vicente Nunes
c80d46791f [svn-r13775]
Modified the current h5dump test script to use h5import/h5diff calls to validate the binary output. At this moment it can only be used with the native test, since h5import does not deal with input endianess.


tested: linux, sunos 5.10
2007-05-21 10:24:50 -05:00
Quincey Koziol
9d2070890b [svn-r13636] Description:
Change H5[D|G|T]<foo>_expand() "temporary" API routines to
H5[D|G|T]<foo>2() "versioned" routines.  Also added
H5[D|G|T](create|commit)_anon() routines to continue to allow "anonymous"
objects to be created in a file.

Tested on:
    Mac OS X/32 10.4.9 (amazon)
    FreeBSD/32 6.2 (duty)
    FreeBSD/64 6.2 (liberty)
    Linux/32 2.6 (chicago)
    Linux/64 2.6 (chicago2)
2007-04-11 11:22:16 -05:00
Pedro Vicente Nunes
ea63d5fb19 [svn-r13582]
Bug fix

Substitute a malloc call for a calloc call while allocating a reference buffer
Add a test for an empty reference case
Tested linux
2007-04-03 15:44:52 -05:00
Quincey Koziol
0fb88ded47 [svn-r13580] Description:
Add version # and flags to external link format (as fields in a single
byte), in order to accomodate future changes/expansions.

Tested on:
    Mac OS X/32 10.4.9 (amazon)
    Linux/32 2.6 (chicago)
    Linux/64 2.6 (chicago2)
    FreeBSD/32 6.2 (duty)
    FreeBSD/64 6.2 (liberty)
2007-04-03 14:51:14 -05:00
Pedro Vicente Nunes
104e1cd848 [svn-r13573]
Bug fixes
Reset external file list slots name_offset to a state when created (0) in H5P_dcrt_copy
so that it conforms to an assertion in H5D_update_entry_info that assumes the name_offset is 0 at this point

this fixes the problem of h5repack and external files, add a new test and files for an external file

h5diff, check for an error return in H5D_get_storage_size

tested linux 32, 64
2007-04-02 15:28:14 -05:00
Quincey Koziol
a64589450e [svn-r13498] Description:
Eliminate storing # of links in "link info" message, regenerate it
when the object is opened instead.

Tested on:
	FreeBSD/32 6.2 (duty)
	Mac OS X/32 10.4.8 (amazon)
2007-03-11 21:38:08 -05:00
Quincey Koziol
e6b818134e [svn-r13497] Description:
Move ref. count of # of links to an object out of the object header's
prefix and make it a header message instead (since it's a "rare" occurence),
eliminating some more space for each object in the file.

        Inserting this "ref. count" message exposed a flaw in the library's
mechanism for locating a message to promote to another chunk and replace
with a continuation message, which required some additional work to fix.
It's still not completely robust, but it's working for more cases now and
detects failures robustly.

        Reduced the minimum size of an object header chunk to just enough to
contain a header message prefix and continuation message.

Tested on:
	FreeBSD/32 6.2 (duty)
2007-03-11 18:15:03 -05:00
Quincey Koziol
0b3cccd0cb [svn-r13496] Description:
Reduce size of "default" link message storage.

Tested on:
	FreeBSD/32 6.2 (duty)
2007-03-10 22:06:05 -05:00
Quincey Koziol
9129a8452a [svn-r13493] Description:
Eliminate storing default values for "group info" fields.

Tested on:
	FreeBSD/32 6.2 (duty)
2007-03-10 20:09:00 -05:00
Quincey Koziol
1eb19fc895 [svn-r13491] Description:
Reduce the size of the value used to store the # of bytes in the
"payload" for chunk 0 of an object header.

Tested on:
	FreeBSD/32 6.2 (duty)
2007-03-10 15:02:55 -05:00