Commit Graph

584 Commits

Author SHA1 Message Date
vchoi-hdfgroup
e7bb05a886
Fix for CVE-2018-15671. h5stat -S $POC will result in a crash with segmenetation fault. (#3427)
It is because the object in the testfile points back to the root group.
When the tool tries to traverse the object, it goes back to the root group and then back again.
2023-08-25 08:13:04 -05:00
jhendersonHDF
915cadb44f
Make h5dump spacing consistent when printing VLEN datatype (#3351) 2023-08-04 09:02:40 -07:00
mattjala
d87efd3a46
Add note about HDF5_VOL_CONNECTOR to tools usage (#3159) 2023-06-28 09:56:35 -07:00
Egbert Eich
b16ec83d4b
Check for overflow when calculating on-disk attribute data size (#2459)
* Remove duplicate code

Signed-off-by: Egbert Eich <eich@suse.com>

* Add test case for CVE-2021-37501

Bogus sizes in this test case causes the on-disk data size
calculation in H5O__attr_decode() to overflow so that the
calculated size becomes 0. This causes the read to overflow
and h5dump to segfault.
This test case was crafted, the test file was not directly
generated by HDF5.
Test case from:
https://github.com/ST4RF4LL/Something_Found/blob/main/HDF5_v1.13.0_h5dump_heap_overflow.md
2023-03-02 11:17:49 -06:00
Dana Robinson
fcf41b3cd6
Onion VFD (#1953)
* Onion VFD feature

* Fixes onion VFD errors with non-sec2 backing store VFDs

* Disables the onion VFD tests w/ ph5diff

* Disables non-sec2 VFDs as onion VFD backing stores

* Committing clang-format changes

* Formatted source

* Typo

* Adds onion VFD tools tests to CMake

* Fixes for v16 API compatibility

* Memset structs to avoid bad frees on errors

* H5Dwrite() calls now use H5T_NATIVE_INT as the memory type vs LE

* Properly decodes checksums on BE machines

* Be more careful about uint64_t to haddr_t/hsize_t conversions

* Another fix for BE data comparison

* Removed double underscores from onion constants

* Replace hard-coded onion header string w/ constant

* Fixes cleanup paths in H5FD__onion_ingest_history()

* Fixed use of size_t revision numbers

* Fix h5dump revision count format string

Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
2022-08-02 12:54:40 -07:00
Dana Robinson
079df9d216
Fixes a stack overflow in the h5dump test generator (#1903)
* Fixes a stack overflow in the h5dump test generator

The opaque type size was declared to be the size of the "buffer of two
opaque elements", causing a memcpy call in H5Dwrite to read outside
the buffer, dumping garbage into the file.

Note that this only affected the test generator and not h5dump itself.

* Committing clang-format changes

Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
2022-07-18 07:35:43 -07:00
Allen Byrne
b6398dd60e
Move error-stack text to top of usage display (#1564)
* Move error-stack text to top of usage display

* Add optional tag ti list

* format changes

* Revert incorrect change
2022-04-01 15:55:22 -05:00
jhendersonHDF
3da0802c40
VFD plugins (#602)
* Implement support for loading of Virtual File Drivers as plugins

Fix plugin caching for VOL connector and VFD plugins

Fix plugin iteration to skip paths that can't be opened

* Enable dynamic loading of VFDs with HDF5_DRIVER environment variable

* Temporarily disable error reporting during H5F_open double file open

* Default to using HDstat in h5_get_file_size for unknown VFDs

* Use macros for some environment variables that HDF5 interprets

* Update "null" and "ctl testing" VFDs
2021-09-29 13:28:12 -05:00
Allen Byrne
e21f7aaac4
Tools long double updates (#522)
* OESS-98 convert plugin option to FetchContent, add tests

* Fixes for pkcfg files because of plugin option

* OESS-98 fix tools test for plugins

* Keep doxygen comments under 100 chars long - format hint

* Whitespace

* HDFFV-11144 - Reclassify CMake messages

* HDFFV-11099/11100 added help text

* Reworked switch statement to compare string instead

* Fix typo

* Update CDash mode

* Correct name of threadsafe

* Correct option name

* Undo accidental commit

* Note LLVM 10 to 11 format default changes

* Update format plugin

* Undo clang-format version 11 changes

* One more correction

* Update supported platforms

* Revert whitespace changes

* Correct whitespace

* Changes from PR#3

* HDFFV-11213 added option to control gcc10 warnings diagnostics

* HDFFV-11212 Use the new references correctly in JNI utility and tests

* format source

* Fix typo

* Add new test file

* HDFFV-11212 - update test and remove unused arg

* Minor non-space formatting changes

* Use H5I_INVALID_ID instead of "-1"

* source formatting

* add missing testfile, update jni function

* Undo commit of debug code

* remove mislocated file

* Fix h5repack test for handling of fapls and id close

* Update h5diff test files usage text

* HDFFV-11212 add new ref tests for JNI export dataset

* src format update

* Remove blank line typo

* src format typo

* long double requires %Lg

* Another long double foramt specifer S.B. %Lg

* issue with t128bit test

* Windows issue with h5dump and type.

* Fix review issues

* refactor function nesting and fix error checks

* format fixes

* Remove untested functions and javadoc quiet comments

* Restore TRY block.

* Change string append errors to memory exception

* revert to H5_JNI_FATAL_ERROR - support functions need work

* Add assertion error for h5util functions

* remove duplicate function

* format fix

* Revert HD function error handling

* Update copyright comments

* GH #386 java folder copyright corrections

* Whitespace

* GH #359 implement and fix tools 1.6 API usage

* remove excessive comments

* Flip inits to correct ifdef section

* rework ifdef to be simpler

* format issue

* Reformat ifdef inits

* remove static attribute

* format compliance

* Update names

* Revert because logic relies on float not being int

* Changes noticed from creating merge of #412

* Double underscore change

* Correct compiler version variable used

* Remove header guard underscores

* Whitespace cleanup

* Split format source and commit changes on repo push

* remove pre-split setting

* Change windows TS to use older VS.

* correct window os name

* HDFFV-11212 JNI export util and Javadoc

* Suggested review changes

* Another change found

* Committing clang-format changes

* HDFFV-11113 long double in tools

* HDFFV-11113 add note

* Disable long double tests for now

* HDFFV-11228 remove arbitrary CMake warning groups.

* Make each flag a string

* Some Javadoc warning fixes

* Updated javadoc fixes

* # WARNING: head commit changed in the meantime

HDFFV-11229 Fix long double usage in tools and java

Changed h5dump and h5ls to just print 128-bit for long double type.
Added test and file for dataset and attributes with all three float
types.

* Committing clang-format changes

* HDFFV-11229 Add release note

* HDFFV-11229 windows testfile needed

* fix typo

* Remove non supported message text

* HDFFV-11229 - change ldouble test to check both native and general

* HDFFV-11229 add second file compare

* HDFFV-11229 fix reference file

* HDFFV-11229 autotools check two refs

* HDFFV-11229 revert back to removal of NATIVE_LDOUBLE in tools output

* Committing clang-format changes

* Update release note

* Update attribute type of ref file

* Change source of ninja for macs

* try port instead of brew

* Recommended is to use brew.

* Undo non long double changes

* remove unneeded file

Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
2021-04-15 07:59:01 -05:00
bmribler
dafc7285bb
Fixed HDFFV-10480 (CVE-2018-11206) and HDFFV-11159 (CVE-2018-14033) (#405)
* Fixed HDFFV-10480 (CVE-2018-11206) and HDFFV-11159 (CVE-2018-14033)
Description
    Checked against buffer size to prevent segfault, in case of data corruption.

    + HDFFV-11159 CVE-2018-14033 Buffer over-read in H5O_layout_decode
    + HDFFV-10480 CVE-2018-11206 Buffer over-read in H5O_fill_new[/old]_decode
Platforms tested:
    Linux/64 (jelly)

* Accidentally left in another occurrence of the previous patch from user
   after a more correct fix was applied, that is the check now accounted
   for the previous advance of the buffer pointer.  Removed it.

* Typo

* Fixed format issues.

* Added test.

* Changed arguments to ADD_H5_TEST

* Fixing arguments to ADD_H5_TEST again.

* Fixing arguments again.

* Took out the CMake changes until Allen can help.

* Added files:

tCVE_2018_11206_fill_old.h5
tCVE_2018_11206_fill_new.h5

* Revert "Took out the CMake changes until Allen can help."

This reverts commit c21324d6e0.

* Revert "Fixing arguments again."

This reverts commit 5832a70674.

* Revert "Fixing arguments to ADD_H5_TEST again."

This reverts commit b45de823c2.

* Revert "Changed arguments to ADD_H5_TEST"

This reverts commit 16719824f5.

* Added first argument to ADD_H5_TEST for HDFFV-10480 fix.

* Changed argument 0 to 1

* Revert "Changed argument 0 to 1"

This reverts commit b343d6613b.

* Revert "Added first argument to ADD_H5_TEST for HDFFV-10480 fix."

This reverts commit b8a0f9a9e8.

* Added first argument and corrected the second.

* Updated fixes for HDFFV-10480 and HDFFV-11159/HDFFV-11049

* Improved error messages.
2021-03-19 08:15:03 -05:00
Allen Byrne
af54fd532c
Fix display of long double in tools (#469)
* OESS-98 convert plugin option to FetchContent, add tests

* Fixes for pkcfg files because of plugin option

* OESS-98 fix tools test for plugins

* Keep doxygen comments under 100 chars long - format hint

* Whitespace

* HDFFV-11144 - Reclassify CMake messages

* HDFFV-11099/11100 added help text

* Reworked switch statement to compare string instead

* Fix typo

* Update CDash mode

* Correct name of threadsafe

* Correct option name

* Undo accidental commit

* Note LLVM 10 to 11 format default changes

* Update format plugin

* Undo clang-format version 11 changes

* One more correction

* Update supported platforms

* Revert whitespace changes

* Correct whitespace

* Changes from PR#3

* HDFFV-11213 added option to control gcc10 warnings diagnostics

* HDFFV-11212 Use the new references correctly in JNI utility and tests

* format source

* Fix typo

* Add new test file

* HDFFV-11212 - update test and remove unused arg

* Minor non-space formatting changes

* Use H5I_INVALID_ID instead of "-1"

* source formatting

* add missing testfile, update jni function

* Undo commit of debug code

* remove mislocated file

* Fix h5repack test for handling of fapls and id close

* Update h5diff test files usage text

* HDFFV-11212 add new ref tests for JNI export dataset

* src format update

* Remove blank line typo

* src format typo

* long double requires %Lg

* Another long double foramt specifer S.B. %Lg

* issue with t128bit test

* Windows issue with h5dump and type.

* Fix review issues

* refactor function nesting and fix error checks

* format fixes

* Remove untested functions and javadoc quiet comments

* Restore TRY block.

* Change string append errors to memory exception

* revert to H5_JNI_FATAL_ERROR - support functions need work

* Add assertion error for h5util functions

* remove duplicate function

* format fix

* Revert HD function error handling

* Update copyright comments

* GH #386 java folder copyright corrections

* Whitespace

* GH #359 implement and fix tools 1.6 API usage

* remove excessive comments

* Flip inits to correct ifdef section

* rework ifdef to be simpler

* format issue

* Reformat ifdef inits

* remove static attribute

* format compliance

* Update names

* Revert because logic relies on float not being int

* Changes noticed from creating merge of #412

* Double underscore change

* Correct compiler version variable used

* Remove header guard underscores

* Whitespace cleanup

* Split format source and commit changes on repo push

* remove pre-split setting

* Change windows TS to use older VS.

* correct window os name

* HDFFV-11212 JNI export util and Javadoc

* Suggested review changes

* Another change found

* Committing clang-format changes

* HDFFV-11113 long double in tools

* HDFFV-11113 add note

Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
2021-03-17 10:27:21 -05:00
Allen Byrne
c7ffe683e5
Primary change is HDFFV-11212 - new refs and JNI (#372)
* OESS-98 convert plugin option to FetchContent, add tests

* Fixes for pkcfg files because of plugin option

* OESS-98 fix tools test for plugins

* Keep doxygen comments under 100 chars long - format hint

* Whitespace

* HDFFV-11144 - Reclassify CMake messages

* HDFFV-11099/11100 added help text

* Reworked switch statement to compare string instead

* Fix typo

* Update CDash mode

* Correct name of threadsafe

* Correct option name

* Undo accidental commit

* Note LLVM 10 to 11 format default changes

* Update format plugin

* Undo clang-format version 11 changes

* One more correction

* Update supported platforms

* Revert whitespace changes

* Correct whitespace

* Changes from PR#3

* HDFFV-11213 added option to control gcc10 warnings diagnostics

* HDFFV-11212 Use the new references correctly in JNI utility and tests

* format source

* Fix typo

* Add new test file

* HDFFV-11212 - update test and remove unused arg

* Minor non-space formatting changes

* Use H5I_INVALID_ID instead of "-1"

* source formatting

* add missing testfile, update jni function

* Undo commit of debug code

* remove mislocated file

* Fix h5repack test for handling of fapls and id close

* Update h5diff test files usage text

* HDFFV-11212 add new ref tests for JNI export dataset

* src format update

* Remove blank line typo

* src format typo

* long double requires %Lg

* Another long double foramt specifer S.B. %Lg

* issue with t128bit test

* Windows issue with h5dump and type.

* Fix review issues

* refactor function nesting and fix error checks

* format fixes

* Remove untested functions and javadoc quiet comments

* Restore TRY block.

* Change string append errors to memory exception

* revert to H5_JNI_FATAL_ERROR - support functions need work

* Add assertion error for h5util functions

* remove duplicate function

* format fix

* Revert HD function error handling

* Update copyright comments
2021-02-25 15:12:57 -06:00
Allen Byrne
e01430ec56 HDFFV-11096 replace backticks with doublequotes 2020-10-04 11:46:42 -05:00
Allen Byrne
31773b7c48 HDFFV-11096 fix typo in h5dump usage text 2020-10-03 08:21:40 -05:00
Allen Byrne
bac0fc6309 Tools refactor
Move error_stack setup to tools library
Add structure for h5dump properties
Eliminate duplicated h5diff functions
Use properties structure in place of argument lists.
2020-07-15 11:20:16 -05:00
Dana Robinson
8cd9d8349e Added VOL command-line options to (p)h5diff, h5ls, h5dump, and h5mkgrp. 2020-04-21 11:36:10 -07:00
Dana Robinson
8e948aa7b7 Updated test ddl files so CMake passes after h5dump failures. 2020-04-06 15:19:55 -07:00
Dana Robinson
85b3b9e670 Updated tools dump output when dataset offset is undefined. 2020-04-02 10:25:46 -07:00
Dana Robinson
36e8e2be4b Misc changes for h5dump VOL changes. 2020-03-30 01:51:36 -07:00
Allen Byrne
23f813026f HDFFV-10976,-10980 Init obj_type before calling H5Rget_obj_type3 2019-12-20 13:23:48 -06:00
Allen Byrne
628c63f125 HDFFV-10980 - h5diff uses new ref APIs 2019-12-19 16:29:43 -06:00
Allen Byrne
eed923ec70 HDFFV-10876 fix compare reference 2019-12-08 13:25:14 -06:00
Allen Byrne
972cf8e60c Add missing quotes 2019-12-07 10:30:38 -06:00
Allen Byrne
7d6d3e2956 Correct indentation 2019-12-07 08:42:19 -06:00
Allen Byrne
e8ad5f6d1a HDFFV-10876 Update dump to match DDL spec 2019-12-05 17:21:02 -06:00
Allen Byrne
4f74548f53 Add extra line 2019-12-04 17:00:45 -06:00
Allen Byrne
11825bc525 Adjust whitespace 2019-12-04 16:59:03 -06:00
Allen Byrne
d552e97eae Update h5ls help and rework dump_mem routines for attrs 2019-12-04 16:36:32 -06:00
Allen Byrne
7ad3d62a5c Update data output reference 2019-12-03 16:05:35 -06:00
Allen Byrne
929490f6bc Update reference files to latest output 2019-12-02 12:01:40 -06:00
Allen Byrne
6b927a773c HDFFV-10876 Update h5dump and h5ls for new ref api. 2019-11-07 09:49:11 -06:00
Allen Byrne
369d17c05c HDFFV-10876 h5dump OPAQUE size should be optional 2019-08-27 15:10:37 -05:00
Larry Knox
8008294578 Squashed commit of the following:
Merge changes from update_merged_S3_HDFS branch into develop.

commit d5034315aea88629929ac0c9c59ebfafd5f21a31
Merge: 9c48823 d3fdcd8
Author: Larry Knox <lrknox@hdfgroup.org>
Date:   Thu Jul 25 08:24:53 2019 -0500

    Merge branch 'develop' into update_merged_S3_HDFS
2019-07-25 11:47:12 -05:00
Allen Byrne
a03d35dc22 HDFFV-9407 Add test 2019-06-24 07:28:39 -05:00
Allen Byrne
e3e24a2989 HDFFV-10632 - split stderr and stdout in cmake 2018-12-03 09:35:59 -06:00
Dana Robinson
a56b4db4ae Remainder of vol_normalization changes (dataset, attribute, files, objects). 2018-09-24 12:23:41 -07:00
Allen Byrne
b14c8bdb1c Add test file with unwritten datasets 2018-07-24 12:00:07 -05:00
Vailin Choi
cf38292064 Fix for HDFFV-10333:
1) Check for valid object header version for a refcount messge
2) Check for invalid fill value size
3) Check for invalid dimension size in a layout message
4) Add --enable-error-stack option to h5stat
5) Add error checks to h5stat.c
6) Add tests to h5stat and h5dump
2018-07-10 22:00:14 -05:00
Allen Byrne
ebb3bb0cfc HDFFV-10384 Add opt arg to help 2018-02-15 16:58:51 -06:00
Allen Byrne
d200cd31a7 HDFFV-10188 emu insists on printing non-native description 2017-08-23 10:16:49 -05:00
Allen Byrne
138c3ee366 HDFFV-10188 Add tests and files 2017-08-21 11:56:31 -05:00
Allen Byrne
f695491f05 HDFFV-10188 enable null space test 2017-08-21 10:35:49 -05:00
Allen Byrne
e1a81b17e0 HDFFV-10256 Add test 2017-08-01 13:52:10 -05:00
Allen Byrne
ec0935a924 HDFFV-10186 xml test also needs endianess aware files 2017-05-10 12:45:06 -05:00
Allen Byrne
772e6e9a7f Fix reference file contents 2017-05-10 11:00:37 -05:00
Allen Byrne
445c805a4c HDFFV-10186 Add endianess to bitfield and fix tests 2017-05-10 10:59:13 -05:00
Allen Byrne
412f68103e Use correct var types for data 2017-05-04 13:32:58 -05:00
Quincey Koziol
0313cbb91d Final merge of page buffering branch to develop 2017-03-13 21:30:37 -07:00
Allen Byrne
7e73366e91 HDFFV-10118 fix data 2017-01-20 16:40:40 -06:00
Allen Byrne
eb42e42ea4 HDFFV-10118 change h5ls to not display extra info 2017-01-20 15:53:45 -06:00