dwarf2_build_psymtabs sets per_bfd->partial_symtabs at its end, in
order to facilitate sharing.
However, in order to diassociate partial symtabs from the objfile, we
want to change the DWARF reader to reference partial symtabs via the
per_bfd. So, we want to set this reference before reading psymtabs.
This is safe to do because the assignment is done unconditionally.
gdb/ChangeLog
2021-03-20 Tom Tromey <tom@tromey.com>
* dwarf2/read.c (dwarf2_build_psymtabs): Set partial_symtabs
earlier.
This changes the psymtab_discarder to not assume that partial symtabs
are attached to the objfile. Instead, a psymtab_storage object is
passed directly to it.
gdb/ChangeLog
2021-03-20 Tom Tromey <tom@tromey.com>
* psympriv.h (psymtab_discarder): Take psymtab_storage parameter.
(~psymtab_discarder, keep): Update.
<m_objfile>: Remove.
<m_partial_symtabs>: New member.
* dwarf2/read.c (dwarf2_build_psymtabs): Update.
Currently, all psymbol readers access the psymtab storage via the
objfile. This is done directly at any spot requiring the storage.
In order to move psymbols out of the objfile, the psymtab_storage must
be passed in explicitly. This patch consolidates the access of the
storage in a single place in these readers, updating various functions
to pass the storage object around. "Hidden" uses, like
"objfile->psymtabs ()", are also updated.
The DWARF reader is not yet touched. That requires more complicated
changes.
gdb/ChangeLog
2021-03-20 Tom Tromey <tom@tromey.com>
* xcoffread.c (xcoff_end_psymtab): Add partial_symtabs parameter.
(xcoff_end_psymtab): Update.
(scan_xcoff_symtab): Add partial_symtabs parameter.
(xcoff_initial_scan): Update.
* stabsread.h (dbx_end_psymtab): Add partial_symtabs parameter.
* mdebugread.c (mdebug_build_psymtabs): Update.
(parse_partial_symbols): Add partial_symtabs parameter.
* dbxread.c (dbx_symfile_read): Update.
(read_dbx_symtab): Add partial_symtabs parameter.
(read_dbx_symtab): Update.
(dbx_end_psymtab): Add partial_symtabs parameter.
objfile::psymbol_map is used to implement a Rust feature. It is
currently specific to partial symbols -- it isn't used by the DWARF
indices.
This patch moves it out of objfile and into psymbol_functions, adding
a new method to quick_symbol_functions to handle the clearing case.
This is needed because the map holds unrelocated addresses.
gdb/ChangeLog
2021-03-20 Tom Tromey <tom@tromey.com>
* quick-symbol.h (struct quick_symbol_functions)
<relocated>: New method.
* psymtab.h (struct psymbol_functions) <relocated>: New
method.
<fill_psymbol_map>: Declare method.
<m_psymbol_map>: New member.
* psymtab.c (psymbol_functions::fill_psymbol_map): Rename.
(psymbol_functions::find_compunit_symtab_by_address): Update.
* objfiles.h (reset_psymtabs): Don't clear psymbol_map.
(struct objfile) <psymbol_map>: Remove.
* objfiles.c (objfile_relocate1): Update.
This changes quick_symbol_functions to be a base class with pure
virtual methods, rather than a struct holding function pointers.
Then, objfile is changed to hold a unique_ptr to an instance of this
class.
struct psymbol_functions is put into psympriv.h, and not psymtab.c,
because that is convenient later in the series.
gdb/ChangeLog
2021-03-20 Tom Tromey <tom@tromey.com>
* psympriv.h (struct psymbol_functions): New.
* symfile.c (syms_from_objfile_1, reread_symbols): Update.
* symfile-debug.c (objfile::find_compunit_symtab_by_address)
(objfile::lookup_global_symbol_language): Update.
* quick-symbol.h (struct quick_symbol_functions): Convert function
pointers to methods. Add virtual destructor.
(quick_symbol_functions_up): New typedef.
* psymtab.h (psym_functions, dwarf2_gdb_index_functions)
(dwarf2_debug_names_functions): Don't declare.
(make_psymbol_functions): Declare.
* psymtab.c (psymbol_functions::map_symtabs_matching_filename)
(psymbol_functions::find_pc_sect_compunit_symtab)
(psymbol_functions::lookup_symbol)
(psymbol_functions::lookup_global_symbol_language)
(psymbol_functions::find_last_source_symtab)
(psymbol_functions::forget_cached_source_info)
(psymbol_functions::print_stats, psymbol_functions::dump)
(psymbol_functions::expand_symtabs_for_function)
(psymbol_functions::expand_all_symtabs)
(psymbol_functions::expand_symtabs_with_fullname)
(psymbol_functions::map_symbol_filenames)
(psymbol_functions::map_matching_symbols)
(psymbol_functions::expand_symtabs_matching)
(psymbol_functions::has_symbols)
(psymbol_functions::find_compunit_symtab_by_address): Rename.
(psym_functions): Remove.
(make_psymbol_functions): New function.
* objfiles.h (struct objfile) <qf>: Change type.
* elfread.c (elf_symfile_read): Update.
* dwarf2/read.c (struct dwarf2_base_index_functions)
(struct dwarf2_gdb_index, struct dwarf2_debug_names_index): New.
(make_dwarf_gdb_index, make_dwarf_debug_names): New functions.
(dwarf2_base_index_functions::find_last_source_symtab)
(dwarf2_base_index_functions::forget_cached_source_info)
(dwarf2_base_index_functions::map_symtabs_matching_filename)
(dwarf2_gdb_index::lookup_symbol)
(dwarf2_base_index_functions::print_stats)
(dwarf2_gdb_index::dump)
(dwarf2_gdb_index::expand_symtabs_for_function)
(dwarf2_base_index_functions::expand_all_symtabs)
(dwarf2_base_index_functions::expand_symtabs_with_fullname):
Rename.
(dwarf2_gdb_index::map_matching_symbols): New method.
(dwarf2_gdb_index::expand_symtabs_matching): New method.
(dwarf2_base_index_functions::find_pc_sect_compunit_symtab)
(dwarf2_base_index_functions::map_symbol_filenames)
(dwarf2_base_index_functions::has_symbols): Rename.
(dwarf2_gdb_index_functions): Remove.
(dwarf2_debug_names_index::lookup_symbol)
(dwarf2_debug_names_index::dump)
(dwarf2_debug_names_index::expand_symtabs_for_function)
(dwarf2_debug_names_index::map_matching_symbols)
(dwarf2_debug_names_index::expand_symtabs_matching): Rename.
(dwarf2_debug_names_functions): Remove.
* dwarf2/public.h (make_dwarf_gdb_index, make_dwarf_debug_names):
Declare.
Currently the "partial" symbol reader is attached to the objfile's
symbol functions. However, in order to allow multiple separate
partial symbol readers, this association must be changed. This patch
moves the "qf" member out of sym_fns as a step toward that goal.
gdb/ChangeLog
2021-03-20 Tom Tromey <tom@tromey.com>
* psymtab.c (require_partial_symbols): Check that 'sf' is not
null.
* xcoffread.c (xcoff_sym_fns): Update.
* symfile.h (struct sym_fns) <qf>: Remove.
* symfile.c (syms_from_objfile_1, reread_symbols): Update.
* symfile-debug.c (objfile::has_partial_symbols)
(objfile::find_last_source_symtab)
(objfile::forget_cached_source_info)
(objfile::map_symtabs_matching_filename, objfile::lookup_symbol)
(objfile::print_stats, objfile::dump)
(objfile::expand_symtabs_for_function)
(objfile::expand_all_symtabs)
(objfile::expand_symtabs_with_fullname)
(objfile::map_matching_symbols)
(objfile::expand_symtabs_matching)
(objfile::find_pc_sect_compunit_symtab)
(objfile::map_symbol_filenames)
(objfile::find_compunit_symtab_by_address)
(objfile::lookup_global_symbol_language, debug_sym_fns)
(install_symfile_debug_logging): Update.
* objfiles.h (struct objfile) <qf>: New member.
* mipsread.c (ecoff_sym_fns): Update.
* machoread.c (macho_sym_fns): Update.
* elfread.c (elf_sym_fns_gdb_index, elf_sym_fns_debug_names):
Don't declare.
(elf_symfile_read, elf_sym_fns, elf_sym_fns_lazy_psyms): Update.
* dbxread.c (aout_sym_fns): Update.
* coffread.c (coff_sym_fns): Update.
This introduces a new header, quick-symbol.h, and moves
quick_symbol_functions and related typedefs into it.
gdb/ChangeLog
2021-03-20 Tom Tromey <tom@tromey.com>
* symfile.h (symbol_compare_ftype, symbol_filename_ftype)
(expand_symtabs_file_matcher_ftype)
(expand_symtabs_symbol_matcher_ftype)
(expand_symtabs_exp_notify_ftype, struct quick_symbol_functions):
Move to quick-symbol.h.
* quick-symbol.h: New file.
This introduces wrappers for each function in quick_symbol_functions.
The wrappers are methods on objfile, and are defined in
symfile-debug.c, so that they can use the symfile_debug variable.
Places that call the quick functions are all updated to call these new
wrapper methods.
gdb/ChangeLog
2021-03-20 Tom Tromey <tom@tromey.com>
* symtab.c (iterate_over_symtabs, expand_symtab_containing_pc)
(lookup_symbol_via_quick_fns, find_quick_global_symbol_language)
(basic_lookup_transparent_type_quick)
(find_pc_sect_compunit_symtab, find_symbol_at_address)
(find_line_symtab, global_symbol_searcher::expand_symtabs):
Update.
* symmisc.c (print_objfile_statistics, dump_objfile)
(maintenance_expand_symtabs): Update.
* symfile.c (symbol_file_add_with_addrs)
(expand_symtabs_matching, map_symbol_filenames): Update.
* symfile-debug.c (objfile::has_partial_symbols)
(objfile::find_last_source_symtab)
(objfile::forget_cached_source_info)
(objfile::map_symtabs_matching_filename, objfile::lookup_symbol)
(objfile::print_stats, objfile::dump)
(objfile::expand_symtabs_for_function)
(objfile::expand_all_symtabs)
(objfile::expand_symtabs_with_fullname)
(objfile::map_matching_symbols)
(objfile::expand_symtabs_matching)
(objfile::find_pc_sect_compunit_symtab)
(objfile::map_symbol_filenames)
(objfile::find_compunit_symtab_by_address)
(objfile::lookup_global_symbol_language): New methods.
(debug_sym_quick_functions): Remove.
(debug_sym_fns, install_symfile_debug_logging): Update.
* source.c (forget_cached_source_info_for_objfile)
(select_source_symtab): Update.
* objfiles.h (struct objfile): Add methods corresponding to
quick_symbol_functions.
* objfiles.c (objfile::has_partial_symbols): Move to
symfile-debug.c.
* linespec.c (iterate_over_all_matching_symtabs): Update.
* cp-support.c (add_symbol_overload_list_qualified): Update.
* ada-lang.c (add_nonlocal_symbols): Update.
This changes objfile_has_partial_symbols to be a method on objfile.
There are some other functions that could benefit from this sort of
change, but this was the only one that was relevant to this series.
gdb/ChangeLog
2021-03-20 Tom Tromey <tom@tromey.com>
* symfile.c (read_symbols): Update.
* objfiles.h (struct objfile) <has_partial_symbols>: New method.
(objfile_has_partial_symbols): Don't declare.
* objfiles.c (objfile::has_partial_symbols): Rename from
objfile_has_partial_symbols.
(objfile_has_symbols, have_partial_symbols): Update.
* elfread.c (elf_symfile_read): Update.
* dwarf2/read.c (dwarf2_has_info): Update.
* coffread.c (coff_symfile_read): Update.
This moves some more DWARF code out of symfile.h and into a new
header, dwarf2/public.h. This header is intended to describe the
public API of the DWARF reader.
gdb/ChangeLog
2021-03-20 Tom Tromey <tom@tromey.com>
* coffread.c: Include dwarf2/public.h.
* dwarf2/frame.c: Include dwarf2/public.h.
* dwarf2/index-write.h: Include dwarf2/public.h, not symfile.h.
* dwarf2/public.h: New file.
* dwarf2/read.c: Include dwarf2/public.h.
* elfread.c: Include dwarf2/public.h.
* machoread.c: Include dwarf2/public.h.
* symfile.h (dwarf2_has_info, enum dw_index_kind)
(dwarf2_initialize_objfile, dwarf2_build_psymtabs)
(dwarf2_build_frame_info): Move to dwarf2/public.h.
* xcoffread.c: Include dwarf2/public.h.
This moves a bit of the DWARF-specific code out of symfile.h and into
dwarf2/read.h.
gdb/ChangeLog
2021-03-20 Tom Tromey <tom@tromey.com>
* symfile.h (enum dwarf2_section_enum)
(dwarf2_get_section_info): Move to dwarf2/read.h.
* dwarf2/read.h (enum dwarf2_section_enum)
(dwarf2_get_section_info): Move from symfile.h.
gold/
PR gold/pr23870
* testsuite/aarch64_pr23870_bar.c: Return a magic value.
* testsuite/aarch64_pr23870_foo.c: Check the magic value and return
success or failure.
The exception for debug sections in clearing SEC_EXCLUDE when
relocatable was really for one specific debug section, so let's make
it do just that.
bfd/
PR 27590
* elf.c (_bfd_elf_make_section_from_shdr): Remove SHF_EXCLUDE
test for .gnu.debuglto*.
ld/
PR 27590
* ldlang.c (lang_gc_sections): Clear SEC_EXCLUDE when relocatable
for all sections except .stabstr.
We're currently seeing testing of native-extended-gdbserver hang while
testing the x86_64 architecture on both Fedora 34 and Fedora Rawhide.
The test responsible for the hang is gdb.threads/fork-plus-threads.exp.
While there is clearly a problem/bug with this test on F34 and
Rawhide, it's also the case that testing should not hang. This commit
prevents the hang by waiting with the "-nowait" flag in
close_gdbserver.
The -nowait flag is also used in the kill_wait_spawned_process proc in
gdb/testsuite/lib/gdb.exp, so there is precedent for doing this.
There are also 15 other uses of "wait -i" scattered throughout the
test suite. While it's tempting to change these to also use the
-nowait flag, I think it might be safer to defer doing so until we
actually see a problem.
I've tested this patch on Fedora 32, 33, 34, and Rawhide. Results are
comparable on Fedora 32 and 33. On Fedora 34 and Rawhide, with this
commit in place, testing completes when the target_board is
native-extended-gdbserver. On those OSes, when not using this commit,
testing usually hangs due to a problem with
gdb.threads/fork-plus-threads.exp. I've also tested on all of the
mentioned OSes with target_board=native-gdbserver; for that testing,
I achieved comparable results over a number of runs. (Unfortunately
results are rarely identical due to racy tests.)
gdb/testsuite/ChangeLog:
* lib/gdbserver-support.exp (gdbserver_exit): Use the
"-nowait" flag when waiting for gdbserver to exit.
Running gdb-term.exp against gdbserver with "maint set target-non-stop
on", runs into this:
[infrun] fetch_inferior_event: exit
[infrun] fetch_inferior_event: enter
/home/pedro/gdb/binutils-gdb/src/gdb/thread.c:72: internal-error: thread_info* inferior_thread(): Assertion `current_thread_ != nullptr' failed.
A problem internal to GDB has been detected,
further debugging may prove unreliable.
This is a bug, please report it. For instructions, see:
<https://www.gnu.org/software/gdb/bugs/>.
FAIL: gdb.base/gdb-sigterm.exp: expect eof #2 (GDB internal error)
Resyncing due to internal error.
ERROR: : spawn id exp9 not open
while executing
"expect {
-i exp9 -timeout 10
-re "Quit this debugging session\\? \\(y or n\\) $" {
send_gdb "n\n" answer
incr count
}
-re "Create ..."
("uplevel" body line 1)
invoked from within
"uplevel $body" NONE : spawn id exp9 not open
ERROR: Could not resync from internal error (timeout)
gdb.base/gdb-sigterm.exp: expect eof #2: stepped 0 times
UNRESOLVED: gdb.base/gdb-sigterm.exp: 50 SIGTERM passes
The assertion fails here:
...
#5 0x000055af4b4a7164 in internal_error (file=0x55af4b5e5de8 "/home/pedro/gdb/binutils-gdb/src/gdb/thread.c", line=72, fmt=0x55af4b5e5ce9 "%s: Assertion `%s' failed.") at /home/pedro/gdb/binutils-gdb/src/gdbsupport/errors.cc:55
#6 0x000055af4b25fc43 in inferior_thread () at /home/pedro/gdb/binutils-gdb/src/gdb/thread.c:72
#7 0x000055af4b26177e in any_thread_of_inferior (inf=0x55af4cf874f0) at /home/pedro/gdb/binutils-gdb/src/gdb/thread.c:638
#8 0x000055af4b26eec8 in kill_or_detach (inf=0x55af4cf874f0, from_tty=0) at /home/pedro/gdb/binutils-gdb/src/gdb/top.c:1665
#9 0x000055af4b26f37f in quit_force (exit_arg=0x0, from_tty=0) at /home/pedro/gdb/binutils-gdb/src/gdb/top.c:1767
#10 0x000055af4b2f72a7 in quit () at /home/pedro/gdb/binutils-gdb/src/gdb/utils.c:633
#11 0x000055af4b2f730b in maybe_quit () at /home/pedro/gdb/binutils-gdb/src/gdb/utils.c:657
#12 0x000055af4b1adb74 in ser_base_wait_for (scb=0x55af4d02e460, timeout=0) at /home/pedro/gdb/binutils-gdb/src/gdb/ser-base.c:236
#13 0x000055af4b1adf0f in do_ser_base_readchar (scb=0x55af4d02e460, timeout=0) at /home/pedro/gdb/binutils-gdb/src/gdb/ser-base.c:365
#14 0x000055af4b1ae06d in generic_readchar (scb=0x55af4d02e460, timeout=0, do_readchar=0x55af4b1adeb1 <do_ser_base_readchar(serial*, int)>) at /home/pedro/gdb/binutils-gdb/src/gdb/ser-base.c:444
...
The bug is that any_thread_of_inferior incorrectly assumes that
there's always a selected thread. This fixes it.
gdb/ChangeLog:
* thread.c (any_thread_of_inferior): Check if there's a selected
thread before calling inferior_thread().
Change-Id: Ica4b9ec746121a7a7c22bef09baea72103b3853d
I don't know really why we should lose a space before a '/'. Possibly
it would make sense if '/' started a comment, but otherwise no.
* app.c (do_scrub_chars): Don't lose spaces before a slash.
commit 994b251328
Author: H.J. Lu <hjl.tools@gmail.com>
Date: Sun Jan 17 20:01:16 2021 -0800
ld/elf: Ignore section symbols when matching linkonce with comdat
ignored section symbols when comparing symbols in 2 sections. Since all
references to debugging sections are done with section symbols, symbols
in debugging sections are ignored and we fail to match symbols in comdat
debugging sections. Also .gnu.debuglto_.debug_* sections aren't treated
as debugging sections.
1. Treate .gnu.debuglto_.debug_ section as debugging section unless it
is marked with SHF_EXCLUDE.
2. Revert commit 994b251328 in elf_create_symbuf.
3. Ignore section symbols only when matching non-debugging sections or
linkonce section with comdat section.
bfd/
PR ld/27590
* elf.c (_bfd_elf_make_section_from_shdr): Treate
.gnu.debuglto_.debug_ section as debugging section unless it is
marked with SHF_EXCLUDE.
* elflink.c (elf_create_symbuf): Revert commit 994b251328.
(bfd_elf_match_symbols_in_sections): Ignore section symbols when
matching non-debugging sections or linkonce section with comdat
section.
ld/
PR ld/27590
* testsuite/ld-elf/pr27590.s: New file.
* testsuite/ld-elf/pr27590a.d: Likewise.
* testsuite/ld-elf/pr27590b.d: Likewise.
* testsuite/ld-i386/i386.exp: Also run ld/27193 test with
--reduce-memory-overheads.
I noticed that dwarf_unit_type_name is nearly identical to
get_DW_UT_name from libiberty; but rather than simply replacing it, it
seemed better to have it work like the other DWARF constant
stringification functions -- return a string showing unrecognized
numeric forms rather than nullptr. (The previous code did include
numeric values for the recognized constants, but this seems to be not
that useful to me.)
2021-03-18 Tom Tromey <tromey@adacore.com>
* dwarf2/stringify.c (dwarf_unit_type_name): New function. Use
get_DW_UT_name.
* dwarf2/stringify.h (dwarf_unit_type_name): Declare.
* dwarf2/comp-unit.c (dwarf_unit_type_name): Remove.
The small example for gdb.Parameter.get_set_string does not return a
string. The documentation is very clear that this method must return
a string, and indeed, inspecting the code in gdb/python/py-param.c
shows that a string return value is required (if an exception is not
thrown).
While inspecting the code in gdb/python/py-param.c I noticed that the
comment for the C++ code that invokes the Python get_set_string method
is wrong, so I updated that too.
gdb/ChangeLog:
* python/py-param.c (get_set_value): Update header comment.
gdb/doc/ChangeLog:
* python.texinfo (Parameters In Python): Return empty string in
small example code.
PR 27478
* readelf.c (dump_section_as_strings): Mention separate filename.
(dump_section_as_bytes): Likewise.
(dump_section_as_ctf): Likewise.
(initialise_dumkps_byname): Only issue a warning for missing
sections if processing the main file.
(process_section_contents): Only issue a warning for unsumped
section numbers in the main file.
(initialise_dump_sects): New function. Contains code extracted
from ...
(process_object): ... here. Also call initialise_dump_sects for
separate files.
The previous commit started to error-check the lookup of
ctf_type_encoding for the underlying type that is internally done when
carrying out a ctf_type_encoding on a slice.
Unfortunately, enums have no encoding, so this has historically been
returning an error (which is ignored) and then populating the cte_format
with uninitialized data. Now the error is not ignored, this is
returning an error, which breaks linking of CTF containing bitfields of
enumerated type.
CTF format v3 does not record the actual underlying type of a enum, but
we can mock up something that is not *too* wrong, and that is at any
rate better than uninitialized data.
ld/ChangeLog
2021-03-18 Nick Alcock <nick.alcock@oracle.com>
* testsuite/ld-ctf/slice.c: Check slices of enums too.
* testsuite/ld-ctf/slice.d: Results adjusted.
libctf/ChangeLog
2021-03-18 Nick Alcock <nick.alcock@oracle.com>
* ctf-types.c (ctf_type_encoding): Support, after a fashion, for enums.
* ctf-dump.c (ctf_dump_format_type): Do not report enums' degenerate
encoding.
Out-of-memory errors initializing the string atoms table were
disregarded (though they would have caused a segfault very shortly
afterwards). Errors hashing types during deduplication were only
reported if they happened on the output dict, which is almost never the
case (most errors are going to be on the dict we're working over, which
is going to be one of the inputs). (The error was detected in both
cases, but the errno was extracted from the wrong dict.)
libctf/ChangeLog
2021-03-18 Nick Alcock <nick.alcock@oracle.com>
* ctf-dedup.c (ctf_dedup_rhash_type): Report errors on the input
dict properly.
* ctf-open.c (ctf_bufopen_internal): Report errors initializing
the atoms table.
This completes the job of unifying what was once three separate code
paths full of duplication for every function dealing with querying the
properties of struct and union members. The dynamic code path was
already removed: this change removes the distinction between small and
large members, by adding a helper that copies out members from the vlen,
expanding small members into large ones as it does so.
This makes it possible to have *more* representations of things like
structure members without needing to change the querying functions at
all. It also lets us check for buffer overruns more effectively,
verifying that we don't accidentally overrun the end of the vlen in
either the dynamic or static type case.
libctf/ChangeLog
2021-03-18 Nick Alcock <nick.alcock@oracle.com>
* ctf-impl.h (ctf_next_t) <ctn_tp>: New.
<u.ctn_mp>: Remove.
<u.ctn_lmp>: Remove.
<u.ctn_vlen>: New.
* ctf-types.c (ctf_struct_member): New.
(ctf_member_next): Use it, dropping separate large/small code paths.
(ctf_type_align): Likewise.
(ctf_member_info): Likewise.
(ctf_type_rvisit): Likewise.
Eliminate the dynamic member storage for structs and unions as we have
for other dynamic types. This is much like the previous enum
elimination, except that structs and unions are the only types for which
a full-sized ctf_type_t might be needed. Up to now, this decision has
been made in the individual ctf_add_{struct,union}_sized functions and
duplicated in ctf_add_member_offset. The vlen machinery lets us
simplify this, always allocating a ctf_lmember_t and setting the
dtd_data's ctt_size to CTF_LSIZE_SENT: we figure out whether this is
really justified and (almost always) repack things down into a
ctf_stype_t at ctf_serialize time.
This allows us to eliminate the dynamic member paths from the iterators and
query functions in ctf-types.c in favour of always using the large-structure
vlen stuff for dynamic types (the diff is ugly but that's just because of the
volume of reindentation this calls for). This also means the large-structure
vlen stuff gets more heavily tested, which is nice because it was an almost
totally unused code path before now (it only kicked in for structures of size
>4GiB, and how often do you see those?)
The only extra complexity here is ctf_add_type. Back in the days of the
nondeduplicating linker this was called a ridiculous number of times for
countless identical copies of structures: eschewing the repeated lookups of the
dtd in ctf_add_member_offset and adding the members directly saved an amazing
amount of time. Now the nondeduplicating linker is gone, this is extreme
overoptimization: we can rip out the direct addition and use ctf_member_next and
ctf_add_member_offset, just like ctf_dedup_emit does.
We augment a ctf_add_type test to try adding a self-referential struct, the only
thing the ctf_add_type part of this change really perturbs.
This completes the elimination of dtd_u.
libctf/ChangeLog
2021-03-18 Nick Alcock <nick.alcock@oracle.com>
* ctf-impl.h (ctf_dtdef_t) <dtu_members>: Remove.
<dtd_u>: Likewise.
(ctf_dmdef_t): Remove.
(struct ctf_next) <u.ctn_dmd>: Remove.
* ctf-create.c (INITIAL_VLEN): New, more-or-less arbitrary initial
vlen size.
(ctf_add_enum): Use it.
(ctf_dtd_delete): Do not free the (removed) dmd; remove string
refs from the vlen on struct deletion.
(ctf_add_struct_sized): Populate the vlen: do it by hand if
promoting forwards. Always populate the full-size
lsizehi/lsizelo members.
(ctf_add_union_sized): Likewise.
(ctf_add_member_offset): Set up the vlen rather than the dmd.
Expand it as needed, repointing string refs via
ctf_str_move_pending. Add the member names as pending strings.
Always populate the full-size lsizehi/lsizelo members.
(membadd): Remove, folding back into...
(ctf_add_type_internal): ... here, adding via an ordinary
ctf_add_struct_sized and _next iteration rather than doing
everything by hand.
* ctf-serialize.c (ctf_copy_smembers): Remove this...
(ctf_copy_lmembers): ... and this...
(ctf_emit_type_sect): ... folding into here. Figure out if a
ctf_stype_t is needed here, not in ctf_add_*_sized.
(ctf_type_sect_size): Figure out the ctf_stype_t stuff the same
way here.
* ctf-types.c (ctf_member_next): Remove the dmd path and always
use the vlen. Force large-structure usage for dynamic types.
(ctf_type_align): Likewise.
(ctf_member_info): Likewise.
(ctf_type_rvisit): Likewise.
* testsuite/libctf-regression/type-add-unnamed-struct-ctf.c: Add a
self-referential type to this test.
* testsuite/libctf-regression/type-add-unnamed-struct.c: Adjusted
accordingly.
* testsuite/libctf-regression/type-add-unnamed-struct.lk: Likewise.
This is the first tricky one, the first complex multi-entry vlen
containing strings. To handle this in vlen form, we have to handle
pending refs moving around on realloc.
We grow vlen regions using a new ctf_grow_vlen function, and iterate
through the existing enums every time a grow happens, telling the string
machinery the distance between the old and new vlen region and letting
it adjust the pending refs accordingly. (This avoids traversing all
outstanding refs to find the refs that need adjusting, at the cost of
having to traverse one enum: an obvious major performance win.)
Addition of enums themselves (and also structs/unions later) is a bit
trickier than earlier forms, because the type might be being promoted
from a forward, and forwards have no vlen: so we have to spot that and
create it if needed.
Serialization of enums simplifies down to just telling the string
machinery about the string refs; all the enum type-lookup code loses all
its dynamic member lookup complexity entirely.
A new test is added that iterates over (and gets values of) an enum with
enough members to force a round of vlen growth.
libctf/ChangeLog
2021-03-18 Nick Alcock <nick.alcock@oracle.com>
* ctf-impl.h (ctf_dtdef_t) <dtd_vlen_alloc>: New.
(ctf_str_move_pending): Declare.
* ctf-string.c (ctf_str_add_ref_internal): Fix error return.
(ctf_str_move_pending): New.
* ctf-create.c (ctf_grow_vlen): New.
(ctf_dtd_delete): Zero out the vlen_alloc after free. Free the
vlen later: iterate over it and free enum name refs first.
(ctf_add_generic): Populate dtd_vlen_alloc from vlen.
(ctf_add_enum): populate the vlen; do it by hand if promoting
forwards.
(ctf_add_enumerator): Set up the vlen rather than the dmd. Expand
it as needed, repointing string refs via ctf_str_move_pending. Add
the enumerand names as pending strings.
* ctf-serialize.c (ctf_copy_emembers): Remove.
(ctf_emit_type_sect): Copy the vlen into place and ref the
strings.
* ctf-types.c (ctf_enum_next): The dynamic portion now uses
the same code as the non-dynamic.
(ctf_enum_name): Likewise.
(ctf_enum_value): Likewise.
* testsuite/libctf-lookup/enum-many-ctf.c: New test.
* testsuite/libctf-lookup/enum-many.lk: New test.
The preceding change revealed a new bug: the string table is sorted for
better compression, so repeated serialization with type (or member)
additions in the middle can move strings around. But every
serialization flushes the set of refs (the memory locations that are
automatically updated with a final string offset when the strtab is
updated), so if we are not to have string offsets go stale, we must do
all ref additions within the serialization code (which walks the
complete set of types and symbols anyway). Unfortunately, we were adding
one ref in another place: the type name in the dynamic type definitions,
which has a ref added to it by ctf_add_generic.
So adding a type, serializing (via, say, one of the ctf_write
functions), adding another type with a name that sorts earlier, and
serializing again will corrupt the name of the first type because it no
longer had a ref pointing to its dtd entry's name when its string offset
was shifted later in the strtab to mae way for the other type.
To ensure that we don't miss strings, we also maintain a set of *pending
refs* that will be added later (during serialization), and remove
entries from that set when the ref is finally added. We always use
ctf_str_add_pending outside ctf-serialize.c, ensure that ctf_serialize
adds all strtab offsets as refs (even those in the dtds) on every
serialization, and mandate that no refs are live on entry to
ctf_serialize and that all pending refs are gone before strtab
finalization. (Of necessity ctf_serialize has to traverse all strtab
offsets in the dtds in order to serialize them, so adding them as refs
at the same time is easy.)
(Note that we still can't erase unused atoms when we roll back, though
we can erase unused refs: members and enums are still not removed by
rollbacks and might reference strings added after the snapshot.)
libctf/ChangeLog
2021-03-18 Nick Alcock <nick.alcock@oracle.com>
* ctf-hash.c (ctf_dynset_elements): New.
* ctf-impl.h (ctf_dynset_elements): Declare it.
(ctf_str_add_pending): Likewise.
(ctf_dict_t) <ctf_str_pending_ref>: New, set of refs that must be
added during serialization.
* ctf-string.c (ctf_str_create_atoms): Initialize it.
(CTF_STR_ADD_REF): New flag.
(CTF_STR_MAKE_PROVISIONAL): Likewise.
(CTF_STR_PENDING_REF): Likewise.
(ctf_str_add_ref_internal): Take a flags word rather than int
params. Populate, and clear out, ctf_str_pending_ref.
(ctf_str_add): Adjust accordingly.
(ctf_str_add_external): Likewise.
(ctf_str_add_pending): New.
(ctf_str_remove_ref): Also remove the potential ref if it is a
pending ref.
* ctf-serialize.c (ctf_serialize): Prohibit addition of strings
with ctf_str_add_ref before serialization. Ensure that the
ctf_str_pending_ref set is empty before strtab finalization.
(ctf_emit_type_sect): Add a ref to the ctt_name.
* ctf-create.c (ctf_add_generic): Add the ctt_name as a pending
ref.
* testsuite/libctf-writable/reserialize-strtab-corruption.*: New test.
One pattern which is rarely done in libctf but which is meant to work is
this:
ctf_create();
ctf_add_*(); // add stuff
ctf_type_*() // look stuff up
ctf_write_*();
ctf_add_*(); // should still work
ctf_type_*() // so should this
ctf_write_*(); // and this
i.e., writing out a dict should not break it and you should be able to
do everything you could do with it before, including writing it out
again.
Unfortunately this has been broken for a while because the field which
indicates the maximum valid type ID was not preserved across
serialization: so type additions after serialization would overwrite
types (obviously disastrous) and type lookups would just fail.
Fix trivial.
libctf/ChangeLog
2021-03-18 Nick Alcock <nick.alcock@oracle.com>
* ctf-serialize.c (ctf_serialize): Preserve ctf_typemax across
serialization.
This stops problems parallel-installing if a relink of libctf is needed.
Also adds corresponding install-strip dependencies.
ChangeLog
2021-03-18 Nick Alcock <nick.alcock@oracle.com>
PR libctf/27482
* Makefile.def: Add install-bfd dependencies for install-libctf and
install-ld, and install-strip-bfd dependencies for
install-strip-libctf and install-strip-ld; move the install-ld
dependency on install-libctf to join it.
* Makefile.in: Regenerated.
One more member vanishes from the dtd_u, leaving only the member for
struct/union/enum members.
There's not much to do here, since as of commit afd78bd6f0 we use
the same representation (type sizes, etc) in the dtu_argv as we will
use in the final vlen, with one exception: the vlen has alignment
padding, and the dtu_argv did not. Simplify things by adding suitable
padding in both cases.
libctf/ChangeLog
2021-03-18 Nick Alcock <nick.alcock@oracle.com>
* ctf-impl.h (ctf_dtdef_t) <dtd_u.dtu_argv>: Remove.
* ctf-create.c (ctf_dtd_delete): No longer free it.
(ctf_add_function): Use the dtd_vlen, not dtu_argv. Properly align.
* ctf-serialize.c (ctf_emit_type_sect): Just copy the dtd_vlen.
* ctf-types.c (ctf_func_type_info): Just use the vlen.
(ctf_func_type_args): Likewise.
This is even simpler than ints, floats and slices, with the only extra
complication being the need to manually transfer the array parameter in
the rarely-used function ctf_set_array. (Arrays are unique in libctf in
that they can be modified post facto, not just created and appended to.
I'm not sure why they got this exemption, but it's easy to maintain.)
libctf/ChangeLog
2021-03-18 Nick Alcock <nick.alcock@oracle.com>
* ctf-impl.h (ctf_dtdef_t) <dtd_u.dtu_arr>: Remove.
* ctf-create.c (ctf_add_array): Use the dtd_vlen, not dtu_arr.
(ctf_set_array): Likewise.
* ctf-serialize.c (ctf_emit_type_sect): Just copy the dtd_vlen.
* ctf-types.c (ctf_array_info): Just use the vlen.
This series eliminates a lot of special-case code to handle dynamic
types (types added to writable dicts and not yet serialized).
Historically, when such types have variable-length data in their final
CTF representations, libctf has always worked by adding such types to a
special union (ctf_dtdef_t.dtd_u) in the dynamic type definition
structure, then picking the members out of this structure at
serialization time and packing them into their final form.
This has the advantage that the ctf_add_* code doesn't need to know
anything about the final CTF representation, but the significant
disadvantage that all code that looks up types in any way needs two code
paths, one for dynamic types, one for all others. Historically libctf
"handled" this by not supporting most type lookups on dynamic types at
all until ctf_update was called to do a complete reserialization of the
entire dict (it didn't emit an error, it just emitted wrong results).
Since commit 676c3ecbad, which eliminated ctf_update in favour of
the internal-only ctf_serialize function, all the type-lookup paths
grew an extra branch to handle dynamic types.
We can eliminate this branch again by dropping the dtd_u stuff and
simply writing out the vlen in (close to) its final form at ctf_add_*
time: type lookup for types using this approach is then identical for
types in writable dicts and types that are in read-only ones, and
serialization is also simplified (we just need to write out the vlen
we already created).
The only complexity lies in type kinds for which multiple
vlen representations are valid depending on properties of the type,
e.g. structures. But we can start simple, adjusting ints, floats,
and slices to work this way, and leaving everything else as is.
libctf/ChangeLog
2021-03-18 Nick Alcock <nick.alcock@oracle.com>
* ctf-impl.h (ctf_dtdef_t) <dtd_u.dtu_enc>: Remove.
<dtd_u.dtu_slice>: Likewise.
<dtd_vlen>: New.
* ctf-create.c (ctf_add_generic): Perhaps allocate it. All
callers adjusted.
(ctf_dtd_delete): Free it.
(ctf_add_slice): Use the dtd_vlen, not dtu_enc.
(ctf_add_encoded): Likewise. Assert that this must be an int or
float.
* ctf-serialize.c (ctf_emit_type_sect): Just copy the dtd_vlen.
* ctf-dedup.c (ctf_dedup_rhash_type): Use the dtd_vlen, not
dtu_slice.
* ctf-types.c (ctf_type_reference): Likewise.
(ctf_type_encoding): Remove most dynamic-type-specific code: just
get the vlen from the right place. Report failure to look up the
underlying type's encoding.
It's formatted like this:
do
{
...
}
while (...);
Not like this:
do
{
...
} while (...);
or this:
do {
...
} while (...);
We used both in various places in libctf. Fixing it necessitated some
light reindentation.
libctf/ChangeLog
2021-03-18 Nick Alcock <nick.alcock@oracle.com>
* ctf-archive.c (ctf_archive_next): GNU style fix for do {} while.
* ctf-dedup.c (ctf_dedup_rhash_type): Likewise.
(ctf_dedup_rwalk_one_output_mapping): Likewise.
* ctf-dump.c (ctf_dump_format_type): Likewise.
* ctf-lookup.c (ctf_symbol_next): Likewise.
* swap.h (swap_thing): Likewise.
ctf_serialize and its various pieces may be split out into a separate
file now, but ctf_serialize is still far too long and disordered, mixing
header initialization, sizing of multiple CTF sections, sorting and
emission of multiple CTF sections, strtab construction and ctf_dict_t
copying into a single ugly organically-grown mess.
Fix the worst of this by migrating all section sizing and emission into
separate functions, two per section (or class of section in the case of
the symtypetabs). Only the variable section is now sized and emitted
directly in ctf_serialize (because it only takes about three lines to do
so).
The section sizes themselves are still maintained by ctf_serialize so
that it can work out the header offsets, but ctf_symtypetab_sect_sizes
and ctf_emit_symtypetab_sects share a lot of extra state: migrate that
into a shared structure, emit_symtypetab_state_t.
(Test results unchanged.)
libctf/ChangeLog
2021-03-18 Nick Alcock <nick.alcock@oracle.com>
* ctf-serialize.c: General reshuffling, and...
(emit_symtypetab_state_t): New, migrated from
local variables in ctf_serialize.
(ctf_serialize): Split out most section sizing and
emission.
(ctf_symtypetab_sect_sizes): New (split out).
(ctf_emit_symtypetab_sects): Likewise.
(ctf_type_sect_size): Likewise.
(ctf_emit_type_sect): Likewise.
It is perfectly possible to have dynamically allocated data owned by a
specific dict: you just have to teach ctf_serialize about it.
libctf/ChangeLog
2021-03-18 Nick Alcock <nick.alcock@oracle.com>
* ctf-impl.h (ctf_dict_t): Fix comment.
