mirror/binutils-gdb
2
0
Fork 0
mirror of https://sourceware.org/git/binutils-gdb.git synced 2025-01-06 12:09:26 +08:00
Code Issues Packages Projects Releases Wiki Activity
118,407 Commits 386 Branches 135 Tags 591 MiB
eb244227d8
Commit Graph

35 Commits

Author SHA1 Message Date
Nick Alcock
483546ce4f libctf: make ctf_serialize() actually serialize 
ctf_serialize() evolved from the old ctf_update(), which mutated the
in-memory CTF dict to make all the dynamic in-memory types into static,
unchanging written-to-the-dict types (by deserializing and reserializing
it): back in the days when you could only do type lookups on static types,
this meant you could see all the types you added recently, at the small,
small cost of making it impossible to change those older types ever again
and inducing an amortized O(n^2) cost if you actually wanted to add
references to types you added at arbitrary times to later types.

It also reset things so that ctf_discard() would throw away only types you
added after the most recent ctf_update() call.

Some time ago this was all changed so that you could look up dynamic types
just as easily as static types: ctf_update() changed so that only its
visible side-effect of affecting ctf_discard() remained: the old
ctf_update() was renamed to ctf_serialize(), made internal to libctf, and
called from the various functions that wrote files out.

... but it was still working by serializing and deserializing the entire
dict, swapping out its guts with the newly-serialized copy in an invasive
and horrible fashion that coupled ctf_serialize() to almost every field in
the ctf_dict_t.  This is totally useless, and fixing it is easy: just rip
all that code out and have ctf_serialize return a serialized representation,
and let everything use that directly.  This simplifies most of its callers
significantly.

(It also points up another bug: ctf_gzwrite() failed to call ctf_serialize()
at all, so it would only ever work for a dict you just ctf_write_mem()ed
yourself, just for its invisible side-effect of serializing the dict!)

This lets us simplify away a bunch of internal-only open-side functionality
for overriding the syn_ext_strtab and some just-added functionality for
forcing in an existing atoms table, without loss of functionality, and lets
us lift the restriction on reserializing a dict that was ctf_open()ed rather
than being ctf_create()d: it's now perfectly OK to open a dict, modify it
(except for adding members to existing structs, unions, or enums, which
fails with -ECTF_RDONLY), and write it out again, just as one would expect.

libctf/

	* ctf-serialize.c (ctf_symtypetab_sect_sizes): Fix typos.
	(ctf_type_sect_size): Add static type sizes too.
	(ctf_serialize): Return the new dict rather than updating the
	existing dict.  No longer fail for dicts with static types;
	copy them onto the start of the new types table.
	(ctf_gzwrite): Actually serialize before gzwriting.
	(ctf_write_mem): Improve forced (test-mode) endian-flipping:
	flip dicts even if they are too small to be compressed.
	Improve confusing variable naming.
	* ctf-archive.c (arc_write_one_ctf): Don't bother to call
	ctf_serialize: both the functions we call do so.
	* ctf-string.c (ctf_str_create_atoms): Drop serializing case
	(atoms arg).
	* ctf-open.c (ctf_simple_open): Call ctf_bufopen directly.
	(ctf_simple_open_internal): Delete.
	(ctf_bufopen_internal): Delete/rename to ctf_bufopen: no
	longer bother with syn_ext_strtab or forced atoms table,
	serialization no longer needs them.
	* ctf-create.c (ctf_create): Call ctf_bufopen directly.
	* ctf-impl.h (ctf_str_create_atoms): Drop atoms arg.
	(ctf_simple_open_internal): Delete.
	(ctf_bufopen_internal): Likewise.
	(ctf_serialize): Adjust.
	* testsuite/libctf-lookup/add-to-opened.c: Adjust now that
	this is supposed to work.
 2024-04-19 16:14:47 +01:00
Nick Alcock
8a60c93096 libctf: support addition of types to dicts read via ctf_open() 
libctf has long declared deserialized dictionaries (out of files or ELF
sections or memory buffers or whatever) to be read-only: back in the
furthest prehistory this was not the case, in that you could add a
few sorts of type to such dicts, but attempting to do so often caused
horrible memory corruption, so I banned the lot.

But it turns out real consumers want it (notably DTrace, which
synthesises pointers to types that don't have them and adds them to the
ctf_open()ed dicts if it needs them). Let's bring it back again, but
without the memory corruption and without the massive code duplication
required in days of yore to distinguish between static and dynamic
types: the representation of both types has been identical for a few
years, with the only difference being that types as a whole are stored in
a big buffer for types read in via ctf_open and per-type hashtables for
newly-added types.

So we discard the internally-visible concept of "readonly dictionaries"
in favour of declaring the *range of types* that were already present
when the dict was read in to be read-only: you can't modify them (say,
by adding members to them if they're structs, or calling ctf_set_array
on them), but you can add more types and point to them.  (The API
remains the same, with calls sometimes returning ECTF_RDONLY, but now
they do so less often.)

This is a fairly invasive change, mostly because code written since the
ban was introduced didn't take the possibility of a static/dynamic split
into account.  Some of these irregularities were hard to define as
anything but bugs.

Notably:

 - The symbol handling was assuming that symbols only needed to be
   looked for in dynamic hashtabs or static linker-laid-out indexed/
   nonindexed layouts, but now we want to check both in case people
   added more symbols to a dict they opened.

 - The code that handles type additions wasn't checking to see if types
   with the same name existed *at all* (so you could do
   ctf_add_typedef (fp, "foo", bar) repeatedly without error).  This
   seems reasonable for types you just added, but we probably *do* want
   to ban addition of types with names that override names we already
   used in the ctf_open()ed portion, since that would probably corrupt
   existing type relationships.  (Doing things this way also avoids
   causing new errors for any existing code that was doing this sort of
   thing.)

 - ctf_lookup_variable entirely failed to work for variables just added
   by ctf_add_variable: you had to write the dict out and read it back
   in again before they appeared.

 - The symbol handling remembered what symbols you looked up but didn't
   remember their types, so you could look up an object symbol and then
   find it popping up when you asked for function symbols, which seems
   less than ideal.  Since we had to rejig things enough to be able to
   distinguish function and object symbols internally anyway (in order
   to give suitable errors if you try to add a symbol with a name that
   already existed in the ctf_open()ed dict), this bug suddenly became
   more visible and was easily fixed.

We do not (yet) support writing out dicts that have been previously read
in via ctf_open() or other deserializer (you can look things up in them,
but not write them out a second time).  This never worked, so there is
no incompatibility; if it is needed at a later date, the serializer is a
little bit closer to having it work now (the only table we don't deal
with is the types table, and that's because the upcoming CTFv4 changes
are likely to make major changes to the way that table is represented
internally, so adding more code that depends on its current form seems
like a bad idea).

There is a new testcase that tests much of this, in particular that
modification of existing types is still banned and that you can add new
ones and chase them without error.

libctf/

	* ctf-impl.h (struct ctf_dict.ctf_symhash): Split into...
	(ctf_dict.ctf_symhash_func): ... this and...
	(ctf_dict.ctf_symhash_objt): ... this.
	(ctf_dict.ctf_stypes): New, counts static types.
	(LCTF_INDEX_TO_TYPEPTR): Use it instead of CTF_RDWR.
	(LCTF_RDWR): Deleted.
	(LCTF_DIRTY): Renumbered.
	(LCTF_LINKING): Likewise.
	(ctf_lookup_variable_here): New.
	(ctf_lookup_by_sym_or_name): Likewise.
	(ctf_symbol_next_static): Likewise.
	(ctf_add_variable_forced): Likewise.
	(ctf_add_funcobjt_sym_forced): Likewise.
	(ctf_simple_open_internal): Adjust.
	(ctf_bufopen_internal): Likewise.
	* ctf-create.c (ctf_grow_ptrtab): Adjust a lot to start with.
	(ctf_create): Migrate a bunch of initializations into bufopen.
	Force recreation of name tables.  Do not forcibly override the
	model, let ctf_bufopen do it.
	(ctf_static_type): New.
	(ctf_update): Drop LCTF_RDWR check.
	(ctf_dynamic_type): Likewise.
	(ctf_add_function): Likewise.
	(ctf_add_type_internal): Likewise.
	(ctf_rollback): Check ctf_stypes, not LCTF_RDWR.
	(ctf_set_array): Likewise.
	(ctf_add_struct_sized): Likewise.
	(ctf_add_union_sized): Likewise.
	(ctf_add_enum): Likewise.
	(ctf_add_enumerator): Likewise (only on the target dict).
	(ctf_add_member_offset): Likewise.
	(ctf_add_generic): Drop LCTF_RDWR check.  Ban addition of types
	with colliding names.
	(ctf_add_forward): Note safety under the new rules.
	(ctf_add_variable): Split all but the existence check into...
	(ctf_add_variable_forced): ... this new function.
	(ctf_add_funcobjt_sym): Likewise...
	(ctf_add_funcobjt_sym_forced): ... for this new function.
	* ctf-link.c (ctf_link_add_linker_symbol): Ban calling on dicts
	with any stypes.
	(ctf_link_add_strtab): Likewise.
	(ctf_link_shuffle_syms): Likewise.
	(ctf_link_intern_extern_string): Note pre-existing prohibition.
	* ctf-lookup.c (ctf_lookup_by_id): Drop LCTF_RDWR check.
	(ctf_lookup_variable): Split out looking in a dict but not
	its parent into...
	(ctf_lookup_variable_here): ... this new function.
	(ctf_lookup_symbol_idx): Track whether looking up a function or
	object: cache them separately.
	(ctf_symbol_next): Split out looking in non-dynamic symtypetab
	entries to...
	(ctf_symbol_next_static): ... this new function.  Don't get confused
	by the simultaneous presence of static and dynamic symtypetab entries.
	(ctf_try_lookup_indexed):  Don't waste time looking up symbols by
	index before there can be any idea how symbols are numbered.
	(ctf_lookup_by_sym_or_name): Distinguish between function and
	data object lookups.  Drop LCTF_RDWR.
	(ctf_lookup_by_symbol): Adjust.
	(ctf_lookup_by_symbol_name): Likewise.
	* ctf-open.c (init_types): Rename to...
	(init_static_types): ... this.  Drop LCTF_RDWR.  Populate ctf_stypes.
	(ctf_simple_open): Drop writable arg.
	(ctf_simple_open_internal): Likewise.
	(ctf_bufopen): Likewise.
	(ctf_bufopen_internal): Populate fields only used for writable dicts.
	Drop LCTF_RDWR.
	(ctf_dict_close): Cater for symhash cache split.
	* ctf-serialize.c (ctf_serialize): Use ctf_stypes, not LCTF_RDWR.
	* ctf-types.c (ctf_variable_next): Drop LCTF_RDWR.
	* testsuite/libctf-lookup/add-to-opened*: New test.
 2024-04-19 16:14:46 +01:00
Nick Alcock
2ba5ec13b2 libctf: fix name lookup in dicts containing base-type bitfields 
The intent of the name lookup code was for lookups to yield non-bitfield
basic types except if none existed with a given name, and only then
return bitfield types with that name.  Unfortunately, the code as
written only does this if the base type has a type ID higher than all
bitfield types, which is most unlikely (the opposite is almost always
the case).

Adjust it so that what ends up in the name table is the highest-width
zero-offset type with a given name, if any such exist, and failing that
the first type with that name we see, no matter its offset.  (We don't
define *which* bitfield type you get, after all, so we might as well
just stuff in the first we find.)

Reported by Stephen Brennan <stephen.brennan@oracle.com>.

libctf/

	* ctf-open.c (init_types): Modify to allow some lookups during open;
	detect bitfield name reuse and prefer less bitfieldy types.
	* testsuite/libctf-writable/libctf-bitfield-name-lookup.*: New test.
 2024-04-19 16:14:46 +01:00
Nick Alcock
9b9e262994 libctf: fix uninitialized variables in testsuite 
Just because a path is an error path doesn't mean the program terminates
there if you don't ask it to.  And we don't want to -- but that means
we need to initialize the variables that are missed if an error happens to
*something*.  Type ID 0 (unimplemented) will do: it'll induce further
ECTF_BADID errors, but that's no bad thing.

libctf/ChangeLog:

	* testsuite/libctf-writable/libctf-errors.c: Initialize variables.
 2024-03-11 17:16:12 +00:00
Alan Modra
fd67aa1129 Update year range in copyright notice of binutils files 
Adds two new external authors to etc/update-copyright.py to cover
bfd/ax_tls.m4, and adds gprofng to dirs handled automatically, then
updates copyright messages as follows:

1) Update cgen/utils.scm emitted copyrights.
2) Run "etc/update-copyright.py --this-year" with an extra external
   author I haven't committed, 'Kalray SA.', to cover gas testsuite
   files (which should have their copyright message removed).
3) Build with --enable-maintainer-mode --enable-cgen-maint=yes.
4) Check out */po/*.pot which we don't update frequently.
 2024-01-04 22:58:12 +10:30
Nick Alcock
cd4a8fc4f0 libctf: fix creation-time parent/child dict confusions 
The fixes applied a few years ago to resolve confusions between parent and
child dicts at lookup time also apply in various forms to creation.  In
general, if you have a type in a parent dict ctf_imported into a child and
you do something to it, and the parent dict is writable (created via
ctf_create, not opened via ctf_open*) it should work just the same to make
changes to that type via a child dict as it does to make the change
to the parent dict directly -- and nothing you're prohibited from doing
to the parent dict when done directly should be allowed just because
you're doing it via a child.

Specifically, the following don't work when doing things from the child, but
should:

 - adding a member of a type in the parent to a struct or union in the
   parent via ctf_add_member or ctf_add_member_offset: this yields
   ECTF_BADID

 - adding a member of a type in the parent to a struct or union in the
   parent via ctf_add_member_encoded: this dumps core (!).

 - adding an enumerand to an enumerator in the parent: this yields
   ECTF_BADID

 - setting the properties of an array in the parent via ctf_set_array;
   this yields ECTF_BADID

Relatedly, some things work when doing things via a child that should fail,
yielding a CTF dictionary with invalid content (readable, but meaningless):
in particular, you can add a child type to a struct in the parent via
any of the ctf_add_member* family and nothing complains at all, even though
you should never be able to add references to children to parents (since any
given parent can be associated with many different children).

A family of tests is added to check each of these cases independently, since
some can result in coredumps and it would be nice to test the other cases
even if some dump core.  They use a common library to do all the actual
work.  The set of affected API calls was determined by code inspection
(auditing all calls to ctf_dtd_lookup): it's possible that I missed a few,
but I doubt it, since other cases use ctf_lookup* functions, which already
climb to the parent where appropriate.

libctf/ChangeLog:

	PR libctf/30985
	* ctf-create.c (ctf_dtd_lookup): Traverse to parents if necessary.
	(ctf_set_array): Likewise.  Report errors on the child; require
	both parent and child to be writable.
	(ctf_add_enumerator): Likewise.
	(ctf_add_member_offset): Likewise.  Prohibit addition of child types
	to structs in the parent.
	(ctf_add_member_encoded): Do not dereference a NULL dtd: report
	ECTF_BADID instead.
	* ctf-string.c (ctf_str_add_ref_internal): Report ENOMEM on the
	dict if addition of a string ref fails.
	* testsuite/libctf-writable/parent-child-dtd-crash-lib.c: New library.
	* testsuite/libctf-writable/parent-child-dtd-enum.*: New test.
	* testsuite/libctf-writable/parent-child-dtd-enumerator.*: New test.
	* testsuite/libctf-writable/parent-child-dtd-member-encoded.*: New test.
	* testsuite/libctf-writable/parent-child-dtd-member-offset.*: New test.
	* testsuite/libctf-writable/parent-child-dtd-set-array.*: New test.
	* testsuite/libctf-writable/parent-child-dtd-struct.*: New test.
	* testsuite/libctf-writable/parent-child-dtd-union.*: New test.
 2023-10-20 18:09:54 +01:00
Nick Alcock
29736fc507 libctf: check for problems with error returns 
We do this as a writable test because the only known-affected platforms
(with ssize_t longer than unsigned long) use PE, and we do not have support
for CTF linkage in the PE linker yet.

	PR libctf/30836
	* libctf/testsuite/libctf-writable/libctf-errors.*: New test.
 2023-10-18 12:41:00 +01:00
Nick Alcock
d7474051e8 libctf: propagate errors from parents correctly 
CTF dicts have per-dict errno values: as with other errno values these
are set on error and left unchanged on success.  This means that all
errors *must* set the CTF errno: if a call leaves it unchanged, the
caller is apt to find a previous, lingering error and misinterpret
it as the real error.

There are many places in libctf where we carry out operations on parent
dicts as a result of carrying out other user-requested operations on
child dicts (e.g. looking up information on a pointer to a type will
look up the type as well: the pointer might well be in a child and the
type it's a pointer to in the parent).  Those operations on the parent
might fail; if they do, the error must be correctly reflected on the
child that the user-visible operation was carried out on.  In many
places this was not happening.

So, audit and fix all those places.  Add tests for as many of those
cases as possible so they don't regress.

libctf/
	* ctf-create.c (ctf_add_slice): Use the original dict.
	* ctf-lookup.c (ctf_lookup_variable): Propagate errors.
	(ctf_lookup_symbol_idx): Likewise.
	* ctf-types.c (ctf_member_next): Likewise.
	(ctf_type_resolve_unsliced): Likewise.
	(ctf_type_aname): Likewise.
	(ctf_member_info): Likewise.
	(ctf_type_rvisit): Likewise.
	(ctf_func_type_info): Set the error on the right dict.
	(ctf_type_encoding): Use the original dict.
	* testsuite/libctf-writable/error-propagation.*: New test.
 2023-04-08 16:07:17 +01:00
Nick Alcock
30a794e9f1 libctf, tests: do not assume host and target have identical field offsets 
The newly-introduced libctf-lookup unnamed-field-info test checks
C compiler-observed field offsets against libctf-computed ones
by #including the testcase in the lookup runner as well as
generating CTF for it.  This only works if the host, on which
the lookup runner is compiled and executed, is the same architecture as
the target, for which the CTF is generated: when crossing, the trick
may fail.

So pass down an indication of whether this is a cross into the
testsuite, and add a new no_cross flag to .lk files that is used to
suppress test execution when a cross-compiler is being tested.

libctf/
	* Makefile.am (check_DEJAGNU): Pass down TEST_CROSS.
	* Makefile.in: Regenerated.
	* testsuite/lib/ctf-lib.exp (run_lookup_test): Use it to
	implement the new no_cross option.
	* testsuite/libctf-lookup/unnamed-field-info.lk: Mark as
	no_cross.
 2023-04-08 16:07:17 +01:00
Nick Alcock
3672e32622 libctf: get the offsets of fields of unnamed structs/unions right 
We were failing to add the offsets of the containing struct/union
in this case, leading to all offsets being relative to the unnamed
struct/union itself.

libctf/
	PR libctf/30264
	* ctf-types.c (ctf_member_info): Add the offset of the unnamed
	member of the current struct as necessary.
	* testsuite/libctf-lookup/unnamed-field-info*: New test.
 2023-03-24 13:37:32 +00:00
Nick Alcock
e2dc08c6f0 libctf: skip the testsuite from inside dejagnu 
The libctf testsuite uses Tcl try/catch to trap run_output errors.  This
is only supported in reasonably recent Tcls, so we detect the lack of
try/catch and suppress the testsuite via an Automake conditional in its
absence.

But this turns out not to work: Automake produces a check-DEJAGNU target
regardless of the value of this conditional and sticks it in an
unconditionally-executed part of the makefile, so the testsuite gets
executed anyway, and fails with a nasty-looking syntax error.  We can't
disable it by taking "dejagnu" out of AUTOMAKE_OPTIONS, because if you
do that Automake stops you using RUNTEST, RUNTESTFLAGS and other
variables users would expect to work.

So move to disabling the testsuite from inside the testsuite itself,
importing the value of the former Automake conditional as a Tcl variable
and exiting very early in default.exp if it's false.

	* configure.ac (TCL_TRY): No longer an Automake conditional.
	Rename to...
	(HAVE_TCL_TRY): ... this.
	* Makefile.am: Drop TCL_TRY.
	(development.exp): Set have_tcl_try.
	* testsuite/config/default.exp: Exit if have_tcl_try is false.

	* configure: Regenerated.
	* Makefile.in: Likewise.
 2023-01-12 14:40:47 +00:00
Alan Modra
d87bef3a7b Update year range in copyright notice of binutils files 
The newer update-copyright.py fixes file encoding too, removing cr/lf
on binutils/bfdtest2.c and ld/testsuite/ld-cygwin/exe-export.exp, and
embedded cr in binutils/testsuite/binutils-all/ar.exp string match.
 2023-01-01 21:50:11 +10:30
Nick Alcock
6a8829ebb2 libctf: tests: prune warnings from compiler output 
We were failing to call prune_warnings appropriately, leading to
false-positive test failures on some platforms (observed on
sparclinux).

libctf/ChangeLog:

	* testsuite/lib/ctf-lib.exp: Prune warnings from compiler and
	linker output.
	* testsuite/libctf-regression/libctf-repeat-cu.exp: Likewise,
	and ar output too.
 2022-06-21 19:27:15 +01:00
Nick Alcock
6bd2318f32 libctf: fix linking together multiple objects derived from the same source 
Right now, if you compile the same .c input repeatedly with CTF enabled
and different compilation flags, then arrange to link all of these
together, then things misbehave in various ways.  libctf may conflate
either inputs (if the .o files have the same name, say if they are
stored in different .a archives), or per-CU outputs when conflicting
types are found: the latter can lead to entirely spurious errors when
it tries to produce multiple per-CU outputs with the same name
(discarding all but the last, but then looking for types in the earlier
ones which have just been thrown away).

Fixing this is multi-pronged.  Both inputs and outputs need to be
differentiated in the hashtables libctf keeps them in: inputs with the
same cuname and filename need to be considered distinct as long as they
have different associated CTF dicts, and per-CU outputs need to be
considered distinct as long as they have different associated input
dicts.  Right now there is nothing tying the two together other than the
CU name: fix this by introducing a new field in the ctf_dict_t named
ctf_link_in_out, which (for input dicts) points to the associated per-CU
output dict (if any), and for output dicts points to the associated
input dict.  At creation time the name used is completely arbitrary:
it's only important that it be distinct if CTF dicts are distinct.  So,
when a clash is found, adjust the CU name by sticking the number of
elements in the input on the end.  At output time, the CU name will
appear in the linked object, so it matters a little more that it look
slightly less ugly: in conflicting cases, append an incrementing
integer, starting at 0.

This naming scheme is not very helpful, but it's hard to see what else
we can do.  The input .o name may be the same.  The input .a name is not
even visible to ctf_link, and even *that* might be the same, because
.a's can contain many members with the same name, all of which
participate in the link.  All we really know is that the two have
distinct dictionaries with distinct types in them, and at least this way
they are all represented, any any symbols, variables etc referring to
those types are accurately stored.

(As a side-effect this also fixes a use-after-free and double-free when
errors are found during variable or symbol emission.)

Use the opportunity to prevent a couple of sources of problems, to wit
changing the active CU mappings when a link has already been done
(no effect on ld, which doesn't use CU mappings at all), and causing
multiple consecutive ctf_link's to have the same net effect as just
doing the last one (no effect on ld, which only ever does one
ctf_link) rather than having the links be a sort of half-incremental
not-really-intended mess.

libctf/ChangeLog:

	PR libctf/29242
	* ctf-impl.h (struct ctf_dict) [ctf_link_in_out]: New.
	* ctf-dedup.c (ctf_dedup_emit_type): Set it.
	* ctf-link.c (ctf_link_add_ctf_internal): Set the input
	CU name uniquely when clashes are found.
	(ctf_link_add): Document what repeated additions do.
	(ctf_new_per_cu_name): New, come up with a consistent
	name for a new per-CU dict.
	(ctf_link_deduplicating): Use it.
	(ctf_create_per_cu): Use it, and ctf_link_in_out, and set
	ctf_link_in_out properly.  Don't overwrite per-CU dicts with
	per-CU dicts relating to different inputs.
	(ctf_link_add_cu_mapping): Prevent per-CU mappings being set up
	if we already have per-CU outputs.
	(ctf_link_one_variable): Adjust ctf_link_per_cu call.
	(ctf_link_deduplicating_one_symtypetab): Likewise.
	(ctf_link_empty_outputs): New, delete all the ctf_link_outputs
	and blank out ctf_link_in_out on the corresponding inputs.
	(ctf_link): Clarify the effect of multiple ctf_link calls.
	Empty ctf_link_outputs if it already exists rather than
	having the old output leak into the new link.  Fix a variable
	name.
	* testsuite/config/default.exp (AR): Add.
	(OBJDUMP): Likewise.
	* testsuite/libctf-regression/libctf-repeat-cu.exp: New test.
	* testsuite/libctf-regression/libctf-repeat-cu*: Main program,
	library, and expected results for the test.
 2022-06-21 19:27:15 +01:00
Alan Modra
a2c5833233 Update year range in copyright notice of binutils files 
The result of running etc/update-copyright.py --this-year, fixing all
the files whose mode is changed by the script, plus a build with
--enable-maintainer-mode --enable-cgen-maint=yes, then checking
out */po/*.pot which we don't update frequently.

The copy of cgen was with commit d1dd5fcc38ead reverted as that commit
breaks building of bfp opcodes files.
 2022-01-02 12:04:28 +10:30
Nick Alcock
6ab5b6d0f3 libctf, lookup: fix bounds of pptrtab lookup 
An off-by-one bug in the check for pptrtab lookup meant that we could
access the pptrtab past its bounds (*well* past its bounds),
particularly if we called ctf_lookup_by_name in a child dict with "*foo"
where "foo" is a type that exists in the parent but not the child and no
previous lookups by name have been carried out.  (Note that "*foo" is
not even a valid thing to call ctf_lookup_by_name with: foo * is.
Nonetheless, users sometimes do call ctf_lookup_by_name with invalid
content, and it should return ECTF_NOTYPE, not crash.)

ctf_pptrtab_len, as its name suggests (and as other tests of it in
ctf-lookup.c confirm), is one higher than the maximum valid permissible
index, so the comparison is wrong.

(Test added, which should fail pretty reliably in the presence of this
bug on any machine with 4KiB pages.)

libctf/ChangeLog
2021-09-27  Nick Alcock  <nick.alcock@oracle.com>

	* ctf-lookup.c (ctf_lookup_by_name_internal): Fix pptrtab bounds.
	* testsuite/libctf-writable/pptrtab-writable-page-deep-lookup.*:
	New test.
 2021-09-27 20:31:26 +01:00
Nick Alcock
e695879142 libctf, testsuite: fix various warnings in tests 
These warnings are all off by default, but if they do fire you get
spurious ERRORs when running make check-libctf.

libctf/ChangeLog
2021-09-27  Nick Alcock  <nick.alcock@oracle.com>

	* testsuite/libctf-lookup/enum-symbol.c: Remove unused label.
	* testsuite/libctf-lookup/conflicting-type-syms.c: Remove unused
	variables.
	* testsuite/libctf-regression/pptrtab.c: Likewise.
	* testsuite/libctf-regression/type-add-unnamed-struct.c: Likewise.
	* testsuite/libctf-writable/pptrtab.c: Likewise.
	* testsuite/libctf-writable/reserialize-strtab-corruption.c:
	Likewise.
	* testsuite/libctf-regression/nonstatic-var-section-ld-r.c: Fix
	format string.
	* testsuite/libctf-regression/nonstatic-var-section-ld.c:
	Likewise.
	* testsuite/libctf-regression/nonstatic-var-section-ld.lk: Adjust.
	* testsuite/libctf-writable/symtypetab-nonlinker-writeout.c: Fix
	initializer.
 2021-09-27 20:31:26 +01:00
Alan Modra
ad77db1c02 CC_FOR_TARGET et al 
The top level Makefile, the ld Makefile and others, define
CC_FOR_TARGET to be a compiler for the binutils target machine.  This
is the compiler that should be used for almost all tests with C
source.  There are _FOR_TARGET versions of CFLAGS, CXX, and CXXFLAGS
too.  This was all supposed to work with the testsuite .exp files
using CC for the target compiler, and CC_FOR_HOST for the host
compiler, with the makefiles passing CC=$CC_FOR_TARGET and
CC_FOR_HOST=$CC to the runtest invocation.

One exception to the rule of using CC_FOR_TARGET is the native-only ld
bootstrap test, which uses the newly built ld to link a copy of
itself.  Since the files being linked were created with the host
compiler, the boostrap test should use CC and CFLAGS, in case some
host compiler option provides needed libraries automatically.
However, bootstrap.exp used CC where it should have used CC_FOR_HOST.
I set about fixing that problem, then decided that playing games in
the makefiles with CC was a bad idea.  Not only is it confusing, but
other dejagnu code knows about CC_FOR_TARGET.  See dejagnu/target.exp.

So this patch gets rid of the makefile variable renaming and changes
all the .exp files to use the correct _FOR_TARGET variables.
CC_FOR_HOST and CFLAGS_FOR_HOST disappear.  A followup patch will
correct bootstrap.exp to use CFLAGS, and a number of other things I
noticed.

binutils/
	* testsuite/lib/binutils-common.exp (run_dump_test): Use
	CC_FOR_TARGET and CFLAGS_FOR_TARGET rather than CC and CFLAGS.
ld/
	* Makefile.am (check-DEJAGNU): Don't set CC to CC_FOR_TARGET
	and similar.  Pass variables with unchanged names.  Don't set
	CC_FOR_HOST or CFLAGS_FOR_HOST.
	* Makefile.in: Regenerate.
	* testsuite/config/default.exp: Update default CC and similar.
	(compiler_supports, plug_opt): Use CC_FOR_TARGET.
	* testsuite/ld-cdtest/cdtest.exp: Replace all uses of CC with
	CC_FOR_TARGET, and similarly for CFLAGS, CXX and CXXFLAGS.
	* testsuite/ld-auto-import/auto-import.exp: Likewise.
	* testsuite/ld-cygwin/exe-export.exp: Likewise.
	* testsuite/ld-elf/dwarf.exp: Likewise.
	* testsuite/ld-elf/indirect.exp: Likewise.
	* testsuite/ld-elf/shared.exp: Likewise.
	* testsuite/ld-elfcomm/elfcomm.exp: Likewise.
	* testsuite/ld-elfvers/vers.exp: Likewise.
	* testsuite/ld-elfvsb/elfvsb.exp: Likewise.
	* testsuite/ld-elfweak/elfweak.exp: Likewise.
	* testsuite/ld-gc/gc.exp: Likewise.
	* testsuite/ld-ifunc/ifunc.exp: Likewise.
	* testsuite/ld-mn10300/mn10300.exp: Likewise.
	* testsuite/ld-pe/pe-compile.exp: Likewise.
	* testsuite/ld-pe/pe-run.exp: Likewise.
	* testsuite/ld-pe/pe-run2.exp: Likewise.
	* testsuite/ld-pie/pie.exp: Likewise.
	* testsuite/ld-plugin/lto.exp: Likewise.
	* testsuite/ld-plugin/plugin.exp: Likewise.
	* testsuite/ld-scripts/crossref.exp: Likewise.
	* testsuite/ld-selective/selective.exp: Likewise.
	* testsuite/ld-sh/sh.exp: Likewise.
	* testsuite/ld-shared/shared.exp: Likewise.
	* testsuite/ld-srec/srec.exp: Likewise.
	* testsuite/ld-undefined/undefined.exp: Likewise.
	* testsuite/ld-unique/unique.exp: Likewise.
	* testsuite/ld-x86-64/tls.exp: Likewise.
	* testsuite/lib/ld-lib.exp: Likewise.
libctf/
	* Makefile.am (check-DEJAGNU): Don't set CC to CC_FOR_TARGET.
	Pass CC and CC_FOR_TARGET.  Don't set CC_FOR_HOST.
	* Makefile.in: Regenerate.
	* testsuite/config/default.exp: Update default CC and similar.
	* testsuite/lib/ctf-lib.exp (run_native_host_cmd): Use CC rather
	than CC_FOR_HOST.
	(run_lookup_test): Use CC_FOR_TARGET and CFLAGS_FOR_TARGET.
 2021-09-03 16:26:09 +09:30
Nick Alcock
ae064303ef libctf, ld: fix test results for upstream GCC 
The tests currently in binutils are aimed at the original GCC-based
implementation of CTF, which emitted CTF directly from GCC's internal
representation.  The approach now under review emits CTF from DWARF,
with an eye to eventually doing this for all non-DWARF debuginfo-like
formats GCC supports.  It also uses a different flag to enable
CTF emission (-gctf rather than -gt).

Adjust the testsuite accordingly.

Given that the ld testsuite results are dependent on type ordering,
which we do not guarantee at all, it's amazing how little changes. We
see a few type ordering differences, slices change because the old GCC
was buggy (slices were emitted "backwards", from the wrong end of the
machine word) and its expected results were wrong, and GCC now emits the
underlying integral type for enumerated types, though CTF has no way to
record this yet (coming in v4).

GCC also now emits even hidden symbols into the symtab (and thus
symtypetab), so one symtypetab test changes its expected results
slightly to compensate.

Also add tests for the CTF_K_UNKNOWN nonrepresentable type: this
couldn't be done before now since the only GCC that emits CTF_K_UNKNOWN
for nonrepresentable types is the new one.

ld/ChangeLog
2021-05-06  Nick Alcock  <nick.alcock@oracle.com>

	* testsuite/ld-ctf/ctf.exp: Use -gctf, not -gt.
	* testsuite/lib/ld-lib.exp: Likewise.
	* testsuite/ld-ctf/nonrepresentable-1.c: New test for nonrepresentable types.
	* testsuite/ld-ctf/nonrepresentable-2.c: Likewise.
	* testsuite/ld-ctf/nonrepresentable.d: Likewise.
	* testsuite/ld-ctf/array.d: Larger type section.
	* testsuite/ld-ctf/data-func-conflicted.d: Likewise.
	* testsuite/ld-ctf/enums.d: Likewise.
	* testsuite/ld-ctf/conflicting-enums.d: Don't compare types.
	* testsuite/ld-ctf/cross-tu-cyclic-conflicting.d: Changed type order.
	* testsuite/ld-ctf/cross-tu-noncyclic.d: Likewise.
	* testsuite/ld-ctf/slice.d: Adjust for improved slice emission.

libctf/ChangeLog
2021-05-06  Nick Alcock  <nick.alcock@oracle.com>

	* testsuite/lib/ctf-lib.exp: Use -gctf, not -gt.
	* testsuite/libctf-regression/nonstatic-var-section-ld-r.lk:
	Hidden symbols now get into the symtypetab anyway.
 2021-05-06 09:31:31 +01:00
Nick Alcock
15131809c2 libctf: fix memory leak in a test 
Harmless, but causes noise that makes it harder to spot other leaks.

libctf/ChangeLog
2021-03-25  Nick Alcock  <nick.alcock@oracle.com>

	* testsuite/libctf-writable/symtypetab-nonlinker-writeout.c: Don't
	leak buf.
 2021-03-25 16:32:50 +00:00
Nick Alcock
08c428aff4 libctf: eliminate dtd_u, part 5: structs / unions 
Eliminate the dynamic member storage for structs and unions as we have
for other dynamic types.  This is much like the previous enum
elimination, except that structs and unions are the only types for which
a full-sized ctf_type_t might be needed.  Up to now, this decision has
been made in the individual ctf_add_{struct,union}_sized functions and
duplicated in ctf_add_member_offset.  The vlen machinery lets us
simplify this, always allocating a ctf_lmember_t and setting the
dtd_data's ctt_size to CTF_LSIZE_SENT: we figure out whether this is
really justified and (almost always) repack things down into a
ctf_stype_t at ctf_serialize time.

This allows us to eliminate the dynamic member paths from the iterators and
query functions in ctf-types.c in favour of always using the large-structure
vlen stuff for dynamic types (the diff is ugly but that's just because of the
volume of reindentation this calls for).  This also means the large-structure
vlen stuff gets more heavily tested, which is nice because it was an almost
totally unused code path before now (it only kicked in for structures of size
>4GiB, and how often do you see those?)

The only extra complexity here is ctf_add_type.  Back in the days of the
nondeduplicating linker this was called a ridiculous number of times for
countless identical copies of structures: eschewing the repeated lookups of the
dtd in ctf_add_member_offset and adding the members directly saved an amazing
amount of time.  Now the nondeduplicating linker is gone, this is extreme
overoptimization: we can rip out the direct addition and use ctf_member_next and
ctf_add_member_offset, just like ctf_dedup_emit does.

We augment a ctf_add_type test to try adding a self-referential struct, the only
thing the ctf_add_type part of this change really perturbs.

This completes the elimination of dtd_u.

libctf/ChangeLog
2021-03-18  Nick Alcock  <nick.alcock@oracle.com>

	* ctf-impl.h (ctf_dtdef_t) <dtu_members>: Remove.
	<dtd_u>: Likewise.
	(ctf_dmdef_t): Remove.
	(struct ctf_next) <u.ctn_dmd>: Remove.
	* ctf-create.c (INITIAL_VLEN): New, more-or-less arbitrary initial
	vlen size.
	(ctf_add_enum): Use it.
	(ctf_dtd_delete): Do not free the (removed) dmd; remove string
	refs from the vlen on struct deletion.
	(ctf_add_struct_sized): Populate the vlen: do it by hand if
	promoting forwards.  Always populate the full-size
	lsizehi/lsizelo members.
	(ctf_add_union_sized): Likewise.
	(ctf_add_member_offset): Set up the vlen rather than the dmd.
	Expand it as needed, repointing string refs via
	ctf_str_move_pending. Add the member names as pending strings.
	Always populate the full-size lsizehi/lsizelo members.
	(membadd): Remove, folding back into...
	(ctf_add_type_internal): ... here, adding via an ordinary
	ctf_add_struct_sized and _next iteration rather than doing
	everything by hand.
	* ctf-serialize.c (ctf_copy_smembers): Remove this...
	(ctf_copy_lmembers): ... and this...
	(ctf_emit_type_sect): ... folding into here. Figure out if a
	ctf_stype_t is needed here, not in ctf_add_*_sized.
	(ctf_type_sect_size): Figure out the ctf_stype_t stuff the same
	way here.
	* ctf-types.c (ctf_member_next): Remove the dmd path and always
	use the vlen.  Force large-structure usage for dynamic types.
	(ctf_type_align): Likewise.
	(ctf_member_info): Likewise.
	(ctf_type_rvisit): Likewise.
	* testsuite/libctf-regression/type-add-unnamed-struct-ctf.c: Add a
	self-referential type to this test.
	* testsuite/libctf-regression/type-add-unnamed-struct.c: Adjusted
	accordingly.
	* testsuite/libctf-regression/type-add-unnamed-struct.lk: Likewise.
 2021-03-18 12:40:40 +00:00
Nick Alcock
77d724a7ec libctf: eliminate dtd_u, part 4: enums 
This is the first tricky one, the first complex multi-entry vlen
containing strings.  To handle this in vlen form, we have to handle
pending refs moving around on realloc.

We grow vlen regions using a new ctf_grow_vlen function, and iterate
through the existing enums every time a grow happens, telling the string
machinery the distance between the old and new vlen region and letting
it adjust the pending refs accordingly.  (This avoids traversing all
outstanding refs to find the refs that need adjusting, at the cost of
having to traverse one enum: an obvious major performance win.)

Addition of enums themselves (and also structs/unions later) is a bit
trickier than earlier forms, because the type might be being promoted
from a forward, and forwards have no vlen: so we have to spot that and
create it if needed.

Serialization of enums simplifies down to just telling the string
machinery about the string refs; all the enum type-lookup code loses all
its dynamic member lookup complexity entirely.

A new test is added that iterates over (and gets values of) an enum with
enough members to force a round of vlen growth.

libctf/ChangeLog
2021-03-18  Nick Alcock  <nick.alcock@oracle.com>

	* ctf-impl.h (ctf_dtdef_t) <dtd_vlen_alloc>: New.
	(ctf_str_move_pending): Declare.
	* ctf-string.c (ctf_str_add_ref_internal): Fix error return.
	(ctf_str_move_pending): New.
	* ctf-create.c (ctf_grow_vlen): New.
	(ctf_dtd_delete): Zero out the vlen_alloc after free.  Free the
	vlen later: iterate over it and free enum name refs first.
	(ctf_add_generic): Populate dtd_vlen_alloc from vlen.
	(ctf_add_enum): populate the vlen; do it by hand if promoting
	forwards.
	(ctf_add_enumerator): Set up the vlen rather than the dmd.  Expand
	it as needed, repointing string refs via ctf_str_move_pending. Add
	the enumerand names as pending strings.
	* ctf-serialize.c (ctf_copy_emembers): Remove.
	(ctf_emit_type_sect): Copy the vlen into place and ref the
	strings.
	* ctf-types.c (ctf_enum_next): The dynamic portion now uses
	the same code as the non-dynamic.
	(ctf_enum_name): Likewise.
	(ctf_enum_value): Likewise.
	* testsuite/libctf-lookup/enum-many-ctf.c: New test.
	* testsuite/libctf-lookup/enum-many.lk: New test.
 2021-03-18 12:40:40 +00:00
Nick Alcock
986e9e3aa0 libctf: do not corrupt strings across ctf_serialize 
The preceding change revealed a new bug: the string table is sorted for
better compression, so repeated serialization with type (or member)
additions in the middle can move strings around.  But every
serialization flushes the set of refs (the memory locations that are
automatically updated with a final string offset when the strtab is
updated), so if we are not to have string offsets go stale, we must do
all ref additions within the serialization code (which walks the
complete set of types and symbols anyway). Unfortunately, we were adding
one ref in another place: the type name in the dynamic type definitions,
which has a ref added to it by ctf_add_generic.

So adding a type, serializing (via, say, one of the ctf_write
functions), adding another type with a name that sorts earlier, and
serializing again will corrupt the name of the first type because it no
longer had a ref pointing to its dtd entry's name when its string offset
was shifted later in the strtab to mae way for the other type.

To ensure that we don't miss strings, we also maintain a set of *pending
refs* that will be added later (during serialization), and remove
entries from that set when the ref is finally added.  We always use
ctf_str_add_pending outside ctf-serialize.c, ensure that ctf_serialize
adds all strtab offsets as refs (even those in the dtds) on every
serialization, and mandate that no refs are live on entry to
ctf_serialize and that all pending refs are gone before strtab
finalization.  (Of necessity ctf_serialize has to traverse all strtab
offsets in the dtds in order to serialize them, so adding them as refs
at the same time is easy.)

(Note that we still can't erase unused atoms when we roll back, though
we can erase unused refs: members and enums are still not removed by
rollbacks and might reference strings added after the snapshot.)

libctf/ChangeLog
2021-03-18  Nick Alcock  <nick.alcock@oracle.com>

	* ctf-hash.c (ctf_dynset_elements): New.
	* ctf-impl.h (ctf_dynset_elements): Declare it.
	(ctf_str_add_pending): Likewise.
	(ctf_dict_t) <ctf_str_pending_ref>: New, set of refs that must be
	added during serialization.
	* ctf-string.c (ctf_str_create_atoms): Initialize it.
	(CTF_STR_ADD_REF): New flag.
	(CTF_STR_MAKE_PROVISIONAL): Likewise.
	(CTF_STR_PENDING_REF): Likewise.
	(ctf_str_add_ref_internal): Take a flags word rather than int
	params.  Populate, and clear out, ctf_str_pending_ref.
	(ctf_str_add): Adjust accordingly.
	(ctf_str_add_external): Likewise.
	(ctf_str_add_pending): New.
	(ctf_str_remove_ref): Also remove the potential ref if it is a
	pending ref.
	* ctf-serialize.c (ctf_serialize): Prohibit addition of strings
	with ctf_str_add_ref before serialization.  Ensure that the
	ctf_str_pending_ref set is empty before strtab finalization.
	(ctf_emit_type_sect): Add a ref to the ctt_name.
	* ctf-create.c (ctf_add_generic): Add the ctt_name as a pending
	ref.
	* testsuite/libctf-writable/reserialize-strtab-corruption.*: New test.
 2021-03-18 12:40:40 +00:00
Nick Alcock
087945261c libctf: fix some tabdamage and move some code around 
ctf-link.c is unnecessarily confusing because ctf_link_lazy_open is
positioned near functions that have nothing to do with opening files.

Move it around, and fix some tabdamage that's crept in lately.

libctf/ChangeLog
2021-03-18  Nick Alcock  <nick.alcock@oracle.com>

	* ctf-link.c (ctf_link_lazy_open): Move up in the file, to near
	ctf_link_add_ctf.
	* ctf-lookup.c (ctf_lookup_symbol_idx): Repair tabdamage.
	(ctf_lookup_by_sym_or_name): Likewise.
	* testsuite/libctf-lookup/struct-iteration.c: Likewise.
	* testsuite/libctf-regression/type-add-unnamed-struct.c: Likewise.
 2021-03-18 12:37:52 +00:00
Nick Alcock
f4f60336da libctf, include: find types of symbols by name 
The existing ctf_lookup_by_symbol and ctf_arc_lookup_symbol functions
suffice to look up the types of symbols if the caller already has a
symbol number.  But the caller often doesn't have one of those and only
knows the name of the symbol: also, in object files, the caller might
not have a useful symbol number in any sense (and neither does libctf:
the 'symbol number' we use in that case literally starts at 0 for the
lexicographically first-sorted symbol in the symtypetab and counts those
symbols, so it corresponds to nothing useful).

This means that even though object files have a symtypetab (generated by
the compiler or by ld -r), the only way we can look up anything in it is
to iterate over all symbols in turn with ctf_symbol_next until we find
the one we want.

This is unhelpful and pointlessly inefficient.

So add a pair of functions to look up symbols by name in a dict and in a
whole archive: ctf_lookup_by_symbol_name and ctf_arc_lookup_symbol_name.
These are identical to the existing functions except that they take
symbol names rather than symbol numbers.

To avoid insane repetition, we do some refactoring in the process, so
that both ctf_lookup_by_symbol and ctf_arc_lookup_symbol turn into thin
wrappers around internal functions that do both lookup by symbol index
and lookup by name.  This massively reduces code duplication because
even the existing lookup-by-index stuff wants to use a name sometimes
(when looking up in indexed sections), and the new lookup-by-name stuff
has to turn it into an index sometimes (when looking up in non-indexed
sections): doing it this way lets us share most of that.

The actual name->index lookup is done by ctf_lookup_symbol_idx.  We do
not anticipate this lookup to be as heavily used as ld.so symbol lookup
by many orders of magnitude, so using the ELF symbol hashes would
probably take more time to read them than is saved by using the hashes,
and it adds a lot of complexity.  Instead, do a linear search for the
symbol name, caching all the name -> index mappings as we go, so that
future searches are likely to hit in the cache.  To avoid having to
repeat this search over and over in a CTF archive when
ctf_arc_lookup_symbol_name is used, have cached archive lookups (the
sort done by ctf_arc_lookup_symbol* and the ctf_archive_next iterator)
pick out the first dict they cache in a given archive and store it in a
new ctf_archive field, ctfi_crossdict_cache.  This can be used to store
cross-dictionary cached state that depends on things like the ELF symbol
table rather than the contents of any one dict.  ctf_lookup_symbol_idx
then caches its name->index mappings in the dictionary named in the
crossdict cache, if any, so that ctf_lookup_symbol_idx in other dicts
in the same archive benefit from the previous linear search, and the
symtab only needs to be scanned at most once.

(Note that if you call ctf_lookup_by_symbol_name in one specific dict,
and then follow it with a ctf_arc_lookup_symbol_name, the former will
not use the crossdict cache because it's only populated by the dict
opens in ctf_arc_lookup_symbol_name. This is harmless except for a small
one-off waste of memory and time: it's only a cache, after all.  We can
fix this later by using the archive caching machinery more
aggressively.)

In ctf-archive, we do similar things, turning ctf_arc_lookup_symbol into
a wrapper around a new function that does both index -> ID and name ->
ID lookups across all dicts in an archive.  We add a new
ctfi_symnamedicts cache that maps symbol names to the ctf_dict_t * that
it was found in (so that linear searches for symbols don't need to be
repeated): but we also *remove* a cache, the ctfi_syms cache that was
memoizing the actual ctf_id_t returned from every call to
ctf_arc_lookup_symbol.  This is pointless: all it saves is one call to
ctf_lookup_by_symbol, and that's basically an array lookup and nothing
more so isn't worth caching.  (Equally, given that symbol -> index
mappings are cached by ctf_lookup_by_symbol_name, those calls are nearly
free after the first call, so there's no point caching the ctf_id_t in
that case either.)

We fix up one test that was doing manual symbol lookup to use
ctf_arc_lookup_symbol instead, and enhance it to check that the caching
layer is not totally broken: we also add a new test to do lookups in a
.o file, and another to do lookups in an archive with conflicted types
and make sure that sort of multi-dict lookup is actually working.

include/ChangeLog
2021-02-17  Nick Alcock  <nick.alcock@oracle.com>

	* ctf-api.h (ctf_arc_lookup_symbol_name): New.
	(ctf_lookup_by_symbol_name): Likewise.

libctf/ChangeLog
2021-02-17  Nick Alcock  <nick.alcock@oracle.com>

	* ctf-impl.h (ctf_dict_t) <ctf_symhash>: New.
	<ctf_symhash_latest>: Likewise.
	(struct ctf_archive_internal) <ctfi_crossdict_cache>: New.
	<ctfi_symnamedicts>: New.
	<ctfi_syms>: Remove.
	(ctf_lookup_symbol_name): Remove.
	* ctf-lookup.c (ctf_lookup_symbol_name): Propagate errors from
	parent properly.  Make static.
	(ctf_lookup_symbol_idx): New, linear search for the symbol name,
	cached in the crossdict cache's ctf_symhash (if available), or
	this dict's (otherwise).
	(ctf_try_lookup_indexed): Allow the symname to be passed in.
	(ctf_lookup_by_symbol): Turn into a wrapper around...
	(ctf_lookup_by_sym_or_name): ... this, supporting name lookup too,
	using ctf_lookup_symbol_idx in non-writable dicts.  Special-case
	name lookup in dynamic dicts without reported symbols, which have
	no symtab or dynsymidx but where name lookup should still work.
	(ctf_lookup_by_symbol_name): New, another wrapper.
	* ctf-archive.c (enosym): Note that this is present in
	ctfi_symnamedicts too.
	(ctf_arc_close): Adjust for removal of ctfi_syms.  Free the
	ctfi_symnamedicts.
	(ctf_arc_flush_caches): Likewise.
	(ctf_dict_open_cached): Memoize the first cached dict in the
	crossdict cache.
	(ctf_arc_lookup_symbol): Turn into a wrapper around...
	(ctf_arc_lookup_sym_or_name): ... this.  No longer cache
	ctf_id_t lookups: just call ctf_lookup_by_symbol as needed (but
	still cache the dicts those lookups succeed in).  Add
	lookup-by-name support, with dicts of successful lookups cached in
	ctfi_symnamedicts.  Refactor the caching code a bit.
	(ctf_arc_lookup_symbol_name): New, another wrapper.
	* ctf-open.c (ctf_dict_close): Free the ctf_symhash.
	* libctf.ver (LIBCTF_1.2): New version.  Add
	ctf_lookup_by_symbol_name, ctf_arc_lookup_symbol_name.
	* testsuite/libctf-lookup/enum-symbol.c (main): Use
	ctf_arc_lookup_symbol rather than looking up the name ourselves.
	Fish it out repeatedly, to make sure that symbol caching isn't
	broken.
	(symidx_64): Remove.
	(symidx_32): Remove.
	* testsuite/libctf-lookup/enum-symbol-obj.lk: Test symbol lookup
	in an unlinked object file (indexed symtypetab sections only).
	* testsuite/libctf-writable/symtypetab-nonlinker-writeout.c
	(try_maybe_reporting): Check symbol types via
	ctf_lookup_by_symbol_name as well as ctf_symbol_next.
	* testsuite/libctf-lookup/conflicting-type-syms.*: New test of
	lookups in a multi-dict archive.
 2021-02-20 16:37:08 +00:00
Alan Modra
c3bf9dc5aa Include ld-lib.exp from ctf-lib.exp 
* testsuite/config/default.exp (ld_L_opt): Define.
	* testsuite/lib/ctf-lib.exp (load_common_lib): Delete.  Instead load
	ld-lib.exp.
	(run_host_cmd, run_host_cmd_yesno, check_compiler_available): Delete.
	(compile_one_cc, check_ctf_available): Delete.
 2021-02-20 18:23:49 +10:30
Nick Alcock
35a01a0454 libctf, ld: fix symtypetab and var section population under ld -r 
The variable section in a CTF dict is meant to contain the types of
variables that do not appear in the symbol table (mostly file-scope
static declarations).  We implement this by having the compiler emit
all potential data symbols into both sections, then delete those
symbols from the variable section that correspond to data symbols the
linker has reported.

Unfortunately, the check for this in ctf_serialize is wrong: rather than
checking the set of linker-reported symbols, we check the set of names
in the data object symtypetab section: if the linker has reported no
symbols at all (usually if ld -r has been run, or if a non-linker
program that does not use symbol tables is calling ctf_link) this will
include every single symbol, emptying the variable section completely.

Worse, when ld -r is in use, we want to force writeout of every
symtypetab entry on the inputs, in an indexed section, whether or not
the linker has reported them, since this isn't a final link yet and the
symbol table is not finalized (and may grow more symbols than the linker
has yet reported).  But the check for this is flawed too: we were
relying on ctf_link_shuffle_syms not having been called if no symbols
exist, but that function is *always* called by ld even when ld -r is in
use: ctf_link_add_linker_symbol is the one that's not called when there
are no symbols.

We clearly need to rethink this.  Using the emptiness of the set of
reported symbols as a test for ld -r is just ugly: the linker already
knows if ld -r is underway and can just tell us.  So add a new linker
flag CTF_LINK_NO_FILTER_REPORTED_SYMS that is set to stop the linker
filtering the symbols in the symtypetab sections using the set that the
linker has reported: use the presence or absence of this flag to
determine whether to emit unindexed symtabs: we only remove entries from
the variable section when filtering symbols, and we only remove them if
they are in the reported symbol set, fixing the case where no symbols
are reported by the linker at all.

(The negative sense of the new CTF_LINK flag is intentional: the common
case, both for ld and for simple tools that want to do a ctf_link with
no ELF symbol table in sight, is probably to filter out symbols that no
linker has reported: i.e., for the simple tools, all of them.)

There's another wrinkle, though.  It is quite possible for a non-linker
to add symbols to a dict via ctf_add_*_sym and then write it out via the
ctf_write APIs: perhaps it's preparing a dict for a later linker
invocation.  Right now this would not lead to anything terribly
meaningful happening: ctf_serialize just assumes it was called via
ctf_link if symbols are present.  So add an (internal-to-libctf) flag
that indicates that a writeout is happening via ctf_link_write, and set
it there (propagating it to child dicts as needed).  ctf_serialize can
then spot when it is not being called by a linker, and arrange to always
write out an indexed, sorted symtypetab for fastest possible future
symbol lookup by name in that case.  (The writeouts done by ld -r are
unsorted, because the only thing likely to use those symtabs is the
linker, which doesn't benefit from symtypetab sorting.)

Tests added for all three linking cases (ld -r, ld -shared, ld), with a
bit of testsuite framework enhancement to stop it unconditionally
linking the CTF to be checked by the lookup program with -shared, so
tests can now examine CTF linked with -r or indeed with no flags at all,
though the output filename is still foo.so even in this case.

Another test added for the non-linker case that endeavours to determine
whether the symtypetab is sorted by examining the order of entries
returned from ctf_symbol_next: nobody outside libctf should rely on
this ordering, but this test is not outside libctf :)

include/ChangeLog
2021-01-26  Nick Alcock  <nick.alcock@oracle.com>

	* ctf-api.h (CTF_LINK_NO_FILTER_REPORTED_SYMS): New.

ld/ChangeLog
2021-01-26  Nick Alcock  <nick.alcock@oracle.com>

	* ldlang.c (lang_merge_ctf): Set CTF_LINK_NO_FILTER_REPORTED_SYMS
	when appropriate.

libctf/ChangeLog
2021-01-27  Nick Alcock  <nick.alcock@oracle.com>

	* ctf-impl.c (_libctf_nonnull_): Add parameters.
	(LCTF_LINKING): New flag.
	(ctf_dict_t) <ctf_link_flags>: Mention it.
	* ctf-link.c (ctf_link): Keep LCTF_LINKING set across call.
	(ctf_write): Likewise, including in child dictionaries.
	(ctf_link_shuffle_syms): Make sure ctf_dynsyms is NULL if there
	are no reported symbols.
	* ctf-create.c (symtypetab_delete_nonstatic_vars): Make sure
	the variable has been reported as a symbol by the linker.
	(symtypetab_skippable): Mention relationship between SYMFP and the
	flags.
	(symtypetab_density): Adjust nonnullity.  Exit early if no symbols
	were reported and force-indexing is off (i.e., we are doing a
	final link).
	(ctf_serialize): Handle the !LCTF_LINKING case by writing out an
	indexed, sorted symtypetab (and allow SYMFP to be NULL in this
	case).  Turn sorting off if this is a non-final link.  Only delete
	nonstatic vars if we are filtering symbols and the linker has
	reported some.
	* testsuite/libctf-regression/nonstatic-var-section-ld-r*:
	New test of variable and symtypetab section population when
	ld -r is used.
	* testsuite/libctf-regression/nonstatic-var-section-ld-executable.lk:
	Likewise, when ld of an executable is used.
	* testsuite/libctf-regression/nonstatic-var-section-ld.lk:
	Likewise, when ld -shared alone is used.
	* testsuite/libctf-regression/nonstatic-var-section-ld*.c:
	Lookup programs for the above.
	* testsuite/libctf-writable/symtypetab-nonlinker-writeout.*: New
	test, testing survival of symbols across ctf_write paths.
	* testsuite/lib/ctf-lib.exp (run_lookup_test): New option,
	nonshared, suppressing linking of the SOURCE with -shared.
 2021-02-04 16:01:53 +00:00
Nick Alcock
26503e2f5e libctf, create: fix ctf_type_add of structs with unnamed members 
Our recent commit to support unnamed structure members better ditched
the old ctf_member_iter iterator body in favour of ctf_member_next.
However, these functions treat unnamed structure members differently:
ctf_member_iter just returned whatever the internal representation
contained, while ctf_member_next took care to always return "" rather
than sometimes returning "" and sometimes NULL depending on whether the
dict was dynamic (a product of ctf_create) or not (a product of
ctf_open).  After this commit, ctf_member_iter did the same.

It was always a bug for external callers not to treat a "" return from
these functions as if it were NULL, so only buggy callers could be
affected -- but one of those buggy callers was ctf_add_type, which
assumed that it could just take whatever name was returned from
ctf_member_iter and slam it directly into the internal representation of
a dynamic dict -- which expects NULL for unnamed members, not "".  The
net effect of all of this is that taking a struct containing unnamed
members and ctf_add_type'ing it into a dynamic dict produced a dict
whose unnamed members were inaccessible to ctf_member_info (though if
you wrote that dict out and then ctf_open'ed it, they would magically
reappear again).

Compensate for this by suitably transforming a "" name into NULL in the
internal representation, as should have been done all along.

libctf/ChangeLog
2021-01-19  Nick Alcock  <nick.alcock@oracle.com>

	* ctf-create.c (membadd): Transform ""-named members into
	NULL-named ones.
	* testsuite/libctf-regression/type-add-unnamed-struct*: New test.
 2021-01-19 12:45:20 +00:00
Nick Alcock
e05a3e5a49 libctf: lookup_by_name: do not return success for nonexistent pointer types 
The recent work allowing lookups of pointers in child dicts when the
pointed-to type is in the parent dict broke the case where a pointer
type that does not exist at all is looked up: we mistakenly return the
pointed-to type, which is likely not a pointer at all.  This causes
considerable confusion.

Fixed, with a new testcase.

libctf/ChangeLog
2021-01-19  Nick Alcock  <nick.alcock@oracle.com>

	* ctf-lookup.c (ctf_lookup_by_name_internal): Do not return the
	base type if looking up a nonexistent pointer type.
	* testsuite/libctf-regression/pptrtab*: Test it.
 2021-01-19 12:45:19 +00:00
Nick Alcock
0814dbfbfc libctf, testsuite: adjust for real return type of ctf_member_count 
This returns an int, not a long int or an ssize_t (as one test was
inconsistently assuming).

libctf/ChangeLog
2021-01-05  Nick Alcock  <nick.alcock@oracle.com>

	* testsuite/libctf-lookup/struct-iteration.c (main):
	ctf_member_count returns an int.
 2021-01-05 19:34:56 +00:00
Nick Alcock
70d3120f32 libctf, testsuite: don't run without a suitable compiler 
We never actually check to see if the compiler supports CTF,
or even if a suitable compiler exists.

libctf/ChangeLog
2021-01-05  Nick Alcock  <nick.alcock@oracle.com>

	* Makefile.am (BASEDIR): New.
	(BFDDIR): Likewise.
	(check-DEJAGNU): Add development.exp to prerequisites.
	(development.exp): New.
	(CONFIG_STATUS_DEPENDENCIES): New.
	(EXTRA_DEJAGNU_SITE_CONFIG): Likewise.
	(DISTCLEANFILES): Likewise.
	* Makefile.in: Regenerated.
	* testsuite/lib/ctf-lib.exp (check_ctf_available): Return boolean.
	* testsuite/libctf-lookup/lookup.exp: Call check_ctf_available.
	* testsuite/libctf-regression/regression.exp: Likewise.
 2021-01-05 17:11:20 +00:00
Nick Alcock
abe4ca69a1 libctf: fix lookups of pointers by name in parent dicts 
When you look up a type by name using ctf_lookup_by_name, in most cases
libctf can just strip off any qualifiers and look for the name, but for
pointer types this doesn't work, since the caller will want the pointer
type itself.  But pointer types are nameless, and while they cite the
types they point to, looking up a type by name requires a link going the
*other way*, from the type pointed to to the pointer type that points to
it.

libctf has always built this up at open time: ctf_ptrtab is an array of
type indexes pointing from the index of every type to the index of the
type that points to it.  But because it is built up at open time (and
because it uses type indexes and not type IDs) it is restricted to
working within a single dict and ignoring parent/child
relationships. This is normally invisible, unless you manage to get a
dict with a type in the parent but the only pointer to it in a child.
The ctf_ptrtab will not track this relationship, so lookups of this
pointer type by name will fail.  Since which type is in the parent and
which in the child is largely opaque to the user (which goes where is up
to the deduplicator, and it can and does reshuffle things to save
space), this leads to a very bad user experience, with an
obviously-visible pointer type which ctf_lookup_by_name claims doesn't
exist.

The fix is to have another array, ctf_pptrtab, which is populated in
child dicts: like the parent's ctf_ptrtab, it has one element per type
in the parent, but is all zeroes except for those types which are
pointed to by types in the child: so it maps parent dict indices to
child dict indices.  The array is grown, and new child types scanned,
whenever a lookup happens and new types have been added to the child
since the last time a lookup happened that might need the pptrtab.
(So for non-writable dicts, this only happens once, since new types
cannot be added to non-writable dicts at all.)

Since this introduces new complexity (involving updating only part of
the ctf_pptrtab) which is only seen when a writable dict is in use, we
introduce a new libctf-writable testsuite that contains lookup tests
with no corresponding CTF-containing .c files (which can thus be run
even on platforms with no .ctf-section support in the linker yet), and
add a test to check that creation of pointers in children to types in
parents and a following lookup by name works as expected.  The non-
writable case is tested in a new libctf-regression testsuite which is
used to track now-fixed outright bugs in libctf.

libctf/ChangeLog
2021-01-05  Nick Alcock  <nick.alcock@oracle.com>

	* ctf-impl.h (ctf_dict_t) <ctf_pptrtab>: New.
	<ctf_pptrtab_len>: New.
	<ctf_pptrtab_typemax>: New.
	* ctf-create.c (ctf_serialize): Update accordingly.
	(ctf_add_reftype): Note that we don't need to update pptrtab here,
	despite updating ptrtab.
	* ctf-open.c (ctf_dict_close): Destroy the pptrtab.
	(ctf_import): Likewise.
	(ctf_import_unref): Likewise.
	* ctf-lookup.c (grow_pptrtab): New.
	(refresh_pptrtab): New, update a pptrtab.
	(ctf_lookup_by_name): Turn into a wrapper around (and rename to)...
	(ctf_lookup_by_name_internal): ... this: construct the pptrtab, and
	use it in addition to the parent's ptrtab when parent dicts are
	searched.
	* testsuite/libctf-regression/regression.exp: New testsuite for
	regression tests.
	* testsuite/libctf-regression/pptrtab*: New test.
	* testsuite/libctf-writable/writable.exp: New testsuite for tests of
	writable CTF dicts.
	* testsuite/libctf-writable/pptrtab*: New test.
 2021-01-05 14:53:40 +00:00
Nick Alcock
6c3a38777b libctf, include: support unnamed structure members better 
libctf has no intrinsic support for the GCC unnamed structure member
extension.  This principally means that you can't look up named members
inside unnamed struct or union members via ctf_member_info: you have to
tiresomely find out the type ID of the unnamed members via iteration,
then look in each of these.

This is ridiculous.  Fix it by extending ctf_member_info so that it
recurses into unnamed members for you: this is still unambiguous because
GCC won't let you create ambiguously-named members even in the presence
of this extension.

For consistency, and because the release hasn't happened and we can
still do this, break the ctf_member_next API and add flags: we specify
one flag, CTF_MN_RECURSE, which if set causes ctf_member_next to
automatically recurse into unnamed members for you, returning not only
the members themselves but all their contained members, so that you can
use ctf_member_next to identify every member that it would be valid to
call ctf_member_info with.

New lookup tests are added for all of this.

include/ChangeLog
2021-01-05  Nick Alcock  <nick.alcock@oracle.com>

	* ctf-api.h (CTF_MN_RECURSE): New.
	(ctf_member_next): Add flags argument.

libctf/ChangeLog
2021-01-05  Nick Alcock  <nick.alcock@oracle.com>

	* ctf-impl.h (struct ctf_next) <u.ctn_next>: Move to...
	<ctn_next>: ... here.
	* ctf-util.c (ctf_next_destroy): Unconditionally destroy it.
	* ctf-lookup.c (ctf_symbol_next): Adjust accordingly.
	* ctf-types.c (ctf_member_iter): Reimplement in terms of...
	(ctf_member_next): ... this.  Support recursive unnamed member
	iteration (off by default).
	(ctf_member_info): Look up members in unnamed sub-structs.
	* ctf-dedup.c (ctf_dedup_rhash_type): Adjust ctf_member_next call.
	(ctf_dedup_emit_struct_members): Likewise.
	* testsuite/libctf-lookup/struct-iteration-ctf.c: Test empty unnamed
	members, and a normal member after the end.
	* testsuite/libctf-lookup/struct-iteration.c: Verify that
	ctf_member_count is consistent with the number of successful returns
	from a non-recursive ctf_member_next.
	* testsuite/libctf-lookup/struct-iteration-*: New, test iteration
	over struct members.
	* testsuite/libctf-lookup/struct-lookup.c: New test.
	* testsuite/libctf-lookup/struct-lookup.lk: New test.
 2021-01-05 14:53:40 +00:00
Nick Alcock
9bc769718d libctf: new test of enum lookups with the _next iterator 
I had reports that this doesn't work.  This test shows it working (and
also shows how annoying it is to do symbol lookup by name with the
present API: we need a ctf_arc_lookup_symbol_name for users that don't
already have a symtab handy).

libctf/ChangeLog
2021-01-05  Nick Alcock  <nick.alcock@oracle.com>

	* testsuite/libctf-lookup/enum-symbol.lk: New symbol-lookup test.
	* testsuite/libctf-lookup/enum-symbol-ctf.c: New CTF input.
	* testsuite/libctf-lookup/enum-symbol.c: New lookup test.
 2021-01-05 14:53:40 +00:00
Nick Alcock
c59e30ed17 libctf: new testsuite 
This introduces a new lookup testsuite under libctf, which operates by
compiling (with libtool) a "lookup" .c file that uses libctf to analyze
some other program, then compiling some number of test object files with
CTF and optionally linking them together and running the lookup program
on the test object files (or linked test binary), before diffing the
result much as run_dump_test does.

This lets us test the portions of libctf that are not previously
testable, notably the portions that do lookup on linked output and
that create dynamic dictionaries and then do lookup on them before
writing them out, something that is not tested by the ld-ctf testsuite
because the linker never does this.

A couple of simple tests are added: one testing the functionality of
enum lookups, and one testing that the recently-added commit adding
extra paranoia to incomplete type handling doesn't break linking and
that the result of the link is an (otherwise-impossible) array of
forward type in the shared CTF dict.

ChangeLog
2021-01-05  Nick Alcock  <nick.alcock@oracle.com>

	* Makefile.def (libctf): No longer no_check.  Checking depends on
	all-ld.
	* Makefile.in: Regenerated.

libctf/ChangeLog
2021-01-05  Nick Alcock  <nick.alcock@oracle.com>

	* Makefile.am (EXPECT): New.
	(RUNTEST): Likewise.
	(RUNTESTFLAGS): Likewise.
	(CC_FOR_TARGET): Likewise.
	(check-DEJAGNU): Likewise.
	(AUTOMAKE_OPTIONS): Add dejagnu.
	* Makefile.in: Regenerated.
	* testsuite/config/default.exp: New.
	* testsuite/lib/ctf-lib.exp: Likewise.
	* testsuite/libctf-lookup/enum.lk: New test.
	* testsuite/libctf-lookup/enum-ctf.c: New CTF input.
	* testsuite/libctf-lookup/enum.c: New lookup test.
	* testsuite/libctf-lookup/ambiguous-struct*.c: New test.
	* testsuite/libctf-lookup/lookup.exp: New.
 2021-01-05 14:53:40 +00:00