When you link TUs that contain conflicting types together, the resulting
CTF section is an archive containing many CTF dicts. These dicts appear
in ctf_link_outputs of the shared dict, with each ctf_import'ing that
shared dict. ctf_importing a dict bumps its refcount to stop it going
away while it's in use -- but if the shared dict (whose refcount is
bumped) has the child dict (doing the bumping) in its ctf_link_outputs,
we have a refcount loop, since the child dict only un-ctf_imports and
drops the parent's refcount when it is freed, but the child is only
freed when the parent's refcount falls to zero.
(In the future, this will be able to go wrong on the inputs too, when an
ld -r'ed deduplicated output with conflicts is relinked. Right now this
cannot happen because we don't ctf_import such dicts at all. This will
be fixed in a later commit in this series.)
Fix this by introducing an internal-use-only ctf_import_unref function
that imports a parent dict *witthout* bumping the parent's refcount, and
using it when we create per-CU outputs. This function is only safe to
use if you know the parent cannot go away while the child exists: but if
the parent *owns* the child, as here, this is necessarily true.
Record in the ctf_file_t whether a parent was imported via ctf_import or
ctf_import_unref, so that if you do another ctf_import later on (or a
ctf_import_unref) it can decide whether to drop the refcount of the
existing parent being replaced depending on which function you used to
import that one. Adjust ctf_serialize so that rather than doing a
ctf_import (which is wrong if the original import was
ctf_import_unref'fed), we just copy the parent field and refcount over
and forcibly flip the unref flag on on the old copy we are going to
discard.
ctf_file_close also needs a bit of tweaking to only close the parent if
it was not imported with ctf_import_unref: while we're at it, guard
against repeated closes with a refcount of zero and stop them causing
double-frees, even if destruction of things freed *inside*
ctf_file_close cause such recursion.
Verified no leaks or accesses to freed memory after all of this with
valgrind. (It was leak-happy before.)
libctf/
* ctf-impl.c (ctf_file_t) <ctf_parent_unreffed>: New.
(ctf_import_unref): New.
* ctf-open.c (ctf_file_close) Drop the refcount all the way to
zero. Don't recurse back in if the refcount is already zero.
(ctf_import): Check ctf_parent_unreffed before deciding whether
to close a pre-existing parent. Set it to zero.
(ctf_import_unreffed): New, as above, setting
ctf_parent_unreffed to 1.
* ctf-create.c (ctf_serialize): Do not ctf_import into the new
child: use direct assignment, and set unreffed on the new and
old children.
* ctf-link.c (ctf_create_per_cu): Import the parent using
ctf_import_unreffed.
The name was just annoyingly long and I kept misspelling it.
It's also a bad name: it's not a mapping the type might be *used* in a
type mapping, but it is itself a representation of a type (a ctf_file_t
/ ctf_id_t pair), not of a mapping at all.
libctf/
* ctf-impl.h (ctf_link_type_mapping_key): Rename to...
(ctf_link_type_key): ... this, adjusting member prefixes to
match.
(ctf_hash_type_mapping_key): Rename to...
(ctf_hash_type_key): ... this.
(ctf_hash_eq_type_mapping_key): Rename to...
(ctf_hash_eq_type_key): ... this.
* ctf-hash.c (ctf_hash_type_mapping_key): Rename to...
(ctf_hash_type_key): ... this, and adjust for member name
changes.
(ctf_hash_eq_type_mapping_key): Rename to...
(ctf_hash_eq_type_key): ... this, and adjust for member name
changes.
* ctf-link.c (ctf_add_type_mapping): Adjust. Note the lack of
need for out-of-memory checking in this code.
(ctf_type_mapping): Adjust.
We've been using this for all of libctf's history in binutils: we should
check for it in configure.
libctf/
configure.ac: Check for vasprintf.
configure: Regenerated.
config.h.in: Likewise.
This is a perfectly possible case, and half of ctf_bfdopen_ctfsect
handled it fine. The other half hit a divide by zero or two before we
got that far, and had no code path to load the strtab from anywhere
in the absence of a symtab to point at it in any case.
So, as a fallback, if there is no symtab, try loading ".strtab"
explicitly by name, like we used to before we started looking for the
strtab the symtab used.
Of course, such a strtab is not kept hold of by BFD, so this means we
have to bring back the code to possibly explicitly free the strtab that
we read in.
libctf/
* ctf-impl.h (struct ctf_archive_internal) <ctfi_free_strsect>
New.
* ctf-open-bfd.c (ctf_bfdopen_ctfsect): Explicitly open a strtab
if the input has no symtab, rather than dividing by
zero. Arrange to free it later via ctfi_free_ctfsect.
* ctf-archive.c (ctf_new_archive_internal): Do not
ctfi_free_strsect by default.
(ctf_arc_close): Possibly free it here.
Now that we can have slices of anything terminating in an int, we must
dump things accordingly, or slices of typedefs appear as
c5b: __u8 -> 16c: __u8 -> 78: short unsigned int (size 0x2)
which is unhelpful. If things *are* printed as slices, the name is
missing:
a15: [slice 0x8:0x4]-> 16c: __u8 -> 78: short unsigned int (size 0x2)
And struct members give no clue they're a slice at all, which is a shame
since bitfields are the major use of this type kind:
[0x8] (ID 0xa15) (kind 10) __u8 dst_reg
Fix things so that everything slicelike or integral gets its encoding
printed, and everything with a name gets the name printed:
a15: __u8 [slice 0x8:0x4] (size 0x1) -> 1ff: __u8 (size 0x1) -> 37: unsigned char [0x0:0x8] (size 0x1)
[0x0] (ID 0xa15) (kind 10) __u8:4 (aligned at 0x1, format 0x2, offset:bits 0x8:0x4)
Bitfield struct members get a technically redundant but much
easier-to-understand dumping now:
[0x0] (ID 0x80000005) (kind 6) struct bpf_insn (aligned at 0x1)
[0x0] (ID 0x222) (kind 10) __u8 code (aligned at 0x1)
[0x8] (ID 0x1e9e) (kind 10) __u8 dst_reg:4 (aligned at 0x1, format 0x2, offset:bits 0x8:0x4)
[0xc] (ID 0x1e46) (kind 10) __u8 src_reg:4 (aligned at 0x1, format 0x2, offset:bits 0xc:0x4)
[0x10] (ID 0xf35) (kind 10) __s16 off (aligned at 0x2)
[0x20] (ID 0x1718) (kind 10) __s32 imm (aligned at 0x4)
This also fixes one place where a failure to format a type would be
erroneously considered an out-of-memory condition.
libctf/
* ctf-dump.c (ctf_is_slice): Delete, unnecessary.
(ctf_dump_format_type): improve slice formatting. Always print
the type size, even of slices.
(ctf_dump_member): Print slices (-> bitfields) differently from
non-slices. Failure to format a type is not an OOM.
If we get an error emitting a single type, variable, or label, right now
we emit the error into the ctf_dprintf stream and propagate the error
all the way up the stack, causing the entire output to be silently
truncated (unless libctf debugging is on).
Instead, emit an error and keep going. (This makes sense for this use
case: if you're dumping types and a type is corrupted, you want to
know!)
Not all instances of this are fixed in this commit, only ones associated
with type formatting: more fixes will come.
libctf/
* ctf-dump.c (ctf_dump_format_type): Emit a warning.
(ctf_dump_label): Swallow errors from ctf_dump_format_type.
(ctf_dump_objts): Likewise.
(ctf_dump_var): Likewise.
(ctf_dump_type): Do not emit a duplicate message. Move to
ctf_err_warning, and swallow all errors.
ctf_decl_sprintf builds up a formatted string in the ctf_decl_t's
cd_buf, but then on error this is hardly ever freed: we assume that
ctf_decl_fini frees it, but it leaks it instead.
Make it free it like any decent ADT should.
libctf/
* ctf-decl.c (ctf_decl_fini): Free the cd_buf.
(ctf_decl_buf): Once it escapes, don't try to free it later.
Somehow this never got implemented, which makes debugging any kind of
bug that has to do with argument types fantastically confusing, because
it *looks* like the func type takes no arguments though in fact it does.
This also lets us simplify the dumper slightly (and introduces our first
uses of ctf_assert and ctf_err_warn: there will be many more).
ctf_type_aname dumps function types without including the function
pointer name itself: ctf_dump search-and-replaces it in. This seems to
give the nicest-looking results for existing users of both, even if it
is a bit fiddly.
libctf/
* ctf-types.c (ctf_type_aname): Print arg types here...
* ctf-dump.c (ctf_dump_funcs): ... not here: but do substitute
in the type name here.
This commit adds a long-missing piece of infrastructure to libctf: the
ability to report errors and warnings using all the power of printf,
rather than being restricted to one errno value. Internally, libctf
calls ctf_err_warn() to add errors and warnings to a list: a new
iterator ctf_errwarning_next() then consumes this list one by one and
hands it to the caller, which can free it. New errors and warnings are
added until the list is consumed by the caller or the ctf_file_t is
closed, so you can dump them at intervals. The caller can of course
choose to print only those warnings it wants. (I am not sure whether we
want objdump, readelf or ld to print warnings or not: right now I'm
printing them, but maybe we only want to print errors? This entirely
depends on whether warnings are voluminous things describing e.g. the
inability to emit single types because of name clashes or something.
There are no users of this infrastructure yet, so it's hard to say.)
There is no internationalization here yet, but this at least adds a
place where internationalization can be added, to one of
ctf_errwarning_next or ctf_err_warn.
We also provide a new ctf_assert() function which uses this
infrastructure to provide non-fatal assertion failures while emitting an
assert-like string to the caller: to save space and avoid needlessly
duplicating unchanging strings, the assertion test is inlined but the
print-things-out failure case is not. All assertions in libctf will be
converted to use this machinery in future commits and propagate
assertion-failure errors up, so that the linker in particular cannot be
killed by libctf assertion failures when it could perfectly well just
print warnings and drop the CTF section.
include/
* ctf-api.h (ECTF_INTERNAL): Adjust error text.
(ctf_errwarning_next): New.
libctf/
* ctf-impl.h (ctf_assert): New.
(ctf_err_warning_t): Likewise.
(ctf_file_t) <ctf_errs_warnings>: Likewise.
(ctf_err_warn): New prototype.
(ctf_assert_fail_internal): Likewise.
* ctf-inlines.h (ctf_assert_internal): Likewise.
* ctf-open.c (ctf_file_close): Free ctf_errs_warnings.
* ctf-create.c (ctf_serialize): Copy it on serialization.
* ctf-subr.c (ctf_err_warn): New, add an error/warning.
(ctf_errwarning_next): New iterator, free and pass back
errors/warnings in succession.
* libctf.ver (ctf_errwarning_next): Add.
ld/
* ldlang.c (lang_ctf_errs_warnings): New, print CTF errors
and warnings. Assert when libctf asserts.
(lang_merge_ctf): Call it.
(land_write_ctf): Likewise.
binutils/
* objdump.c (ctf_archive_member): Print CTF errors and warnings.
* readelf.c (dump_ctf_archive_member): Likewise.
ctf_variable_iter was returning a (positive!) error code rather than
setting the error in the passed-in ctf_file_t.
Reviewed-by: Nick Alcock <nick.alcock@oracle.com>
libctf/
* ctf-types.c (ctf_variable_iter): Fix error return.
When wrapping qsort_r on a system like FreeBSD on which the compar
argument comes first, we wrap the passed arg in a thunk so we can pass
down both the caller-supplied comparator function and its argument. We
should pass the *argument* down to the comparator, not the thunk, which
is basically random nonsense on the stack from the point of view of the
caller of qsort_r.
libctf/
ctf-decls.h (ctf_qsort_compar_thunk): Fix arg passing.
This lets you iterate over dynhashes and dynsets using the _next API.
dynhashes can be iterated over in sorted order, which works by
populating an array of key/value pairs using ctf_dynhash_next itself,
then sorting it with qsort.
Convenience inline functions named ctf_dyn{hash,set}_cnext are also
provided that take (-> return) const keys and values.
libctf/
* ctf-impl.h (ctf_next_hkv_t): New, kv-pairs passed to
sorting functions.
(ctf_next_t) <u.ctn_sorted_hkv>: New, sorted kv-pairs for
ctf_dynhash_next_sorted.
<cu.ctn_h>: New, pointer to the dynhash under iteration.
<cu.ctn_s>: New, pointer to the dynset under iteration.
(ctf_hash_sort_f): Sorting function passed to...
(ctf_dynhash_next_sorted): ... this new function.
(ctf_dynhash_next): New.
(ctf_dynset_next): New.
* ctf-inlines.h (ctf_dynhash_cnext_sorted): New.
(ctf_dynhash_cnext): New.
(ctf_dynset_cnext): New.
* ctf-hash.c (ctf_dynhash_next_sorted): New.
(ctf_dynhash_next): New.
(ctf_dynset_next): New.
* ctf-util.c (ctf_next_destroy): Free the u.ctn_sorted_hkv if
needed.
(ctf_next_copy): Alloc-and-copy the u.ctn_sorted_hkv if needed.
The libctf machinery currently only provides one way to iterate over its
data structures: ctf_*_iter functions that take a callback and an arg
and repeatedly call it.
This *works*, but if you are doing a lot of iteration it is really quite
inconvenient: you have to package up your local variables into
structures over and over again and spawn lots of little functions even
if it would be clearer in a single run of code. Look at ctf-string.c
for an extreme example of how unreadable this can get, with
three-line-long functions proliferating wildly.
The deduplicator takes this to the Nth level. It iterates over a whole
bunch of things: if we'd had to use _iter-class iterators for all of
them there would be twenty additional functions in the deduplicator
alone, for no other reason than that the iterator API requires it.
Let's do something better. strtok_r gives us half the design: generators
in a number of other languages give us the other half.
The *_next API allows you to iterate over CTF-like entities in a single
function using a normal while loop. e.g. here we are iterating over all
the types in a dict:
ctf_next_t *i = NULL;
int *hidden;
ctf_id_t id;
while ((id = ctf_type_next (fp, &i, &hidden, 1)) != CTF_ERR)
{
/* do something with 'hidden' and 'id' */
}
if (ctf_errno (fp) != ECTF_NEXT_END)
/* iteration error */
Here we are walking through the members of a struct with CTF ID
'struct_type':
ctf_next_t *i = NULL;
ssize_t offset;
const char *name;
ctf_id_t membtype;
while ((offset = ctf_member_next (fp, struct_type, &i, &name,
&membtype)) >= 0
{
/* do something with offset, name, and membtype */
}
if (ctf_errno (fp) != ECTF_NEXT_END)
/* iteration error */
Like every other while loop, this means you have access to all the local
variables outside the loop while inside it, with no need to tiresomely
package things up in structures, move the body of the loop into a
separate function, etc, as you would with an iterator taking a callback.
ctf_*_next allocates 'i' for you on first entry (when it must be NULL),
and frees and NULLs it and returns a _next-dependent flag value when the
iteration is over: the fp errno is set to ECTF_NEXT_END when the
iteartion ends normally. If you want to exit early, call
ctf_next_destroy on the iterator. You can copy iterators using
ctf_next_copy, which copies their current iteration position so you can
remember loop positions and go back to them later (or ctf_next_destroy
them if you don't need them after all).
Each _next function returns an always-likely-to-be-useful property of
the thing being iterated over, and takes pointers to parameters for the
others: with very few exceptions all those parameters can be NULLs if
you're not interested in them, so e.g. you can iterate over only the
offsets of members of a structure this way:
while ((offset = ctf_member_next (fp, struct_id, &i, NULL, NULL)) >= 0)
If you pass an iterator in use by one iteration function to another one,
you get the new error ECTF_NEXT_WRONGFUN back; if you try to change
ctf_file_t in mid-iteration, you get ECTF_NEXT_WRONGFP back.
Internally the ctf_next_t remembers the iteration function in use,
various sizes and increments useful for almost all iterations, then
uses unions to overlap the actual entities being iterated over to keep
ctf_next_t size down.
Iterators available in the public API so far (all tested in actual use
in the deduplicator):
/* Iterate over the members of a STRUCT or UNION, returning each member's
offset and optionally name and member type in turn. On end-of-iteration,
returns -1. */
ssize_t
ctf_member_next (ctf_file_t *fp, ctf_id_t type, ctf_next_t **it,
const char **name, ctf_id_t *membtype);
/* Iterate over the members of an enum TYPE, returning each enumerand's
NAME or NULL at end of iteration or error, and optionally passing
back the enumerand's integer VALue. */
const char *
ctf_enum_next (ctf_file_t *fp, ctf_id_t type, ctf_next_t **it,
int *val);
/* Iterate over every type in the given CTF container (not including
parents), optionally including non-user-visible types, returning
each type ID and optionally the hidden flag in turn. Returns CTF_ERR
on end of iteration or error. */
ctf_id_t
ctf_type_next (ctf_file_t *fp, ctf_next_t **it, int *flag,
int want_hidden);
/* Iterate over every variable in the given CTF container, in arbitrary
order, returning the name and type of each variable in turn. The
NAME argument is not optional. Returns CTF_ERR on end of iteration
or error. */
ctf_id_t
ctf_variable_next (ctf_file_t *fp, ctf_next_t **it, const char **name);
/* Iterate over all CTF files in an archive, returning each dict in turn as a
ctf_file_t, and NULL on error or end of iteration. It is the caller's
responsibility to close it. Parent dicts may be skipped. Regardless of
whether they are skipped or not, the caller must ctf_import the parent if
need be. */
ctf_file_t *
ctf_archive_next (const ctf_archive_t *wrapper, ctf_next_t **it,
const char **name, int skip_parent, int *errp);
ctf_label_next is prototyped but not implemented yet.
include/
* ctf-api.h (ECTF_NEXT_END): New error.
(ECTF_NEXT_WRONGFUN): Likewise.
(ECTF_NEXT_WRONGFP): Likewise.
(ECTF_NERR): Adjust.
(ctf_next_t): New.
(ctf_next_create): New prototype.
(ctf_next_destroy): Likewise.
(ctf_next_copy): Likewise.
(ctf_member_next): Likewise.
(ctf_enum_next): Likewise.
(ctf_type_next): Likewise.
(ctf_label_next): Likewise.
(ctf_variable_next): Likewise.
libctf/
* ctf-impl.h (ctf_next): New.
(ctf_get_dict): New prototype.
* ctf-lookup.c (ctf_get_dict): New, split out of...
(ctf_lookup_by_id): ... here.
* ctf-util.c (ctf_next_create): New.
(ctf_next_destroy): New.
(ctf_next_copy): New.
* ctf-types.c (includes): Add <assert.h>.
(ctf_member_next): New.
(ctf_enum_next): New.
(ctf_type_iter): Document the lack of iteration over parent
types.
(ctf_type_next): New.
(ctf_variable_next): New.
* ctf-archive.c (ctf_archive_next): New.
* libctf.ver: Add new public functions.
This allows you to bump the refcount on a ctf_file_t, so that you can
smuggle it out of iterators which open and close the ctf_file_t for you
around the loop body (like ctf_archive_iter).
You still can't use this to preserve a ctf_file_t for longer than the
lifetime of its containing entity (e.g. ctf_archive).
include/
* ctf-api.h (ctf_ref): New.
libctf/
* libctf.ver (ctf_ref): New.
* ctf-open.c (ctf_ref): Implement it.
The internals of the deduplicator want to know if something is a type
that can have a forward to it fairly often, often enough that inlining
it brings a noticeable performance gain. Convert the one place in
libctf that can already benefit, even though it doesn't bring any sort
of performance gain there.
libctf/
* ctf-inlines.h (ctf_forwardable_kind): New.
* ctf-create.c (ctf_add_forward): Use it.
There are many places in the deduplicator which use hashtables as tiny
sets: keys with no value (and usually, but not always, no freeing
function) often with only one or a few members. For each of these, even
after the last change to not store the freeing functions, we are storing
a little malloced block for each item just to track the key/value pair,
and a little malloced block for the hash table itself just to track the
freeing function because we can't use libiberty hashtab's freeing
function because we are using that to free the little malloced per-item
block.
If we only have a key, we don't need any of that: we can ditch the
per-malloced block because we don't have a value, and we can ditch the
per-hashtab structure because we don't need to independently track the
freeing functions since libiberty hashtab is doing it for us. That
means we don't need an owner field in the (now nonexistent) item block
either.
Roughly speaking, this datatype saves about 25% in time and 20% in peak
memory usage for normal links, even fairly big ones. So this might seem
redundant, but it's really worth it.
Instead of a _lookup function, a dynset has two distinct functions:
ctf_dynset_exists, which returns true or false and an optional pointer
to the set member, and ctf_dynhash_lookup_any, which is used if all
members of the set are expected to be equivalent and we just want *any*
member and we don't care which one.
There is no iterator in this set of functions, not because we don't
iterate over dynset members -- we do, a lot -- but because the iterator
here is a member of an entirely new family of much more convenient
iteration functions, introduced in the next commit.
libctf/
* ctf-hash.c (ctf_dynset_eq_string): New.
(ctf_dynset_create): New.
(DYNSET_EMPTY_ENTRY_REPLACEMENT): New.
(DYNSET_DELETED_ENTRY_REPLACEMENT): New.
(key_to_internal): New.
(internal_to_key): New.
(ctf_dynset_insert): New.
(ctf_dynset_remove): New.
(ctf_dynset_destroy): New.
(ctf_dynset_lookup): New.
(ctf_dynset_exists): New.
(ctf_dynset_lookup_any): New.
(ctf_hash_insert_type): Coding style.
(ctf_hash_define_type): Likewise.
* ctf-impl.h (ctf_dynset_t): New.
(ctf_dynset_eq_string): New.
(ctf_dynset_create): New.
(ctf_dynset_insert): New.
(ctf_dynset_remove): New.
(ctf_dynset_destroy): New.
(ctf_dynset_lookup): New.
(ctf_dynset_exists): New.
(ctf_dynset_lookup_any): New.
* ctf-inlines.h (ctf_dynset_cinsert): New.
The libctf dynhash hashtab abstraction supports per-hashtab arbitrary
key/item freeing functions -- but it also has a constant slot type that
holds both key and value requested by the user, so it needs to use its
own freeing function to free that -- and it has nowhere to store the
freeing functions the caller requested.
So it copies them into every hash item, bloating every slot, even though
all items in a given hash table must have the same key and value freeing
functions.
So point back to the owner using a back-pointer, but don't even spend
space in the item or the hashtab allocating those freeing functions
unless necessary: if none are needed, we can simply arrange to not pass
in ctf_dynhash_item_free as a del_f to hashtab_create_alloc, and none of
those fields will ever be accessed.
The only downside is that this makes the code sensitive to the order of
fields in the ctf_helem_t and ctf_hashtab_t: but the deduplicator
allocates so many hash tables that doing this alone cuts memory usage
during deduplication by about 10%. (libiberty hashtab itself has a lot
of per-hashtab bloat: in the future we might trim that down, or make a
trimmer version.)
libctf/
* ctf-hash.c (ctf_helem_t) <key_free>: Remove.
<value_free>: Likewise.
<owner>: New.
(ctf_dynhash_item_free): Indirect through the owner.
(ctf_dynhash_create): Only pass in ctf_dynhash_item_free and
allocate space for the key_free and value_free fields fields
if necessary.
(ctf_hashtab_insert): Likewise. Fix OOM errno value.
(ctf_dynhash_insert): Only access ctf_hashtab's key_free and
value_free if they will exist. Set the slot's owner, but only
if it exists.
(ctf_dynhash_remove): Adjust.
Right now, if you insert a key/value pair into a dynhash, the old slot's
key is freed and the new one always assigned. This seemed sane to me
when I wrote it, but I got it wrong time and time again. It's much
less confusing to free the key passed in: if a key-freeing function
was passed, you are asserting that the dynhash owns the key in any
case, so if you pass in a key it is always buggy to assume it sticks
around. Freeing the old key means that you can't even safely look up a
key from out of a dynhash and hold on to it, because some other matching
key might force it to be freed at any time.
In the new model, you can always get a key out of a dynhash with
ctf_dynhash_lookup_kv and hang on to it until the kv-pair is actually
deleted from the dynhash. In the old model the pointer to the key might
be freed at any time if a matching key was inserted.
libctf/
* ctf-hash.c (ctf_hashtab_insert): Free the key passed in if
there is a key-freeing function and the key already exists.
Future commits will use these.
ctf_dynhash_elements: count elements in a dynhash
ctf_dynhash_lookup_kv: look up and return pointers to the original key
and value in a dynhash (the only way of getting
a reference to the original key)
ctf_dynhash_iter_find: iterate until an item is found, then return its
key
ctf_dynhash_cinsert: insert a const key / value into a dynhash (a thim
wrapper in a new header dedicated to inline
functions).
As with the rest of ctf_dynhash, this is not public API. No impact
on existing callers is expected.
libctf/
* ctf-inlines.h: New file.
* ctf-impl.h: Include it.
(ctf_hash_iter_find_f): New typedef.
(ctf_dynhash_elements): New.
(ctf_dynhash_lookup_kv): New.
(ctf_dynhash_iter_find): New.
* ctf-hash.c (ctf_dynhash_lookup_kv): New.
(ctf_traverse_find_cb_arg_t): New.
(ctf_hashtab_traverse_find): New.
(ctf_dynhash_iter_find): New.
(ctf_dynhash_elements): New.
Another count that was otherwise unavailable without doing expensive
operations.
include/
* ctf-api.h (ctf_archive_count): New.
libctf/
* ctf-archive.c (ctf_archive_count): New.
* libctf.ver: New public function.
This returns the number of members in a struct or union, or the number
of enumerations in an enum. (This was only available before now by
iterating across every member, but it can be returned much faster than
that.)
include/
* ctf-api.h (ctf_member_count): New.
libctf/
* ctf-types.c (ctf_member_count): New.
* libctf.ver: New public function.
This is just like ctf_type_kind, except that forwards get the
type of the thing being pointed to rather than CTF_K_FORWARD.
include/
* ctf-api.h (ctf_type_kind_forwarded): New.
libctf/
* ctf-types.c (ctf_type_kind_forwarded): New.
We already have a function ctf_type_aname_raw, which returns the raw
name of a type with no decoration for structures or arrays or anything
like that: just the underlying name of whatever it is that's being
ultimately pointed at.
But this can be inconvenient to use, becauswe it always allocates new
storage for the string and copies it in, so it can potentially fail.
Add ctf_type_name_raw, which just returns the string directly out of
libctf's guts: it will live until the ctf_file_t is closed (if we later
gain the ability to remove types from writable dicts, it will live as
long as the type lives).
Reimplement ctf_type_aname_raw in terms of it.
include/
* ctf-api.c (ctf_type_name_raw): New.
libctf/
* ctf-types.c (ctf_type_name_raw): New.
(ctf_type_aname_raw): Reimplement accordingly.
The deduplicator can emit enormous amounts of debugging output,
so much so that a later commit will introduce a new configure flag
that configures most of it out (and configures it out by default).
It became clear that when this configure flag is on, but debugging is
not enabled via the LIBCTF_DEBUG environment variable, up to 10% of
runtime can be spent on branch mispredictions checking the _libctf_debug
variable. Mark it unlikely to be set (when it is set, performance is
likely to be the least of your concerns).
libctf/
* ctf-subr.c (ctf_dprintf): _libctf_debug is unlikely to be set.
The archive machinery mmap()s its archives when possible: so it arranges
to do appropriately-sized unmaps by recording the unmap length in the
ctfa_magic value and unmapping that.
This brilliant (horrible) trick works less well when ctf_arc_bufopen is
called with an existing buffer (which might be a readonly mapping).
ctf_arc_bufopen always returns a ctf_archive_t wrapper, so record in
there the necessity to not unmap anything when a bufopen'ed archive is
closed again.
libctf/
* ctf-impl.h (struct ctf_archive_internal)
<ctfi_unmap_on_close>: New.
(ctf_new_archive_internal): Adjust.
* ctf-archive.c (ctf_new_archive_internal): Likewise.
Initialize ctfi_unmap_on_close. Adjust error path.
(ctf_arc_bufopen): Adjust ctf_new_archive_internal call
(unmap_on_close is 0).
(ctf_arc_close): Only unmap if ctfi_unmap_on_close.
* ctf-open-bfd.c (ctf_fdopen): Adjust.
Report them as such, rather than letting ctf_decl_sprintf wrongly
conclude that the printing of zero characters means we are out of
memory.
libctf/
* ctf-types.c (ctf_type_aname): Return ECTF_CORRUPT if
ints, floats or typedefs have no name. Fix comment typo.
It is perfectly valid C to say e.g.
typedef u64 int;
struct foo_t
{
const volatile u64 wibble:2;
};
i.e. bitfields have to be integral types, but they can be cv-qualified
integral types or typedefs of same, etc.
This is easy to fix: do a ctf_type_resolve_unsliced() at creation time
to ensure the ultimate type is integral, and ctf_type_resolve() at
lookup time so that if you somehow have e.g. a slice of a typedef of a
slice of a cv-qualified int, we pull the encoding that the topmost slice
is based on out of the subsidiary slice (and then modify it), not out of
the underlying int. (This last bit is rather academic right now, since
all slices override exactly the same properties of the underlying type,
but it's still the right thing to do.)
libctf/
* ctf-create.c (ctf_add_slice): Support slices of any kind that
resolves to an integral type.
* ctf-types.c (ctf_type_encoding): Resolve the type before
fishing its encoding out.
Without this, an empty dict that is written out immediately never gets
any content at all: even the header is left empty.
libctf/
* ctf-create.c (ctf_create): Mark dirty.
A Solaris-era bug causes us to check the offsets of types with no names
against the first such type when ctf_add_type()ing members to a struct
or union. Members with no names (i.e. anonymous struct/union members)
can appear as many times as you like in a struct/union, so this check
should be skipped in this case.
libctf/
* ctf-create.c (membcmp) Skip nameless members.
This matters for the case of unnamed bitfields, whose names are the null
string. These are special in that they are the only members whose
"names" are allowed to be duplicated in a single struct, but we were
only handling this for the case where name == NULL. Translate "" to
NULL to help callers.
libctf/
* ctf-create.c (ctf_add_member_offset): Support names of ""
as if they were the null pointer.
When opening, we consider a forward with a kind above the maximum
allowable set of kinds and a forward of kind CTF_K_UNKNOWN to be a
forward to a struct. Whatever CTF version it was that produced
forwards with no associated kind, it predates anything we can read:
remove this wart.
libctf/
* ctf-open.c (init_types): Remove typeless CTF_K_FORWARD
special-casing.
One spot was missed when we rejigged ctf_update into ctf_serialize and
allowed all operations on dynamic containers: ctf_type_reference of
slices. A dynamic slice's vlen state is stored in the dtu_slice member,
so fetch it from there.
libctf/
* ctf-types.c (ctf_type_reference): Add support for dynamic slices.
This is technically unnecessary -- the compiler is quite capable of
doing the range reduction for us -- but it does mean that all
assignments of a ctf_id_t to its final uint32_t representation now have
appropriate explicit casts.
libctf/
* ctf-create.c (ctf_serialize): Add cast.
(ctf_add_slice): Likewise.
ctf_add_function assumes that function types' arglists are of type
ctf_id_t. Since they are CTF IDs, they are 32 bits wide, a uint32_t:
unfortunately ctf_id_t is a forward-compatible user-facing 64 bits wide,
and should never ever reach the CTF storage level.
All the CTF code other than ctf_add_function correctly assumes that
function arglists outside dynamic containers are 32 bits wide, so the
serialization machinery ends up cutting off half the arglist, corrupting
all args but the first (a good sign is a bunch of args of ID 0, the
unimplemented type, popping up).
Fix this by copying the arglist into place item by item, casting it
properly, at the same time as we validate the arg types. Fix the type
of the dtu_argv in the dynamic container and drop the now-unnecessary
cast in the serializer.
libctf/
* ctf-impl.h (ctf_dtdef_t) <dtu_argv>: Fix type.
* ctf-create.c (ctf_add_function): Check for unimplemented type
and populate at the same time. Populate one-by-one, not via
memcpy.
(ctf_serialize): Remove unnecessary cast.
* ctf-types.c (ctf_func_type_info): Likewise.
(ctf_func_type_args): Likewise. Fix comment typo.
The deduplicating linker adds types from the linker inputs to the output
via the same API everyone else does, so it's important that we can emit
everything that the compiler wants us to. Unfortunately, the compiler
may represent the unimplemented type (used for compiler constructs that
CTF cannot currently encode) as type zero or as a type of kind
CTF_K_UNKNOWN, and we don't allow the addition of types that cite the
former.
Adding this support adds a tiny bit of extra complexity: additions of
structure members immediately following a member of the unimplemented
type must be via ctf_add_member_offset or ctf_add_member_encoded, since
we have no idea how big members of the unimplemented type are.
(Attempts to do otherwise return -ECTF_NONREPRESENTABLE, like other
attempts to do forbidden things with the unimplemented type.)
Even slices of the unimplemented type are permitted: this is the only
case in which you can slice a type that terminates in a non-integral
type, on the grounds that it was likely integral in the source code,
it's just that we can't represent that sort of integral type properly
yet.
libctf/
* ctf-create.c (ctf_add_reftype): Support refs to type zero.
(ctf_add_array): Support array contents of type zero.
(ctf_add_function): Support arguments and return types of
type zero.
(ctf_add_typedef): Support typedefs to type zero.
(ctf_add_member_offset): Support members of type zero,
unless added at unspecified (naturally-aligned) offset.
Jose Marchesi noted that the traditional-Unix error array in ctf-error.c
introduces one reloc per error to initialize the array: 58 so far. We
can reduce this to zero using an array of carefully-sized individual
members which is used to construct a string table, that is then
referenced by the lookup functions: but doing this automatically is a
pain.
Bruno Haible wrote suitable code years ago: I got permission to reuse it
(Bruno says "... which I hereby put in the public domain"); I modified
it a tiny bit (similarly to what Ulrich Drepper did in the dsohowto
text, but I redid it from scratch), commented it up a bit, and shifted
the error table into that form, migrating it into the new file
ctf-error.h.
This has the advantage that it spotted both typos in the text of the
errors in the comments in ctf-api.h and typos in the error defines in
the comments in ctf-error.c, and places where the two were simply not
in sync. All are now fixed.
One new constant exists in ctf-api.h: CTF_NERR, since the old method of
working out the number of errors in ctf-error.c was no longer usable,
and it seems that the number of CTF errors is something users might
reasonably want as well. It should be pretty easy to keep up to date as
new errors are introduced.
include/
* ctf-api.h (ECTF_*): Improve comments.
(ECTF_NERR): New.
libctf/
* ctf-error.c: Include <stddef.h>, for offsetof.
(_ctf_errlist): Migrate to...
(_ctf_errlist_t): ... this.
(_ctf_erridx): New, indexes into _ctf_errlist_t.
(_ctf_nerr): Remove.
(ctf_errmsg): Adjust accordingly.
* Makefile.am (BUILT_SOURCES): Note...
(ctf-error.h): ... this new rule.
* Makefile.in: Regenerate.
* mkerrors.sed: New, process ctf-api.h to generate ctf-error.h.
* .gitignore: New, ignore ctf-error.h.
readelf * readelf.c (parse_args): Silence potential warnings about a
memory resource leak when allocating space for ctf option values.
(dump_section_as_ctf): Fix typo checking dump_ctf_strtab_name
variable.
libctf * ctf-archive.c (ctf_arc_write): Avoid calling close twice on the
same file descriptor.
We were not using the right configure machinery to spot libintl on
platforms where it was required, leading to the spurious failure of
various configure tests (e.g. for things like ELF support in BFD).
libctf/
* aclocal.m4: Add config/gettext-sister.m4: Shuffle into
alphabetical order.
* configure.ac: Add ZW_GNU_GETTEXT_SISTER_DIR.
* config.h.in: Regenerated.
* Makefile.in: Likewise.
* configure: Likewise.
At least one C library (uclibc-ng) defines some of these only when
the compiler is GCC. We might as well test for all three cases and
handle any of them being missing.
Very similar code exists in libctf and split between elfcpp and gold:
fix both.
(Also sync up elfcpp with a change made to libctf swap.h a few months
ago: since there is no out-of-line definition of the bswap replacements,
they should be declared static inline, not just inline, to prevent the
linker generating out-of-line references to them.)
PR libctf/25120
libctf/
* configure.ac: Check for bswap_16, bswap_32, and bswap_64 decls.
* swap.h (bswap_16): Do not assume that presence of <byteswap.h>
means this is declared.
(bswap_32): Likewise.
(bswap_64): Likewise.
(bswap_identity_64): Remove, unused.
* configure: Regenerated.
* config.h.in: Likewise.
gold/
* configure.ac: Check for bswap_16, bswap_32, and bswap_64 decls.
* configure: Regenerated.
* config.h.in: Likewise.
elfcpp/
* elfcpp_swap.h (bswap_16): Do not assume that presence of
<byteswap.h> means this is declared. Make static inline, matching
recent change to libctf, since there is no non-inline definition
of these functions.
(bswap_32): Likewise.
(bswap_64): Likewise.
This keeps archive searching threadsafe using the new bsearch_r that was
just added to libiberty.
PR25120
libctf/
* ctf-archive.c (search_nametbl): No longer global: declare...
(ctf_arc_open_by_name_internal): ... here. Use bsearch_r.
(search_modent_by_name): Take and use ARG for the nametbl.
objdump and readelf have one major CTF-related behavioural difference:
objdump can read .ctf sections that contain CTF archives and extract and
dump their members, while readelf cannot. Since the linker often emits
CTF archives, this means that readelf intermittently and (from the
user's perspective) randomly fails to read CTF in files that ld emits,
with a confusing error message wrongly claiming that the CTF content is
corrupt. This is purely because the archive-opening code in libctf was
needlessly tangled up with the BFD code, so readelf couldn't use it.
Here, we disentangle it, moving ctf_new_archive_internal from
ctf-open-bfd.c into ctf-archive.c and merging it with the helper
function in ctf-archive.c it was already using. We add a new public API
function ctf_arc_bufopen, that looks very like ctf_bufopen but returns
an archive given suitable section data rather than a ctf_file_t: the
archive is a ctf_archive_t, so it can be called on raw CTF dictionaries
(with no archive present) and will return a single-member synthetic
"archive".
There is a tiny lifetime tweak here: before now, the archive code could
assume that the symbol section in the ctf_archive_internal wrapper
structure was always owned by BFD if it was present and should always be
freed: now, the caller can pass one in via ctf_arc_bufopen, wihch has
the usual lifetime rules for such sections (caller frees): so we add an
extra field to track whether this is an internal call from ctf-open-bfd,
in which case we still free the symbol section.
include/
* ctf-api.h (ctf_arc_bufopen): New.
libctf/
* ctf-impl.h (ctf_new_archive_internal): Declare.
(ctf_arc_bufopen): Remove.
(ctf_archive_internal) <ctfi_free_symsect>: New.
* ctf-archive.c (ctf_arc_close): Use it.
(ctf_arc_bufopen): Fuse into...
(ctf_new_archive_internal): ... this, moved across from...
* ctf-open-bfd.c: ... here.
(ctf_bfdopen_ctfsect): Use ctf_arc_bufopen.
* libctf.ver: Add it.
binutils/
* readelf.c (dump_section_as_ctf): Support .ctf archives using
ctf_arc_bufopen. Automatically load the .ctf member of such
archives as the parent of all other members, unless specifically
overridden via --ctf-parent. Split out dumping code into...
(dump_ctf_archive_member): ... here, as in objdump, and call
it once per archive member.
(dump_ctf_indent_lines): Code style fix.
The C namespace a forward is located in is always the same as the
namespace of the corresponding complete type: 'struct foo' is in the
struct namespace and does not collide with, say, 'union foo'.
libctf allowed for this in many places, but inconsistently: in
particular, forward *addition* never allowed for this, and was interning
forwards in the default namespace, which is always wrong, since you can
only forward structs, unions and enums, all of which are in their own
namespaces in C.
Forward removal needs corresponding adjustment to remove the names form
the right namespace, as does ctf_rollback.
libctf/
* ctf-create.c (ctf_add_forward): Intern in the right namespace.
(ctf_dtd_delete): Remove correspondingly.
(ctf_rollback): Likewise.
