binutils-gdb/include/ctf.h

622 lines
26 KiB
C
Raw Normal View History

/* CTF format description.
Copyright (C) 2019-2023 Free Software Foundation, Inc.
This file is part of libctf.
libctf is free software; you can redistribute it and/or modify it under
the terms of the GNU General Public License as published by the Free
Software Foundation; either version 3, or (at your option) any later
version.
This program is distributed in the hope that it will be useful, but
WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
See the GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; see the file COPYING. If not see
<http://www.gnu.org/licenses/>. */
#ifndef _CTF_H
#define _CTF_H
#include <sys/types.h>
#include <limits.h>
#include <stdint.h>
#ifdef __cplusplus
extern "C"
{
#endif
/* CTF - Compact ANSI-C Type Format
This file format can be used to compactly represent the information needed
by a debugger to interpret the ANSI-C types used by a given program.
Traditionally, this kind of information is generated by the compiler when
invoked with the -g flag and is stored in "stabs" strings or in the more
modern DWARF format. CTF provides a representation of only the information
that is relevant to debugging a complex, optimized C program such as the
operating system kernel in a form that is significantly more compact than
the equivalent stabs or DWARF representation. The format is data-model
independent, so consumers do not need different code depending on whether
they are 32-bit or 64-bit programs; libctf automatically compensates for
endianness variations. CTF assumes that a standard ELF symbol table is
available for use in the debugger, and uses the structure and data of the
symbol table to avoid storing redundant information. The CTF data may be
compressed on disk or in memory, indicated by a bit in the header. CTF may
be interpreted in a raw disk file, or it may be stored in an ELF section,
typically named .ctf. Data structures are aligned so that a raw CTF file or
CTF ELF section may be manipulated using mmap(2).
The CTF file or section itself has the following structure:
+--------+--------+---------+----------+--------+----------+...
| file | type | data | function | object | function |...
| header | labels | objects | info | index | index |...
+--------+--------+---------+----------+--------+----------+...
...+----------+-------+--------+
...| variable | data | string |
...| info | types | table |
+----------+-------+--------+
The file header stores a magic number and version information, encoding
flags, and the byte offset of each of the sections relative to the end of the
header itself. If the CTF data has been uniquified against another set of
CTF data, a reference to that data also appears in the the header. This
reference is the name of the label corresponding to the types uniquified
against.
Following the header is a list of labels, used to group the types included in
the data types section. Each label is accompanied by a type ID i. A given
label refers to the group of types whose IDs are in the range [0, i].
libctf: symbol type linking support This adds facilities to write out the function info and data object sections, which efficiently map from entries in the symbol table to types. The write-side code is entirely new: the read-side code was merely significantly changed and support for indexed tables added (pointed to by the no-longer-unused cth_objtidxoff and cth_funcidxoff header fields). With this in place, you can use ctf_lookup_by_symbol to look up the types of symbols of function and object type (and, as before, you can use ctf_lookup_variable to look up types of file-scope variables not present in the symbol table, as long as you know their name: but variables that are also data objects are now found in the data object section instead.) (Compatible) file format change: The CTF spec has always said that the function info section looks much like the CTF_K_FUNCTIONs in the type section: an info word (including an argument count) followed by a return type and N argument types. This format is suboptimal: it means function symbols cannot be deduplicated and it causes a lot of ugly code duplication in libctf. But conveniently the compiler has never emitted this! Because it has always emitted a rather different format that libctf has never accepted, we can be sure that there are no instances of this function info section in the wild, and can freely change its format without compatibility concerns or a file format version bump. (And since it has never been emitted in any code that generated any older file format version, either, we need keep no code to read the format as specified at all!) So the function info section is now specified as an array of uint32_t, exactly like the object data section: each entry is a type ID in the type section which must be of kind CTF_K_FUNCTION, the prototype of this function. This allows function types to be deduplicated and also correctly encodes the fact that all functions declared in C really are types available to the program: so they should be stored in the type section like all other types. (In format v4, we will be able to represent the types of static functions as well, but that really does require a file format change.) We introduce a new header flag, CTF_F_NEWFUNCINFO, which is set if the new function info format is in use. A sufficiently new compiler will always set this flag. New libctf will always set this flag: old libctf will refuse to open any CTF dicts that have this flag set. If the flag is not set on a dict being read in, new libctf will disregard the function info section. Format v4 will remove this flag (or, rather, the flag has no meaning there and the bit position may be recycled for some other purpose). New API: Symbol addition: ctf_add_func_sym: Add a symbol with a given name and type. The type must be of kind CTF_K_FUNCTION (a function pointer). Internally this adds a name -> type mapping to the ctf_funchash in the ctf_dict. ctf_add_objt_sym: Add a symbol with a given name and type. The type kind can be anything, including function pointers. This adds to ctf_objthash. These both treat symbols as name -> type mappings: the linker associates symbol names with symbol indexes via the ctf_link_shuffle_syms callback, which sets up the ctf_dynsyms/ctf_dynsymidx/ctf_dynsymmax fields in the ctf_dict. Repeated relinks can add more symbols. Variables that are also exposed as symbols are removed from the variable section at serialization time. CTF symbol type sections which have enough pads, defined by CTF_INDEX_PAD_THRESHOLD (whether because they are in dicts with symbols where most types are unknown, or in archive where most types are defined in some child or parent dict, not in this specific dict) are sorted by name rather than symidx and accompanied by an index which associates each symbol type entry with a name: the existing ctf_lookup_by_symbol will map symbol indexes to symbol names and look the names up in the index automatically. (This is currently ELF-symbol-table-dependent, but there is almost nothing specific to ELF in here and we can add support for other symbol table formats easily). The compiler also uses index sections to communicate the contents of object file symbol tables without relying on any specific ordering of symbols: it doesn't need to sort them, and libctf will detect an unsorted index section via the absence of the new CTF_F_IDXSORTED header flag, and sort it if needed. Iteration: ctf_symbol_next: Iterator which returns the types and names of symbols one by one, either for function or data symbols. This does not require any sorting: the ctf_link machinery uses it to pull in all the compiler-provided symbols cheaply, but it is not restricted to that use. (Compatible) changes in API: ctf_lookup_by_symbol: can now be called for object and function symbols: never returns ECTF_NOTDATA (which is now not thrown by anything, but is kept for compatibility and because it is a plausible error that we might start throwing again at some later date). Internally we also have changes to the ctf-string functionality so that "external" strings (those where we track a string -> offset mapping, but only write out an offset) can be consulted via the usual means (ctf_strptr) before the strtab is written out. This is important because ctf_link_add_linker_symbol can now be handed symbols named via strtab offsets, and ctf_link_shuffle_syms must figure out their actual names by looking in the external symtab we have just been fed by the ctf_link_add_strtab callback, long before that strtab is written out. include/ChangeLog 2020-11-20 Nick Alcock <nick.alcock@oracle.com> * ctf-api.h (ctf_symbol_next): New. (ctf_add_objt_sym): Likewise. (ctf_add_func_sym): Likewise. * ctf.h: Document new function info section format. (CTF_F_NEWFUNCINFO): New. (CTF_F_IDXSORTED): New. (CTF_F_MAX): Adjust accordingly. libctf/ChangeLog 2020-11-20 Nick Alcock <nick.alcock@oracle.com> * ctf-impl.h (CTF_INDEX_PAD_THRESHOLD): New. (_libctf_nonnull_): Likewise. (ctf_in_flight_dynsym_t): New. (ctf_dict_t) <ctf_funcidx_names>: Likewise. <ctf_objtidx_names>: Likewise. <ctf_nfuncidx>: Likewise. <ctf_nobjtidx>: Likewise. <ctf_funcidx_sxlate>: Likewise. <ctf_objtidx_sxlate>: Likewise. <ctf_objthash>: Likewise. <ctf_funchash>: Likewise. <ctf_dynsyms>: Likewise. <ctf_dynsymidx>: Likewise. <ctf_dynsymmax>: Likewise. <ctf_in_flight_dynsym>: Likewise. (struct ctf_next) <u.ctn_next>: Likewise. (ctf_symtab_skippable): New prototype. (ctf_add_funcobjt_sym): Likewise. (ctf_dynhash_sort_by_name): Likewise. (ctf_sym_to_elf64): Rename to... (ctf_elf32_to_link_sym): ... this, and... (ctf_elf64_to_link_sym): ... this. * ctf-open.c (init_symtab): Check for lack of CTF_F_NEWFUNCINFO flag, and presence of index sections. Refactor out ctf_symtab_skippable and ctf_elf*_to_link_sym, and use them. Use ctf_link_sym_t, not Elf64_Sym. Skip initializing objt or func sxlate sections if corresponding index section is present. Adjust for new func info section format. (ctf_bufopen_internal): Add ctf_err_warn to corrupt-file error handling. Report incorrect-length index sections. Always do an init_symtab, even if there is no symtab section (there may be index sections still). (flip_objts): Adjust comment: func and objt sections are actually identical in structure now, no need to caveat. (ctf_dict_close): Free newly-added data structures. * ctf-create.c (ctf_create): Initialize them. (ctf_symtab_skippable): New, refactored out of init_symtab, with st_nameidx_set check added. (ctf_add_funcobjt_sym): New, add a function or object symbol to the ctf_objthash or ctf_funchash, by name. (ctf_add_objt_sym): Call it. (ctf_add_func_sym): Likewise. (symtypetab_delete_nonstatic_vars): New, delete vars also present as data objects. (CTF_SYMTYPETAB_EMIT_FUNCTION): New flag to symtypetab emitters: this is a function emission, not a data object emission. (CTF_SYMTYPETAB_EMIT_PAD): New flag to symtypetab emitters: emit pads for symbols with no type (only set for unindexed sections). (CTF_SYMTYPETAB_FORCE_INDEXED): New flag to symtypetab emitters: always emit indexed. (symtypetab_density): New, figure out section sizes. (emit_symtypetab): New, emit a symtypetab. (emit_symtypetab_index): New, emit a symtypetab index. (ctf_serialize): Call them, emitting suitably sorted symtypetab sections and indexes. Set suitable header flags. Copy over new fields. * ctf-hash.c (ctf_dynhash_sort_by_name): New, used to impose an order on symtypetab index sections. * ctf-link.c (ctf_add_type_mapping): Delete erroneous comment relating to code that was never committed. (ctf_link_one_variable): Improve variable name. (check_sym): New, symtypetab analogue of check_variable. (ctf_link_deduplicating_one_symtypetab): New. (ctf_link_deduplicating_syms): Likewise. (ctf_link_deduplicating): Call them. (ctf_link_deduplicating_per_cu): Note that we don't call them in this case (yet). (ctf_link_add_strtab): Set the error on the fp correctly. (ctf_link_add_linker_symbol): New (no longer a do-nothing stub), add a linker symbol to the in-flight list. (ctf_link_shuffle_syms): New (no longer a do-nothing stub), turn the in-flight list into a mapping we can use, now its names are resolvable in the external strtab. * ctf-string.c (ctf_str_rollback_atom): Don't roll back atoms with external strtab offsets. (ctf_str_rollback): Adjust comment. (ctf_str_write_strtab): Migrate ctf_syn_ext_strtab population from writeout time... (ctf_str_add_external): ... to string addition time. * ctf-lookup.c (ctf_lookup_var_key_t): Rename to... (ctf_lookup_idx_key_t): ... this, now we use it for syms too. <clik_names>: New member, a name table. (ctf_lookup_var): Adjust accordingly. (ctf_lookup_variable): Likewise. (ctf_lookup_by_id): Shuffle further up in the file. (ctf_symidx_sort_arg_cb): New, callback for... (sort_symidx_by_name): ... this new function to sort a symidx found to be unsorted (likely originating from the compiler). (ctf_symidx_sort): New, sort a symidx. (ctf_lookup_symbol_name): Support dynamic symbols with indexes provided by the linker. Use ctf_link_sym_t, not Elf64_Sym. Check the parent if a child lookup fails. (ctf_lookup_by_symbol): Likewise. Work for function symbols too. (ctf_symbol_next): New, iterate over symbols with types (without sorting). (ctf_lookup_idx_name): New, bsearch for symbol names in indexes. (ctf_try_lookup_indexed): New, attempt an indexed lookup. (ctf_func_info): Reimplement in terms of ctf_lookup_by_symbol. (ctf_func_args): Likewise. (ctf_get_dict): Move... * ctf-types.c (ctf_get_dict): ... here. * ctf-util.c (ctf_sym_to_elf64): Re-express as... (ctf_elf64_to_link_sym): ... this. Add new st_symidx field, and st_nameidx_set (always 0, so st_nameidx can be ignored). Look in the ELF strtab for names. (ctf_elf32_to_link_sym): Likewise, for Elf32_Sym. (ctf_next_destroy): Destroy ctf_next_t.u.ctn_next if need be. * libctf.ver: Add ctf_symbol_next, ctf_add_objt_sym and ctf_add_func_sym.
2020-11-20 21:34:04 +08:00
Data object and function records (collectively, "symtypetabs") are stored in
the same order as they appear in the corresponding symbol table, except that
symbols marked SHN_UNDEF are not stored and symbols that have no type data
are padded out with zeroes. For each entry in these tables, the type ID (a
small integer) is recorded. (Functions get CTF_K_FUNCTION types, just like
data objects that are function pointers.)
For situations in which the order of the symbols in the symtab is not known,
libctf: symbol type linking support This adds facilities to write out the function info and data object sections, which efficiently map from entries in the symbol table to types. The write-side code is entirely new: the read-side code was merely significantly changed and support for indexed tables added (pointed to by the no-longer-unused cth_objtidxoff and cth_funcidxoff header fields). With this in place, you can use ctf_lookup_by_symbol to look up the types of symbols of function and object type (and, as before, you can use ctf_lookup_variable to look up types of file-scope variables not present in the symbol table, as long as you know their name: but variables that are also data objects are now found in the data object section instead.) (Compatible) file format change: The CTF spec has always said that the function info section looks much like the CTF_K_FUNCTIONs in the type section: an info word (including an argument count) followed by a return type and N argument types. This format is suboptimal: it means function symbols cannot be deduplicated and it causes a lot of ugly code duplication in libctf. But conveniently the compiler has never emitted this! Because it has always emitted a rather different format that libctf has never accepted, we can be sure that there are no instances of this function info section in the wild, and can freely change its format without compatibility concerns or a file format version bump. (And since it has never been emitted in any code that generated any older file format version, either, we need keep no code to read the format as specified at all!) So the function info section is now specified as an array of uint32_t, exactly like the object data section: each entry is a type ID in the type section which must be of kind CTF_K_FUNCTION, the prototype of this function. This allows function types to be deduplicated and also correctly encodes the fact that all functions declared in C really are types available to the program: so they should be stored in the type section like all other types. (In format v4, we will be able to represent the types of static functions as well, but that really does require a file format change.) We introduce a new header flag, CTF_F_NEWFUNCINFO, which is set if the new function info format is in use. A sufficiently new compiler will always set this flag. New libctf will always set this flag: old libctf will refuse to open any CTF dicts that have this flag set. If the flag is not set on a dict being read in, new libctf will disregard the function info section. Format v4 will remove this flag (or, rather, the flag has no meaning there and the bit position may be recycled for some other purpose). New API: Symbol addition: ctf_add_func_sym: Add a symbol with a given name and type. The type must be of kind CTF_K_FUNCTION (a function pointer). Internally this adds a name -> type mapping to the ctf_funchash in the ctf_dict. ctf_add_objt_sym: Add a symbol with a given name and type. The type kind can be anything, including function pointers. This adds to ctf_objthash. These both treat symbols as name -> type mappings: the linker associates symbol names with symbol indexes via the ctf_link_shuffle_syms callback, which sets up the ctf_dynsyms/ctf_dynsymidx/ctf_dynsymmax fields in the ctf_dict. Repeated relinks can add more symbols. Variables that are also exposed as symbols are removed from the variable section at serialization time. CTF symbol type sections which have enough pads, defined by CTF_INDEX_PAD_THRESHOLD (whether because they are in dicts with symbols where most types are unknown, or in archive where most types are defined in some child or parent dict, not in this specific dict) are sorted by name rather than symidx and accompanied by an index which associates each symbol type entry with a name: the existing ctf_lookup_by_symbol will map symbol indexes to symbol names and look the names up in the index automatically. (This is currently ELF-symbol-table-dependent, but there is almost nothing specific to ELF in here and we can add support for other symbol table formats easily). The compiler also uses index sections to communicate the contents of object file symbol tables without relying on any specific ordering of symbols: it doesn't need to sort them, and libctf will detect an unsorted index section via the absence of the new CTF_F_IDXSORTED header flag, and sort it if needed. Iteration: ctf_symbol_next: Iterator which returns the types and names of symbols one by one, either for function or data symbols. This does not require any sorting: the ctf_link machinery uses it to pull in all the compiler-provided symbols cheaply, but it is not restricted to that use. (Compatible) changes in API: ctf_lookup_by_symbol: can now be called for object and function symbols: never returns ECTF_NOTDATA (which is now not thrown by anything, but is kept for compatibility and because it is a plausible error that we might start throwing again at some later date). Internally we also have changes to the ctf-string functionality so that "external" strings (those where we track a string -> offset mapping, but only write out an offset) can be consulted via the usual means (ctf_strptr) before the strtab is written out. This is important because ctf_link_add_linker_symbol can now be handed symbols named via strtab offsets, and ctf_link_shuffle_syms must figure out their actual names by looking in the external symtab we have just been fed by the ctf_link_add_strtab callback, long before that strtab is written out. include/ChangeLog 2020-11-20 Nick Alcock <nick.alcock@oracle.com> * ctf-api.h (ctf_symbol_next): New. (ctf_add_objt_sym): Likewise. (ctf_add_func_sym): Likewise. * ctf.h: Document new function info section format. (CTF_F_NEWFUNCINFO): New. (CTF_F_IDXSORTED): New. (CTF_F_MAX): Adjust accordingly. libctf/ChangeLog 2020-11-20 Nick Alcock <nick.alcock@oracle.com> * ctf-impl.h (CTF_INDEX_PAD_THRESHOLD): New. (_libctf_nonnull_): Likewise. (ctf_in_flight_dynsym_t): New. (ctf_dict_t) <ctf_funcidx_names>: Likewise. <ctf_objtidx_names>: Likewise. <ctf_nfuncidx>: Likewise. <ctf_nobjtidx>: Likewise. <ctf_funcidx_sxlate>: Likewise. <ctf_objtidx_sxlate>: Likewise. <ctf_objthash>: Likewise. <ctf_funchash>: Likewise. <ctf_dynsyms>: Likewise. <ctf_dynsymidx>: Likewise. <ctf_dynsymmax>: Likewise. <ctf_in_flight_dynsym>: Likewise. (struct ctf_next) <u.ctn_next>: Likewise. (ctf_symtab_skippable): New prototype. (ctf_add_funcobjt_sym): Likewise. (ctf_dynhash_sort_by_name): Likewise. (ctf_sym_to_elf64): Rename to... (ctf_elf32_to_link_sym): ... this, and... (ctf_elf64_to_link_sym): ... this. * ctf-open.c (init_symtab): Check for lack of CTF_F_NEWFUNCINFO flag, and presence of index sections. Refactor out ctf_symtab_skippable and ctf_elf*_to_link_sym, and use them. Use ctf_link_sym_t, not Elf64_Sym. Skip initializing objt or func sxlate sections if corresponding index section is present. Adjust for new func info section format. (ctf_bufopen_internal): Add ctf_err_warn to corrupt-file error handling. Report incorrect-length index sections. Always do an init_symtab, even if there is no symtab section (there may be index sections still). (flip_objts): Adjust comment: func and objt sections are actually identical in structure now, no need to caveat. (ctf_dict_close): Free newly-added data structures. * ctf-create.c (ctf_create): Initialize them. (ctf_symtab_skippable): New, refactored out of init_symtab, with st_nameidx_set check added. (ctf_add_funcobjt_sym): New, add a function or object symbol to the ctf_objthash or ctf_funchash, by name. (ctf_add_objt_sym): Call it. (ctf_add_func_sym): Likewise. (symtypetab_delete_nonstatic_vars): New, delete vars also present as data objects. (CTF_SYMTYPETAB_EMIT_FUNCTION): New flag to symtypetab emitters: this is a function emission, not a data object emission. (CTF_SYMTYPETAB_EMIT_PAD): New flag to symtypetab emitters: emit pads for symbols with no type (only set for unindexed sections). (CTF_SYMTYPETAB_FORCE_INDEXED): New flag to symtypetab emitters: always emit indexed. (symtypetab_density): New, figure out section sizes. (emit_symtypetab): New, emit a symtypetab. (emit_symtypetab_index): New, emit a symtypetab index. (ctf_serialize): Call them, emitting suitably sorted symtypetab sections and indexes. Set suitable header flags. Copy over new fields. * ctf-hash.c (ctf_dynhash_sort_by_name): New, used to impose an order on symtypetab index sections. * ctf-link.c (ctf_add_type_mapping): Delete erroneous comment relating to code that was never committed. (ctf_link_one_variable): Improve variable name. (check_sym): New, symtypetab analogue of check_variable. (ctf_link_deduplicating_one_symtypetab): New. (ctf_link_deduplicating_syms): Likewise. (ctf_link_deduplicating): Call them. (ctf_link_deduplicating_per_cu): Note that we don't call them in this case (yet). (ctf_link_add_strtab): Set the error on the fp correctly. (ctf_link_add_linker_symbol): New (no longer a do-nothing stub), add a linker symbol to the in-flight list. (ctf_link_shuffle_syms): New (no longer a do-nothing stub), turn the in-flight list into a mapping we can use, now its names are resolvable in the external strtab. * ctf-string.c (ctf_str_rollback_atom): Don't roll back atoms with external strtab offsets. (ctf_str_rollback): Adjust comment. (ctf_str_write_strtab): Migrate ctf_syn_ext_strtab population from writeout time... (ctf_str_add_external): ... to string addition time. * ctf-lookup.c (ctf_lookup_var_key_t): Rename to... (ctf_lookup_idx_key_t): ... this, now we use it for syms too. <clik_names>: New member, a name table. (ctf_lookup_var): Adjust accordingly. (ctf_lookup_variable): Likewise. (ctf_lookup_by_id): Shuffle further up in the file. (ctf_symidx_sort_arg_cb): New, callback for... (sort_symidx_by_name): ... this new function to sort a symidx found to be unsorted (likely originating from the compiler). (ctf_symidx_sort): New, sort a symidx. (ctf_lookup_symbol_name): Support dynamic symbols with indexes provided by the linker. Use ctf_link_sym_t, not Elf64_Sym. Check the parent if a child lookup fails. (ctf_lookup_by_symbol): Likewise. Work for function symbols too. (ctf_symbol_next): New, iterate over symbols with types (without sorting). (ctf_lookup_idx_name): New, bsearch for symbol names in indexes. (ctf_try_lookup_indexed): New, attempt an indexed lookup. (ctf_func_info): Reimplement in terms of ctf_lookup_by_symbol. (ctf_func_args): Likewise. (ctf_get_dict): Move... * ctf-types.c (ctf_get_dict): ... here. * ctf-util.c (ctf_sym_to_elf64): Re-express as... (ctf_elf64_to_link_sym): ... this. Add new st_symidx field, and st_nameidx_set (always 0, so st_nameidx can be ignored). Look in the ELF strtab for names. (ctf_elf32_to_link_sym): Likewise, for Elf32_Sym. (ctf_next_destroy): Destroy ctf_next_t.u.ctn_next if need be. * libctf.ver: Add ctf_symbol_next, ctf_add_objt_sym and ctf_add_func_sym.
2020-11-20 21:34:04 +08:00
or most symbols have no type in this dict and most entries would be
zero-pads, a pair of optional indexes follow the data object and function
info sections: each of these is an array of strtab indexes, mapped 1:1 to the
corresponding data object / function info section, giving each entry in those
sections a name so that the linker can correlate them with final symtab
entries and reorder them accordingly (dropping the indexes in the process).
Variable records (as distinct from data objects) provide a modicum of support
include, libctf, ld: extend variable section to contain functions too The CTF variable section is an optional (usually-not-present) section in the CTF dict which contains name -> type mappings corresponding to data symbols that are present in the linker input but not in the output symbol table: the idea is that programs that use their own symbol- resolution mechanisms can use this section to look up the types of symbols they have found using their own mechanism. Because these removed symbols (mostly static variables, functions, etc) all have names that are unlikely to appear in the ELF symtab and because very few programs have their own symbol-resolution mechanisms, a special linker flag (--ctf-variables) is needed to emit this section. Historically, we emitted only removed data symbols into the variable section. This seemed to make sense at the time, but in hindsight it really doesn't: functions are symbols too, and a C program can look them up just like any other type. So extend the variable section so that it contains all static function symbols too (if it is emitted at all), with types of kind CTF_K_FUNCTION. This is a little fiddly. We relied on compiler assistance for data symbols: the compiler simply emits all data symbols twice, once into the symtypetab as an indexed symbol and once into the variable section. Rather than wait for a suitably adjusted compiler that does the same for function symbols, we can pluck unreported function symbols out of the symtab and add them to the variable section ourselves. While we're at it, we do the same with data symbols: this is redundant right now because the compiler does it, but it costs very little time and lets the compiler drop this kludge and save a little space in .o files. include/ * ctf.h: Mention the new things we can see in the variable section. ld/ * testsuite/ld-ctf/data-func-conflicted-vars.d: New test. libctf/ * ctf-link.c (ctf_link_deduplicating_variables): Duplicate symbols into the variable section too. * ctf-serialize.c (symtypetab_delete_nonstatic_vars): Rename to... (symtypetab_delete_nonstatics): ... this. Check the funchash when pruning redundant variables. (ctf_symtypetab_sect_sizes): Adjust accordingly. * NEWS: Describe this change.
2022-03-16 23:29:25 +08:00
for non-ELF systems, mapping a variable or function name to a CTF type ID.
The names are sorted into ASCIIbetical order, permitting binary searching.
We do not define how the consumer maps these variable names to addresses or
anything else, or indeed what these names represent: they might be names
looked up at runtime via dlsym() or names extracted at runtime by a debugger
libctf: symbol type linking support This adds facilities to write out the function info and data object sections, which efficiently map from entries in the symbol table to types. The write-side code is entirely new: the read-side code was merely significantly changed and support for indexed tables added (pointed to by the no-longer-unused cth_objtidxoff and cth_funcidxoff header fields). With this in place, you can use ctf_lookup_by_symbol to look up the types of symbols of function and object type (and, as before, you can use ctf_lookup_variable to look up types of file-scope variables not present in the symbol table, as long as you know their name: but variables that are also data objects are now found in the data object section instead.) (Compatible) file format change: The CTF spec has always said that the function info section looks much like the CTF_K_FUNCTIONs in the type section: an info word (including an argument count) followed by a return type and N argument types. This format is suboptimal: it means function symbols cannot be deduplicated and it causes a lot of ugly code duplication in libctf. But conveniently the compiler has never emitted this! Because it has always emitted a rather different format that libctf has never accepted, we can be sure that there are no instances of this function info section in the wild, and can freely change its format without compatibility concerns or a file format version bump. (And since it has never been emitted in any code that generated any older file format version, either, we need keep no code to read the format as specified at all!) So the function info section is now specified as an array of uint32_t, exactly like the object data section: each entry is a type ID in the type section which must be of kind CTF_K_FUNCTION, the prototype of this function. This allows function types to be deduplicated and also correctly encodes the fact that all functions declared in C really are types available to the program: so they should be stored in the type section like all other types. (In format v4, we will be able to represent the types of static functions as well, but that really does require a file format change.) We introduce a new header flag, CTF_F_NEWFUNCINFO, which is set if the new function info format is in use. A sufficiently new compiler will always set this flag. New libctf will always set this flag: old libctf will refuse to open any CTF dicts that have this flag set. If the flag is not set on a dict being read in, new libctf will disregard the function info section. Format v4 will remove this flag (or, rather, the flag has no meaning there and the bit position may be recycled for some other purpose). New API: Symbol addition: ctf_add_func_sym: Add a symbol with a given name and type. The type must be of kind CTF_K_FUNCTION (a function pointer). Internally this adds a name -> type mapping to the ctf_funchash in the ctf_dict. ctf_add_objt_sym: Add a symbol with a given name and type. The type kind can be anything, including function pointers. This adds to ctf_objthash. These both treat symbols as name -> type mappings: the linker associates symbol names with symbol indexes via the ctf_link_shuffle_syms callback, which sets up the ctf_dynsyms/ctf_dynsymidx/ctf_dynsymmax fields in the ctf_dict. Repeated relinks can add more symbols. Variables that are also exposed as symbols are removed from the variable section at serialization time. CTF symbol type sections which have enough pads, defined by CTF_INDEX_PAD_THRESHOLD (whether because they are in dicts with symbols where most types are unknown, or in archive where most types are defined in some child or parent dict, not in this specific dict) are sorted by name rather than symidx and accompanied by an index which associates each symbol type entry with a name: the existing ctf_lookup_by_symbol will map symbol indexes to symbol names and look the names up in the index automatically. (This is currently ELF-symbol-table-dependent, but there is almost nothing specific to ELF in here and we can add support for other symbol table formats easily). The compiler also uses index sections to communicate the contents of object file symbol tables without relying on any specific ordering of symbols: it doesn't need to sort them, and libctf will detect an unsorted index section via the absence of the new CTF_F_IDXSORTED header flag, and sort it if needed. Iteration: ctf_symbol_next: Iterator which returns the types and names of symbols one by one, either for function or data symbols. This does not require any sorting: the ctf_link machinery uses it to pull in all the compiler-provided symbols cheaply, but it is not restricted to that use. (Compatible) changes in API: ctf_lookup_by_symbol: can now be called for object and function symbols: never returns ECTF_NOTDATA (which is now not thrown by anything, but is kept for compatibility and because it is a plausible error that we might start throwing again at some later date). Internally we also have changes to the ctf-string functionality so that "external" strings (those where we track a string -> offset mapping, but only write out an offset) can be consulted via the usual means (ctf_strptr) before the strtab is written out. This is important because ctf_link_add_linker_symbol can now be handed symbols named via strtab offsets, and ctf_link_shuffle_syms must figure out their actual names by looking in the external symtab we have just been fed by the ctf_link_add_strtab callback, long before that strtab is written out. include/ChangeLog 2020-11-20 Nick Alcock <nick.alcock@oracle.com> * ctf-api.h (ctf_symbol_next): New. (ctf_add_objt_sym): Likewise. (ctf_add_func_sym): Likewise. * ctf.h: Document new function info section format. (CTF_F_NEWFUNCINFO): New. (CTF_F_IDXSORTED): New. (CTF_F_MAX): Adjust accordingly. libctf/ChangeLog 2020-11-20 Nick Alcock <nick.alcock@oracle.com> * ctf-impl.h (CTF_INDEX_PAD_THRESHOLD): New. (_libctf_nonnull_): Likewise. (ctf_in_flight_dynsym_t): New. (ctf_dict_t) <ctf_funcidx_names>: Likewise. <ctf_objtidx_names>: Likewise. <ctf_nfuncidx>: Likewise. <ctf_nobjtidx>: Likewise. <ctf_funcidx_sxlate>: Likewise. <ctf_objtidx_sxlate>: Likewise. <ctf_objthash>: Likewise. <ctf_funchash>: Likewise. <ctf_dynsyms>: Likewise. <ctf_dynsymidx>: Likewise. <ctf_dynsymmax>: Likewise. <ctf_in_flight_dynsym>: Likewise. (struct ctf_next) <u.ctn_next>: Likewise. (ctf_symtab_skippable): New prototype. (ctf_add_funcobjt_sym): Likewise. (ctf_dynhash_sort_by_name): Likewise. (ctf_sym_to_elf64): Rename to... (ctf_elf32_to_link_sym): ... this, and... (ctf_elf64_to_link_sym): ... this. * ctf-open.c (init_symtab): Check for lack of CTF_F_NEWFUNCINFO flag, and presence of index sections. Refactor out ctf_symtab_skippable and ctf_elf*_to_link_sym, and use them. Use ctf_link_sym_t, not Elf64_Sym. Skip initializing objt or func sxlate sections if corresponding index section is present. Adjust for new func info section format. (ctf_bufopen_internal): Add ctf_err_warn to corrupt-file error handling. Report incorrect-length index sections. Always do an init_symtab, even if there is no symtab section (there may be index sections still). (flip_objts): Adjust comment: func and objt sections are actually identical in structure now, no need to caveat. (ctf_dict_close): Free newly-added data structures. * ctf-create.c (ctf_create): Initialize them. (ctf_symtab_skippable): New, refactored out of init_symtab, with st_nameidx_set check added. (ctf_add_funcobjt_sym): New, add a function or object symbol to the ctf_objthash or ctf_funchash, by name. (ctf_add_objt_sym): Call it. (ctf_add_func_sym): Likewise. (symtypetab_delete_nonstatic_vars): New, delete vars also present as data objects. (CTF_SYMTYPETAB_EMIT_FUNCTION): New flag to symtypetab emitters: this is a function emission, not a data object emission. (CTF_SYMTYPETAB_EMIT_PAD): New flag to symtypetab emitters: emit pads for symbols with no type (only set for unindexed sections). (CTF_SYMTYPETAB_FORCE_INDEXED): New flag to symtypetab emitters: always emit indexed. (symtypetab_density): New, figure out section sizes. (emit_symtypetab): New, emit a symtypetab. (emit_symtypetab_index): New, emit a symtypetab index. (ctf_serialize): Call them, emitting suitably sorted symtypetab sections and indexes. Set suitable header flags. Copy over new fields. * ctf-hash.c (ctf_dynhash_sort_by_name): New, used to impose an order on symtypetab index sections. * ctf-link.c (ctf_add_type_mapping): Delete erroneous comment relating to code that was never committed. (ctf_link_one_variable): Improve variable name. (check_sym): New, symtypetab analogue of check_variable. (ctf_link_deduplicating_one_symtypetab): New. (ctf_link_deduplicating_syms): Likewise. (ctf_link_deduplicating): Call them. (ctf_link_deduplicating_per_cu): Note that we don't call them in this case (yet). (ctf_link_add_strtab): Set the error on the fp correctly. (ctf_link_add_linker_symbol): New (no longer a do-nothing stub), add a linker symbol to the in-flight list. (ctf_link_shuffle_syms): New (no longer a do-nothing stub), turn the in-flight list into a mapping we can use, now its names are resolvable in the external strtab. * ctf-string.c (ctf_str_rollback_atom): Don't roll back atoms with external strtab offsets. (ctf_str_rollback): Adjust comment. (ctf_str_write_strtab): Migrate ctf_syn_ext_strtab population from writeout time... (ctf_str_add_external): ... to string addition time. * ctf-lookup.c (ctf_lookup_var_key_t): Rename to... (ctf_lookup_idx_key_t): ... this, now we use it for syms too. <clik_names>: New member, a name table. (ctf_lookup_var): Adjust accordingly. (ctf_lookup_variable): Likewise. (ctf_lookup_by_id): Shuffle further up in the file. (ctf_symidx_sort_arg_cb): New, callback for... (sort_symidx_by_name): ... this new function to sort a symidx found to be unsorted (likely originating from the compiler). (ctf_symidx_sort): New, sort a symidx. (ctf_lookup_symbol_name): Support dynamic symbols with indexes provided by the linker. Use ctf_link_sym_t, not Elf64_Sym. Check the parent if a child lookup fails. (ctf_lookup_by_symbol): Likewise. Work for function symbols too. (ctf_symbol_next): New, iterate over symbols with types (without sorting). (ctf_lookup_idx_name): New, bsearch for symbol names in indexes. (ctf_try_lookup_indexed): New, attempt an indexed lookup. (ctf_func_info): Reimplement in terms of ctf_lookup_by_symbol. (ctf_func_args): Likewise. (ctf_get_dict): Move... * ctf-types.c (ctf_get_dict): ... here. * ctf-util.c (ctf_sym_to_elf64): Re-express as... (ctf_elf64_to_link_sym): ... this. Add new st_symidx field, and st_nameidx_set (always 0, so st_nameidx can be ignored). Look in the ELF strtab for names. (ctf_elf32_to_link_sym): Likewise, for Elf32_Sym. (ctf_next_destroy): Destroy ctf_next_t.u.ctn_next if need be. * libctf.ver: Add ctf_symbol_next, ctf_add_objt_sym and ctf_add_func_sym.
2020-11-20 21:34:04 +08:00
or anything else the consumer likes. Variable records with identically-
include, libctf, ld: extend variable section to contain functions too The CTF variable section is an optional (usually-not-present) section in the CTF dict which contains name -> type mappings corresponding to data symbols that are present in the linker input but not in the output symbol table: the idea is that programs that use their own symbol- resolution mechanisms can use this section to look up the types of symbols they have found using their own mechanism. Because these removed symbols (mostly static variables, functions, etc) all have names that are unlikely to appear in the ELF symtab and because very few programs have their own symbol-resolution mechanisms, a special linker flag (--ctf-variables) is needed to emit this section. Historically, we emitted only removed data symbols into the variable section. This seemed to make sense at the time, but in hindsight it really doesn't: functions are symbols too, and a C program can look them up just like any other type. So extend the variable section so that it contains all static function symbols too (if it is emitted at all), with types of kind CTF_K_FUNCTION. This is a little fiddly. We relied on compiler assistance for data symbols: the compiler simply emits all data symbols twice, once into the symtypetab as an indexed symbol and once into the variable section. Rather than wait for a suitably adjusted compiler that does the same for function symbols, we can pluck unreported function symbols out of the symtab and add them to the variable section ourselves. While we're at it, we do the same with data symbols: this is redundant right now because the compiler does it, but it costs very little time and lets the compiler drop this kludge and save a little space in .o files. include/ * ctf.h: Mention the new things we can see in the variable section. ld/ * testsuite/ld-ctf/data-func-conflicted-vars.d: New test. libctf/ * ctf-link.c (ctf_link_deduplicating_variables): Duplicate symbols into the variable section too. * ctf-serialize.c (symtypetab_delete_nonstatic_vars): Rename to... (symtypetab_delete_nonstatics): ... this. Check the funchash when pruning redundant variables. (ctf_symtypetab_sect_sizes): Adjust accordingly. * NEWS: Describe this change.
2022-03-16 23:29:25 +08:00
named entries in the data object or function index section are removed.
The data types section is a list of variable size records that represent each
type, in order by their ID. The types themselves form a directed graph,
where each node may contain one or more outgoing edges to other type nodes,
denoted by their ID. Most type nodes are standalone or point backwards to
earlier nodes, but this is not required: nodes can point to later nodes,
particularly structure and union members.
Strings are recorded as a string table ID (0 or 1) and a byte offset into the
string table. String table 0 is the internal CTF string table. String table
1 is the external string table, which is the string table associated with the
libctf: symbol type linking support This adds facilities to write out the function info and data object sections, which efficiently map from entries in the symbol table to types. The write-side code is entirely new: the read-side code was merely significantly changed and support for indexed tables added (pointed to by the no-longer-unused cth_objtidxoff and cth_funcidxoff header fields). With this in place, you can use ctf_lookup_by_symbol to look up the types of symbols of function and object type (and, as before, you can use ctf_lookup_variable to look up types of file-scope variables not present in the symbol table, as long as you know their name: but variables that are also data objects are now found in the data object section instead.) (Compatible) file format change: The CTF spec has always said that the function info section looks much like the CTF_K_FUNCTIONs in the type section: an info word (including an argument count) followed by a return type and N argument types. This format is suboptimal: it means function symbols cannot be deduplicated and it causes a lot of ugly code duplication in libctf. But conveniently the compiler has never emitted this! Because it has always emitted a rather different format that libctf has never accepted, we can be sure that there are no instances of this function info section in the wild, and can freely change its format without compatibility concerns or a file format version bump. (And since it has never been emitted in any code that generated any older file format version, either, we need keep no code to read the format as specified at all!) So the function info section is now specified as an array of uint32_t, exactly like the object data section: each entry is a type ID in the type section which must be of kind CTF_K_FUNCTION, the prototype of this function. This allows function types to be deduplicated and also correctly encodes the fact that all functions declared in C really are types available to the program: so they should be stored in the type section like all other types. (In format v4, we will be able to represent the types of static functions as well, but that really does require a file format change.) We introduce a new header flag, CTF_F_NEWFUNCINFO, which is set if the new function info format is in use. A sufficiently new compiler will always set this flag. New libctf will always set this flag: old libctf will refuse to open any CTF dicts that have this flag set. If the flag is not set on a dict being read in, new libctf will disregard the function info section. Format v4 will remove this flag (or, rather, the flag has no meaning there and the bit position may be recycled for some other purpose). New API: Symbol addition: ctf_add_func_sym: Add a symbol with a given name and type. The type must be of kind CTF_K_FUNCTION (a function pointer). Internally this adds a name -> type mapping to the ctf_funchash in the ctf_dict. ctf_add_objt_sym: Add a symbol with a given name and type. The type kind can be anything, including function pointers. This adds to ctf_objthash. These both treat symbols as name -> type mappings: the linker associates symbol names with symbol indexes via the ctf_link_shuffle_syms callback, which sets up the ctf_dynsyms/ctf_dynsymidx/ctf_dynsymmax fields in the ctf_dict. Repeated relinks can add more symbols. Variables that are also exposed as symbols are removed from the variable section at serialization time. CTF symbol type sections which have enough pads, defined by CTF_INDEX_PAD_THRESHOLD (whether because they are in dicts with symbols where most types are unknown, or in archive where most types are defined in some child or parent dict, not in this specific dict) are sorted by name rather than symidx and accompanied by an index which associates each symbol type entry with a name: the existing ctf_lookup_by_symbol will map symbol indexes to symbol names and look the names up in the index automatically. (This is currently ELF-symbol-table-dependent, but there is almost nothing specific to ELF in here and we can add support for other symbol table formats easily). The compiler also uses index sections to communicate the contents of object file symbol tables without relying on any specific ordering of symbols: it doesn't need to sort them, and libctf will detect an unsorted index section via the absence of the new CTF_F_IDXSORTED header flag, and sort it if needed. Iteration: ctf_symbol_next: Iterator which returns the types and names of symbols one by one, either for function or data symbols. This does not require any sorting: the ctf_link machinery uses it to pull in all the compiler-provided symbols cheaply, but it is not restricted to that use. (Compatible) changes in API: ctf_lookup_by_symbol: can now be called for object and function symbols: never returns ECTF_NOTDATA (which is now not thrown by anything, but is kept for compatibility and because it is a plausible error that we might start throwing again at some later date). Internally we also have changes to the ctf-string functionality so that "external" strings (those where we track a string -> offset mapping, but only write out an offset) can be consulted via the usual means (ctf_strptr) before the strtab is written out. This is important because ctf_link_add_linker_symbol can now be handed symbols named via strtab offsets, and ctf_link_shuffle_syms must figure out their actual names by looking in the external symtab we have just been fed by the ctf_link_add_strtab callback, long before that strtab is written out. include/ChangeLog 2020-11-20 Nick Alcock <nick.alcock@oracle.com> * ctf-api.h (ctf_symbol_next): New. (ctf_add_objt_sym): Likewise. (ctf_add_func_sym): Likewise. * ctf.h: Document new function info section format. (CTF_F_NEWFUNCINFO): New. (CTF_F_IDXSORTED): New. (CTF_F_MAX): Adjust accordingly. libctf/ChangeLog 2020-11-20 Nick Alcock <nick.alcock@oracle.com> * ctf-impl.h (CTF_INDEX_PAD_THRESHOLD): New. (_libctf_nonnull_): Likewise. (ctf_in_flight_dynsym_t): New. (ctf_dict_t) <ctf_funcidx_names>: Likewise. <ctf_objtidx_names>: Likewise. <ctf_nfuncidx>: Likewise. <ctf_nobjtidx>: Likewise. <ctf_funcidx_sxlate>: Likewise. <ctf_objtidx_sxlate>: Likewise. <ctf_objthash>: Likewise. <ctf_funchash>: Likewise. <ctf_dynsyms>: Likewise. <ctf_dynsymidx>: Likewise. <ctf_dynsymmax>: Likewise. <ctf_in_flight_dynsym>: Likewise. (struct ctf_next) <u.ctn_next>: Likewise. (ctf_symtab_skippable): New prototype. (ctf_add_funcobjt_sym): Likewise. (ctf_dynhash_sort_by_name): Likewise. (ctf_sym_to_elf64): Rename to... (ctf_elf32_to_link_sym): ... this, and... (ctf_elf64_to_link_sym): ... this. * ctf-open.c (init_symtab): Check for lack of CTF_F_NEWFUNCINFO flag, and presence of index sections. Refactor out ctf_symtab_skippable and ctf_elf*_to_link_sym, and use them. Use ctf_link_sym_t, not Elf64_Sym. Skip initializing objt or func sxlate sections if corresponding index section is present. Adjust for new func info section format. (ctf_bufopen_internal): Add ctf_err_warn to corrupt-file error handling. Report incorrect-length index sections. Always do an init_symtab, even if there is no symtab section (there may be index sections still). (flip_objts): Adjust comment: func and objt sections are actually identical in structure now, no need to caveat. (ctf_dict_close): Free newly-added data structures. * ctf-create.c (ctf_create): Initialize them. (ctf_symtab_skippable): New, refactored out of init_symtab, with st_nameidx_set check added. (ctf_add_funcobjt_sym): New, add a function or object symbol to the ctf_objthash or ctf_funchash, by name. (ctf_add_objt_sym): Call it. (ctf_add_func_sym): Likewise. (symtypetab_delete_nonstatic_vars): New, delete vars also present as data objects. (CTF_SYMTYPETAB_EMIT_FUNCTION): New flag to symtypetab emitters: this is a function emission, not a data object emission. (CTF_SYMTYPETAB_EMIT_PAD): New flag to symtypetab emitters: emit pads for symbols with no type (only set for unindexed sections). (CTF_SYMTYPETAB_FORCE_INDEXED): New flag to symtypetab emitters: always emit indexed. (symtypetab_density): New, figure out section sizes. (emit_symtypetab): New, emit a symtypetab. (emit_symtypetab_index): New, emit a symtypetab index. (ctf_serialize): Call them, emitting suitably sorted symtypetab sections and indexes. Set suitable header flags. Copy over new fields. * ctf-hash.c (ctf_dynhash_sort_by_name): New, used to impose an order on symtypetab index sections. * ctf-link.c (ctf_add_type_mapping): Delete erroneous comment relating to code that was never committed. (ctf_link_one_variable): Improve variable name. (check_sym): New, symtypetab analogue of check_variable. (ctf_link_deduplicating_one_symtypetab): New. (ctf_link_deduplicating_syms): Likewise. (ctf_link_deduplicating): Call them. (ctf_link_deduplicating_per_cu): Note that we don't call them in this case (yet). (ctf_link_add_strtab): Set the error on the fp correctly. (ctf_link_add_linker_symbol): New (no longer a do-nothing stub), add a linker symbol to the in-flight list. (ctf_link_shuffle_syms): New (no longer a do-nothing stub), turn the in-flight list into a mapping we can use, now its names are resolvable in the external strtab. * ctf-string.c (ctf_str_rollback_atom): Don't roll back atoms with external strtab offsets. (ctf_str_rollback): Adjust comment. (ctf_str_write_strtab): Migrate ctf_syn_ext_strtab population from writeout time... (ctf_str_add_external): ... to string addition time. * ctf-lookup.c (ctf_lookup_var_key_t): Rename to... (ctf_lookup_idx_key_t): ... this, now we use it for syms too. <clik_names>: New member, a name table. (ctf_lookup_var): Adjust accordingly. (ctf_lookup_variable): Likewise. (ctf_lookup_by_id): Shuffle further up in the file. (ctf_symidx_sort_arg_cb): New, callback for... (sort_symidx_by_name): ... this new function to sort a symidx found to be unsorted (likely originating from the compiler). (ctf_symidx_sort): New, sort a symidx. (ctf_lookup_symbol_name): Support dynamic symbols with indexes provided by the linker. Use ctf_link_sym_t, not Elf64_Sym. Check the parent if a child lookup fails. (ctf_lookup_by_symbol): Likewise. Work for function symbols too. (ctf_symbol_next): New, iterate over symbols with types (without sorting). (ctf_lookup_idx_name): New, bsearch for symbol names in indexes. (ctf_try_lookup_indexed): New, attempt an indexed lookup. (ctf_func_info): Reimplement in terms of ctf_lookup_by_symbol. (ctf_func_args): Likewise. (ctf_get_dict): Move... * ctf-types.c (ctf_get_dict): ... here. * ctf-util.c (ctf_sym_to_elf64): Re-express as... (ctf_elf64_to_link_sym): ... this. Add new st_symidx field, and st_nameidx_set (always 0, so st_nameidx can be ignored). Look in the ELF strtab for names. (ctf_elf32_to_link_sym): Likewise, for Elf32_Sym. (ctf_next_destroy): Destroy ctf_next_t.u.ctn_next if need be. * libctf.ver: Add ctf_symbol_next, ctf_add_objt_sym and ctf_add_func_sym.
2020-11-20 21:34:04 +08:00
ELF dynamic symbol table for this object. CTF does not record any strings
that are already in the symbol table, and the CTF string table does not
contain any duplicated strings.
If the CTF data has been merged with another parent CTF object, some outgoing
edges may refer to type nodes that exist in another CTF object. The debugger
and libctf library are responsible for connecting the appropriate objects
together so that the full set of types can be explored and manipulated.
libctf, include: CTF-archive-wide symbol lookup CTF archives may contain multiple dicts, each of which contain many types and possibly a bunch of symtypetab entries relating to those types: each symtypetab entry is going to appear in exactly one dict, with the corresponding entries in the other dicts empty (either pads, or indexed symtypetabs that do not mention that symbol). But users of libctf usually want to get back the type associated with a symbol without having to dig around to find out which dict that type might be in. This adds machinery to do that -- and since you probably want to do it repeatedly, it adds internal caching to the ctf-archive machinery so that iteration over archives via ctf_archive_next and repeated symbol lookups do not have to repeatedly reopen the archive. (Iteration using ctf_archive_iter will gain caching soon.) Two new API functions: ctf_dict_t * ctf_arc_lookup_symbol (ctf_archive_t *arc, unsigned long symidx, ctf_id_t *typep, int *errp); This looks up the symbol with index SYMIDX in the archive ARC, returning the dictionary in which it resides and optionally the type index as well. Errors are returned in ERRP. The dict should be ctf_dict_close()d when done, but is also cached inside the ctf_archive so that the open cost is only paid once. The result of the symbol lookup is also cached internally, so repeated lookups of the same symbol are nearly free. void ctf_arc_flush_caches (ctf_archive_t *arc); Flush all the caches. Done at close time, but also available as an API function if users want to do it by hand. include/ChangeLog 2020-11-20 Nick Alcock <nick.alcock@oracle.com> * ctf-api.h (ctf_arc_lookup_symbol): New. (ctf_arc_flush_caches): Likewise. * ctf.h: Document new auto-ctf_import behaviour. libctf/ChangeLog 2020-11-20 Nick Alcock <nick.alcock@oracle.com> * ctf-impl.h (struct ctf_archive_internal) <ctfi_dicts>: New, dicts the archive machinery has opened and cached. <ctfi_symdicts>: New, cache of dicts containing symbols looked up. <ctfi_syms>: New, cache of types of symbols looked up. * ctf-archive.c (ctf_arc_close): Free them on close. (enosym): New, flag entry for 'symbol not present'. (ctf_arc_import_parent): New, automatically import the parent from ".ctf" if this is a child in an archive and ".ctf" is present. (ctf_dict_open_sections): Use it. (ctf_archive_iter_internal): Likewise. (ctf_cached_dict_close): New, thunk around ctf_dict_close. (ctf_dict_open_cached): New, open and cache a dict. (ctf_arc_flush_caches): New, flush the caches. (ctf_arc_lookup_symbol): New, look up a symbol in (all members of) an archive, and cache the lookup. (ctf_archive_iter): Note the new caching behaviour. (ctf_archive_next): Use ctf_dict_open_cached. * libctf.ver: Add ctf_arc_lookup_symbol and ctf_arc_flush_caches.
2020-11-20 21:34:04 +08:00
This connection is done purely using the ctf_import() function. The
ctf_archive machinery (and thus ctf_open et al) automatically imports archive
members named ".ctf" into child dicts if available in the same archive, to
match the relationship set up by the linker, but callers can call ctf_import
themselves as well if need be, if they know a different relationship is in
force. */
#define CTF_MAX_TYPE 0xfffffffe /* Max type identifier value. */
#define CTF_MAX_PTYPE 0x7fffffff /* Max parent type identifier value. */
#define CTF_MAX_NAME 0x7fffffff /* Max offset into a string table. */
#define CTF_MAX_VLEN 0xffffff /* Max struct, union, enum members or args. */
/* See ctf_type_t */
#define CTF_MAX_SIZE 0xfffffffe /* Max size of a v2 type in bytes. */
#define CTF_LSIZE_SENT 0xffffffff /* Sentinel for v2 ctt_size. */
# define CTF_MAX_TYPE_V1 0xffff /* Max type identifier value. */
# define CTF_MAX_PTYPE_V1 0x7fff /* Max parent type identifier value. */
# define CTF_MAX_VLEN_V1 0x3ff /* Max struct, union, enums or args. */
# define CTF_MAX_SIZE_V1 0xfffe /* Max size of a type in bytes. */
# define CTF_LSIZE_SENT_V1 0xffff /* Sentinel for v1 ctt_size. */
/* Start of actual data structure definitions.
Every field in these structures must have corresponding code in the
endianness-swapping machinery in libctf/ctf-open.c. */
typedef struct ctf_preamble
{
unsigned short ctp_magic; /* Magic number (CTF_MAGIC). */
unsigned char ctp_version; /* Data format version number (CTF_VERSION). */
unsigned char ctp_flags; /* Flags (see below). */
} ctf_preamble_t;
libctf: allow the header to change between versions libctf supports dynamic upgrading of the type table as file format versions change, but before now has not supported changes to the CTF header. Doing this is complicated by the baroque storage method used: the CTF header is kept prepended to the rest of the CTF data, just as when read from the file, and written out from there, and is endian-flipped in place. This makes accessing it needlessly hard and makes it almost impossible to make the header larger if we add fields. The general storage machinery around the malloced ctf pointer (the 'ctf_base') is also overcomplicated: the pointer is sometimes malloced locally and sometimes assigned from a parameter, so freeing it requires checking to see if that parameter was used, needlessly coupling ctf_bufopen and ctf_file_close together. So split the header out into a new ctf_file_t.ctf_header, which is written out explicitly: squeeze it out of the CTF buffer whenever we reallocate it, and use ctf_file_t.ctf_buf to skip past the header when we do not need to reallocate (when no upgrading or endian-flipping is required). We now track whether the CTF base can be freed explicitly via a new ctf_dynbase pointer which is non-NULL only when freeing is possible. With all this done, we can upgrade the header on the fly and add new fields as desired, via a new upgrade_header function in ctf-open. As with other forms of upgrading, libctf upgrades older headers automatically to the latest supported version at open time. For a first use of this field, we add a new string field cth_cuname, and a corresponding setter/getter pair ctf_cuname_set and ctf_cuname: this is used by debuggers to determine whether a CTF section's types relate to a single compilation unit, or to all compilation units in the program. (Types with ambiguous definitions in different CUs have only one of these types placed in the top-level shared .ctf container: the rest are placed in much smaller per-CU containers, which have the shared container as their parent. Since CTF must be useful in the absence of DWARF, we store the names of the relevant CUs ourselves, so the debugger can look them up.) v5: fix tabdamage. include/ * ctf-api.h (ctf_cuname): New function. (ctf_cuname_set): Likewise. * ctf.h: Improve comment around upgrading, no longer implying that v2 is the target of upgrades (it is v3 now). (ctf_header_v2_t): New, old-format header for backward compatibility. (ctf_header_t): Add cth_cuname: this is the first of several header changes in format v3. libctf/ * ctf-impl.h (ctf_file_t): New fields ctf_header, ctf_dynbase, ctf_cuname, ctf_dyncuname: ctf_base and ctf_buf are no longer const. * ctf-open.c (ctf_set_base): Preserve the gap between ctf_buf and ctf_base: do not assume that it is always sizeof (ctf_header_t). Print out ctf_cuname: only print out ctf_parname if set. (ctf_free_base): Removed, ctf_base is no longer freed: free ctf_dynbase instead. (ctf_set_version): Fix spacing. (upgrade_header): New, in-place header upgrading. (upgrade_types): Rename to... (upgrade_types_v1): ... this. Free ctf_dynbase, not ctf_base. No longer track old and new headers separately. No longer allow for header sizes explicitly: squeeze the headers out on upgrade (they are preserved in fp->ctf_header). Set ctf_dynbase, ctf_base and ctf_buf explicitly. Use ctf_free, not ctf_free_base. (upgrade_types): New, also handle ctf_parmax updating. (flip_header): Flip ctf_cuname. (flip_types): Flip BUF explicitly rather than deriving BUF from BASE. (ctf_bufopen): Store the header in fp->ctf_header. Correct minimum required alignment of objtoff and funcoff. No longer store it in the ctf_buf unless that buf is derived unmodified from the input. Set ctf_dynbase where ctf_base is dynamically allocated. Drop locals that duplicate fields in ctf_file: move allocation of ctf_file further up instead. Call upgrade_header as needed. Move version-specific ctf_parmax initialization into upgrade_types. More concise error handling. (ctf_file_close): No longer test for null pointers before freeing. Free ctf_dyncuname, ctf_dynbase, and ctf_header. Do not call ctf_free_base. (ctf_cuname): New. (ctf_cuname_set): New. * ctf-create.c (ctf_update): Populate ctf_cuname. (ctf_gzwrite): Write out the header explicitly. Remove obsolescent comment. (ctf_write): Likewise. (ctf_compress_write): Get the header from ctf_header, not ctf_base. Fix the compression length: fp->ctf_size never counted the CTF header. Simplify the compress call accordingly.
2019-07-07 00:36:21 +08:00
typedef struct ctf_header_v2
{
ctf_preamble_t cth_preamble;
uint32_t cth_parlabel; /* Ref to name of parent lbl uniq'd against. */
uint32_t cth_parname; /* Ref to basename of parent. */
uint32_t cth_lbloff; /* Offset of label section. */
uint32_t cth_objtoff; /* Offset of object section. */
uint32_t cth_funcoff; /* Offset of function section. */
uint32_t cth_varoff; /* Offset of variable section. */
uint32_t cth_typeoff; /* Offset of type section. */
uint32_t cth_stroff; /* Offset of string section. */
uint32_t cth_strlen; /* Length of string section in bytes. */
} ctf_header_v2_t;
typedef struct ctf_header
{
ctf_preamble_t cth_preamble;
uint32_t cth_parlabel; /* Ref to name of parent lbl uniq'd against. */
uint32_t cth_parname; /* Ref to basename of parent. */
libctf: allow the header to change between versions libctf supports dynamic upgrading of the type table as file format versions change, but before now has not supported changes to the CTF header. Doing this is complicated by the baroque storage method used: the CTF header is kept prepended to the rest of the CTF data, just as when read from the file, and written out from there, and is endian-flipped in place. This makes accessing it needlessly hard and makes it almost impossible to make the header larger if we add fields. The general storage machinery around the malloced ctf pointer (the 'ctf_base') is also overcomplicated: the pointer is sometimes malloced locally and sometimes assigned from a parameter, so freeing it requires checking to see if that parameter was used, needlessly coupling ctf_bufopen and ctf_file_close together. So split the header out into a new ctf_file_t.ctf_header, which is written out explicitly: squeeze it out of the CTF buffer whenever we reallocate it, and use ctf_file_t.ctf_buf to skip past the header when we do not need to reallocate (when no upgrading or endian-flipping is required). We now track whether the CTF base can be freed explicitly via a new ctf_dynbase pointer which is non-NULL only when freeing is possible. With all this done, we can upgrade the header on the fly and add new fields as desired, via a new upgrade_header function in ctf-open. As with other forms of upgrading, libctf upgrades older headers automatically to the latest supported version at open time. For a first use of this field, we add a new string field cth_cuname, and a corresponding setter/getter pair ctf_cuname_set and ctf_cuname: this is used by debuggers to determine whether a CTF section's types relate to a single compilation unit, or to all compilation units in the program. (Types with ambiguous definitions in different CUs have only one of these types placed in the top-level shared .ctf container: the rest are placed in much smaller per-CU containers, which have the shared container as their parent. Since CTF must be useful in the absence of DWARF, we store the names of the relevant CUs ourselves, so the debugger can look them up.) v5: fix tabdamage. include/ * ctf-api.h (ctf_cuname): New function. (ctf_cuname_set): Likewise. * ctf.h: Improve comment around upgrading, no longer implying that v2 is the target of upgrades (it is v3 now). (ctf_header_v2_t): New, old-format header for backward compatibility. (ctf_header_t): Add cth_cuname: this is the first of several header changes in format v3. libctf/ * ctf-impl.h (ctf_file_t): New fields ctf_header, ctf_dynbase, ctf_cuname, ctf_dyncuname: ctf_base and ctf_buf are no longer const. * ctf-open.c (ctf_set_base): Preserve the gap between ctf_buf and ctf_base: do not assume that it is always sizeof (ctf_header_t). Print out ctf_cuname: only print out ctf_parname if set. (ctf_free_base): Removed, ctf_base is no longer freed: free ctf_dynbase instead. (ctf_set_version): Fix spacing. (upgrade_header): New, in-place header upgrading. (upgrade_types): Rename to... (upgrade_types_v1): ... this. Free ctf_dynbase, not ctf_base. No longer track old and new headers separately. No longer allow for header sizes explicitly: squeeze the headers out on upgrade (they are preserved in fp->ctf_header). Set ctf_dynbase, ctf_base and ctf_buf explicitly. Use ctf_free, not ctf_free_base. (upgrade_types): New, also handle ctf_parmax updating. (flip_header): Flip ctf_cuname. (flip_types): Flip BUF explicitly rather than deriving BUF from BASE. (ctf_bufopen): Store the header in fp->ctf_header. Correct minimum required alignment of objtoff and funcoff. No longer store it in the ctf_buf unless that buf is derived unmodified from the input. Set ctf_dynbase where ctf_base is dynamically allocated. Drop locals that duplicate fields in ctf_file: move allocation of ctf_file further up instead. Call upgrade_header as needed. Move version-specific ctf_parmax initialization into upgrade_types. More concise error handling. (ctf_file_close): No longer test for null pointers before freeing. Free ctf_dyncuname, ctf_dynbase, and ctf_header. Do not call ctf_free_base. (ctf_cuname): New. (ctf_cuname_set): New. * ctf-create.c (ctf_update): Populate ctf_cuname. (ctf_gzwrite): Write out the header explicitly. Remove obsolescent comment. (ctf_write): Likewise. (ctf_compress_write): Get the header from ctf_header, not ctf_base. Fix the compression length: fp->ctf_size never counted the CTF header. Simplify the compress call accordingly.
2019-07-07 00:36:21 +08:00
uint32_t cth_cuname; /* Ref to CU name (may be 0). */
uint32_t cth_lbloff; /* Offset of label section. */
uint32_t cth_objtoff; /* Offset of object section. */
uint32_t cth_funcoff; /* Offset of function section. */
uint32_t cth_objtidxoff; /* Offset of object index section. */
uint32_t cth_funcidxoff; /* Offset of function index section. */
uint32_t cth_varoff; /* Offset of variable section. */
uint32_t cth_typeoff; /* Offset of type section. */
uint32_t cth_stroff; /* Offset of string section. */
uint32_t cth_strlen; /* Length of string section in bytes. */
} ctf_header_t;
#define cth_magic cth_preamble.ctp_magic
#define cth_version cth_preamble.ctp_version
#define cth_flags cth_preamble.ctp_flags
#define CTF_MAGIC 0xdff2 /* Magic number identifying header. */
/* Data format version number. */
libctf: allow the header to change between versions libctf supports dynamic upgrading of the type table as file format versions change, but before now has not supported changes to the CTF header. Doing this is complicated by the baroque storage method used: the CTF header is kept prepended to the rest of the CTF data, just as when read from the file, and written out from there, and is endian-flipped in place. This makes accessing it needlessly hard and makes it almost impossible to make the header larger if we add fields. The general storage machinery around the malloced ctf pointer (the 'ctf_base') is also overcomplicated: the pointer is sometimes malloced locally and sometimes assigned from a parameter, so freeing it requires checking to see if that parameter was used, needlessly coupling ctf_bufopen and ctf_file_close together. So split the header out into a new ctf_file_t.ctf_header, which is written out explicitly: squeeze it out of the CTF buffer whenever we reallocate it, and use ctf_file_t.ctf_buf to skip past the header when we do not need to reallocate (when no upgrading or endian-flipping is required). We now track whether the CTF base can be freed explicitly via a new ctf_dynbase pointer which is non-NULL only when freeing is possible. With all this done, we can upgrade the header on the fly and add new fields as desired, via a new upgrade_header function in ctf-open. As with other forms of upgrading, libctf upgrades older headers automatically to the latest supported version at open time. For a first use of this field, we add a new string field cth_cuname, and a corresponding setter/getter pair ctf_cuname_set and ctf_cuname: this is used by debuggers to determine whether a CTF section's types relate to a single compilation unit, or to all compilation units in the program. (Types with ambiguous definitions in different CUs have only one of these types placed in the top-level shared .ctf container: the rest are placed in much smaller per-CU containers, which have the shared container as their parent. Since CTF must be useful in the absence of DWARF, we store the names of the relevant CUs ourselves, so the debugger can look them up.) v5: fix tabdamage. include/ * ctf-api.h (ctf_cuname): New function. (ctf_cuname_set): Likewise. * ctf.h: Improve comment around upgrading, no longer implying that v2 is the target of upgrades (it is v3 now). (ctf_header_v2_t): New, old-format header for backward compatibility. (ctf_header_t): Add cth_cuname: this is the first of several header changes in format v3. libctf/ * ctf-impl.h (ctf_file_t): New fields ctf_header, ctf_dynbase, ctf_cuname, ctf_dyncuname: ctf_base and ctf_buf are no longer const. * ctf-open.c (ctf_set_base): Preserve the gap between ctf_buf and ctf_base: do not assume that it is always sizeof (ctf_header_t). Print out ctf_cuname: only print out ctf_parname if set. (ctf_free_base): Removed, ctf_base is no longer freed: free ctf_dynbase instead. (ctf_set_version): Fix spacing. (upgrade_header): New, in-place header upgrading. (upgrade_types): Rename to... (upgrade_types_v1): ... this. Free ctf_dynbase, not ctf_base. No longer track old and new headers separately. No longer allow for header sizes explicitly: squeeze the headers out on upgrade (they are preserved in fp->ctf_header). Set ctf_dynbase, ctf_base and ctf_buf explicitly. Use ctf_free, not ctf_free_base. (upgrade_types): New, also handle ctf_parmax updating. (flip_header): Flip ctf_cuname. (flip_types): Flip BUF explicitly rather than deriving BUF from BASE. (ctf_bufopen): Store the header in fp->ctf_header. Correct minimum required alignment of objtoff and funcoff. No longer store it in the ctf_buf unless that buf is derived unmodified from the input. Set ctf_dynbase where ctf_base is dynamically allocated. Drop locals that duplicate fields in ctf_file: move allocation of ctf_file further up instead. Call upgrade_header as needed. Move version-specific ctf_parmax initialization into upgrade_types. More concise error handling. (ctf_file_close): No longer test for null pointers before freeing. Free ctf_dyncuname, ctf_dynbase, and ctf_header. Do not call ctf_free_base. (ctf_cuname): New. (ctf_cuname_set): New. * ctf-create.c (ctf_update): Populate ctf_cuname. (ctf_gzwrite): Write out the header explicitly. Remove obsolescent comment. (ctf_write): Likewise. (ctf_compress_write): Get the header from ctf_header, not ctf_base. Fix the compression length: fp->ctf_size never counted the CTF header. Simplify the compress call accordingly.
2019-07-07 00:36:21 +08:00
/* v1 upgraded to a later version is not quite the same as the native form,
because the boundary between parent and child types is different but not
recorded anywhere, and you can write it out again via ctf_compress_write(),
so we must track whether the thing was originally v1 or not. If we were
writing the header from scratch, we would add a *pair* of version number
fields to allow for this, but this will do for now. (A flag will not do,
because we need to encode both the version we came from and the version we
went to, not just "we were upgraded".) */
# define CTF_VERSION_1 1
# define CTF_VERSION_1_UPGRADED_3 2
# define CTF_VERSION_2 3
#define CTF_VERSION_3 4
#define CTF_VERSION CTF_VERSION_3 /* Current version. */
libctf: symbol type linking support This adds facilities to write out the function info and data object sections, which efficiently map from entries in the symbol table to types. The write-side code is entirely new: the read-side code was merely significantly changed and support for indexed tables added (pointed to by the no-longer-unused cth_objtidxoff and cth_funcidxoff header fields). With this in place, you can use ctf_lookup_by_symbol to look up the types of symbols of function and object type (and, as before, you can use ctf_lookup_variable to look up types of file-scope variables not present in the symbol table, as long as you know their name: but variables that are also data objects are now found in the data object section instead.) (Compatible) file format change: The CTF spec has always said that the function info section looks much like the CTF_K_FUNCTIONs in the type section: an info word (including an argument count) followed by a return type and N argument types. This format is suboptimal: it means function symbols cannot be deduplicated and it causes a lot of ugly code duplication in libctf. But conveniently the compiler has never emitted this! Because it has always emitted a rather different format that libctf has never accepted, we can be sure that there are no instances of this function info section in the wild, and can freely change its format without compatibility concerns or a file format version bump. (And since it has never been emitted in any code that generated any older file format version, either, we need keep no code to read the format as specified at all!) So the function info section is now specified as an array of uint32_t, exactly like the object data section: each entry is a type ID in the type section which must be of kind CTF_K_FUNCTION, the prototype of this function. This allows function types to be deduplicated and also correctly encodes the fact that all functions declared in C really are types available to the program: so they should be stored in the type section like all other types. (In format v4, we will be able to represent the types of static functions as well, but that really does require a file format change.) We introduce a new header flag, CTF_F_NEWFUNCINFO, which is set if the new function info format is in use. A sufficiently new compiler will always set this flag. New libctf will always set this flag: old libctf will refuse to open any CTF dicts that have this flag set. If the flag is not set on a dict being read in, new libctf will disregard the function info section. Format v4 will remove this flag (or, rather, the flag has no meaning there and the bit position may be recycled for some other purpose). New API: Symbol addition: ctf_add_func_sym: Add a symbol with a given name and type. The type must be of kind CTF_K_FUNCTION (a function pointer). Internally this adds a name -> type mapping to the ctf_funchash in the ctf_dict. ctf_add_objt_sym: Add a symbol with a given name and type. The type kind can be anything, including function pointers. This adds to ctf_objthash. These both treat symbols as name -> type mappings: the linker associates symbol names with symbol indexes via the ctf_link_shuffle_syms callback, which sets up the ctf_dynsyms/ctf_dynsymidx/ctf_dynsymmax fields in the ctf_dict. Repeated relinks can add more symbols. Variables that are also exposed as symbols are removed from the variable section at serialization time. CTF symbol type sections which have enough pads, defined by CTF_INDEX_PAD_THRESHOLD (whether because they are in dicts with symbols where most types are unknown, or in archive where most types are defined in some child or parent dict, not in this specific dict) are sorted by name rather than symidx and accompanied by an index which associates each symbol type entry with a name: the existing ctf_lookup_by_symbol will map symbol indexes to symbol names and look the names up in the index automatically. (This is currently ELF-symbol-table-dependent, but there is almost nothing specific to ELF in here and we can add support for other symbol table formats easily). The compiler also uses index sections to communicate the contents of object file symbol tables without relying on any specific ordering of symbols: it doesn't need to sort them, and libctf will detect an unsorted index section via the absence of the new CTF_F_IDXSORTED header flag, and sort it if needed. Iteration: ctf_symbol_next: Iterator which returns the types and names of symbols one by one, either for function or data symbols. This does not require any sorting: the ctf_link machinery uses it to pull in all the compiler-provided symbols cheaply, but it is not restricted to that use. (Compatible) changes in API: ctf_lookup_by_symbol: can now be called for object and function symbols: never returns ECTF_NOTDATA (which is now not thrown by anything, but is kept for compatibility and because it is a plausible error that we might start throwing again at some later date). Internally we also have changes to the ctf-string functionality so that "external" strings (those where we track a string -> offset mapping, but only write out an offset) can be consulted via the usual means (ctf_strptr) before the strtab is written out. This is important because ctf_link_add_linker_symbol can now be handed symbols named via strtab offsets, and ctf_link_shuffle_syms must figure out their actual names by looking in the external symtab we have just been fed by the ctf_link_add_strtab callback, long before that strtab is written out. include/ChangeLog 2020-11-20 Nick Alcock <nick.alcock@oracle.com> * ctf-api.h (ctf_symbol_next): New. (ctf_add_objt_sym): Likewise. (ctf_add_func_sym): Likewise. * ctf.h: Document new function info section format. (CTF_F_NEWFUNCINFO): New. (CTF_F_IDXSORTED): New. (CTF_F_MAX): Adjust accordingly. libctf/ChangeLog 2020-11-20 Nick Alcock <nick.alcock@oracle.com> * ctf-impl.h (CTF_INDEX_PAD_THRESHOLD): New. (_libctf_nonnull_): Likewise. (ctf_in_flight_dynsym_t): New. (ctf_dict_t) <ctf_funcidx_names>: Likewise. <ctf_objtidx_names>: Likewise. <ctf_nfuncidx>: Likewise. <ctf_nobjtidx>: Likewise. <ctf_funcidx_sxlate>: Likewise. <ctf_objtidx_sxlate>: Likewise. <ctf_objthash>: Likewise. <ctf_funchash>: Likewise. <ctf_dynsyms>: Likewise. <ctf_dynsymidx>: Likewise. <ctf_dynsymmax>: Likewise. <ctf_in_flight_dynsym>: Likewise. (struct ctf_next) <u.ctn_next>: Likewise. (ctf_symtab_skippable): New prototype. (ctf_add_funcobjt_sym): Likewise. (ctf_dynhash_sort_by_name): Likewise. (ctf_sym_to_elf64): Rename to... (ctf_elf32_to_link_sym): ... this, and... (ctf_elf64_to_link_sym): ... this. * ctf-open.c (init_symtab): Check for lack of CTF_F_NEWFUNCINFO flag, and presence of index sections. Refactor out ctf_symtab_skippable and ctf_elf*_to_link_sym, and use them. Use ctf_link_sym_t, not Elf64_Sym. Skip initializing objt or func sxlate sections if corresponding index section is present. Adjust for new func info section format. (ctf_bufopen_internal): Add ctf_err_warn to corrupt-file error handling. Report incorrect-length index sections. Always do an init_symtab, even if there is no symtab section (there may be index sections still). (flip_objts): Adjust comment: func and objt sections are actually identical in structure now, no need to caveat. (ctf_dict_close): Free newly-added data structures. * ctf-create.c (ctf_create): Initialize them. (ctf_symtab_skippable): New, refactored out of init_symtab, with st_nameidx_set check added. (ctf_add_funcobjt_sym): New, add a function or object symbol to the ctf_objthash or ctf_funchash, by name. (ctf_add_objt_sym): Call it. (ctf_add_func_sym): Likewise. (symtypetab_delete_nonstatic_vars): New, delete vars also present as data objects. (CTF_SYMTYPETAB_EMIT_FUNCTION): New flag to symtypetab emitters: this is a function emission, not a data object emission. (CTF_SYMTYPETAB_EMIT_PAD): New flag to symtypetab emitters: emit pads for symbols with no type (only set for unindexed sections). (CTF_SYMTYPETAB_FORCE_INDEXED): New flag to symtypetab emitters: always emit indexed. (symtypetab_density): New, figure out section sizes. (emit_symtypetab): New, emit a symtypetab. (emit_symtypetab_index): New, emit a symtypetab index. (ctf_serialize): Call them, emitting suitably sorted symtypetab sections and indexes. Set suitable header flags. Copy over new fields. * ctf-hash.c (ctf_dynhash_sort_by_name): New, used to impose an order on symtypetab index sections. * ctf-link.c (ctf_add_type_mapping): Delete erroneous comment relating to code that was never committed. (ctf_link_one_variable): Improve variable name. (check_sym): New, symtypetab analogue of check_variable. (ctf_link_deduplicating_one_symtypetab): New. (ctf_link_deduplicating_syms): Likewise. (ctf_link_deduplicating): Call them. (ctf_link_deduplicating_per_cu): Note that we don't call them in this case (yet). (ctf_link_add_strtab): Set the error on the fp correctly. (ctf_link_add_linker_symbol): New (no longer a do-nothing stub), add a linker symbol to the in-flight list. (ctf_link_shuffle_syms): New (no longer a do-nothing stub), turn the in-flight list into a mapping we can use, now its names are resolvable in the external strtab. * ctf-string.c (ctf_str_rollback_atom): Don't roll back atoms with external strtab offsets. (ctf_str_rollback): Adjust comment. (ctf_str_write_strtab): Migrate ctf_syn_ext_strtab population from writeout time... (ctf_str_add_external): ... to string addition time. * ctf-lookup.c (ctf_lookup_var_key_t): Rename to... (ctf_lookup_idx_key_t): ... this, now we use it for syms too. <clik_names>: New member, a name table. (ctf_lookup_var): Adjust accordingly. (ctf_lookup_variable): Likewise. (ctf_lookup_by_id): Shuffle further up in the file. (ctf_symidx_sort_arg_cb): New, callback for... (sort_symidx_by_name): ... this new function to sort a symidx found to be unsorted (likely originating from the compiler). (ctf_symidx_sort): New, sort a symidx. (ctf_lookup_symbol_name): Support dynamic symbols with indexes provided by the linker. Use ctf_link_sym_t, not Elf64_Sym. Check the parent if a child lookup fails. (ctf_lookup_by_symbol): Likewise. Work for function symbols too. (ctf_symbol_next): New, iterate over symbols with types (without sorting). (ctf_lookup_idx_name): New, bsearch for symbol names in indexes. (ctf_try_lookup_indexed): New, attempt an indexed lookup. (ctf_func_info): Reimplement in terms of ctf_lookup_by_symbol. (ctf_func_args): Likewise. (ctf_get_dict): Move... * ctf-types.c (ctf_get_dict): ... here. * ctf-util.c (ctf_sym_to_elf64): Re-express as... (ctf_elf64_to_link_sym): ... this. Add new st_symidx field, and st_nameidx_set (always 0, so st_nameidx can be ignored). Look in the ELF strtab for names. (ctf_elf32_to_link_sym): Likewise, for Elf32_Sym. (ctf_next_destroy): Destroy ctf_next_t.u.ctn_next if need be. * libctf.ver: Add ctf_symbol_next, ctf_add_objt_sym and ctf_add_func_sym.
2020-11-20 21:34:04 +08:00
/* All of these flags bar CTF_F_COMPRESS and CTF_F_IDXSORTED are bug-workaround
flags and are valid only in format v3: in v2 and below they cannot occur and
in v4 and later, they will be recycled for other purposes. */
#define CTF_F_COMPRESS 0x1 /* Data buffer is compressed by libctf. */
libctf: symbol type linking support This adds facilities to write out the function info and data object sections, which efficiently map from entries in the symbol table to types. The write-side code is entirely new: the read-side code was merely significantly changed and support for indexed tables added (pointed to by the no-longer-unused cth_objtidxoff and cth_funcidxoff header fields). With this in place, you can use ctf_lookup_by_symbol to look up the types of symbols of function and object type (and, as before, you can use ctf_lookup_variable to look up types of file-scope variables not present in the symbol table, as long as you know their name: but variables that are also data objects are now found in the data object section instead.) (Compatible) file format change: The CTF spec has always said that the function info section looks much like the CTF_K_FUNCTIONs in the type section: an info word (including an argument count) followed by a return type and N argument types. This format is suboptimal: it means function symbols cannot be deduplicated and it causes a lot of ugly code duplication in libctf. But conveniently the compiler has never emitted this! Because it has always emitted a rather different format that libctf has never accepted, we can be sure that there are no instances of this function info section in the wild, and can freely change its format without compatibility concerns or a file format version bump. (And since it has never been emitted in any code that generated any older file format version, either, we need keep no code to read the format as specified at all!) So the function info section is now specified as an array of uint32_t, exactly like the object data section: each entry is a type ID in the type section which must be of kind CTF_K_FUNCTION, the prototype of this function. This allows function types to be deduplicated and also correctly encodes the fact that all functions declared in C really are types available to the program: so they should be stored in the type section like all other types. (In format v4, we will be able to represent the types of static functions as well, but that really does require a file format change.) We introduce a new header flag, CTF_F_NEWFUNCINFO, which is set if the new function info format is in use. A sufficiently new compiler will always set this flag. New libctf will always set this flag: old libctf will refuse to open any CTF dicts that have this flag set. If the flag is not set on a dict being read in, new libctf will disregard the function info section. Format v4 will remove this flag (or, rather, the flag has no meaning there and the bit position may be recycled for some other purpose). New API: Symbol addition: ctf_add_func_sym: Add a symbol with a given name and type. The type must be of kind CTF_K_FUNCTION (a function pointer). Internally this adds a name -> type mapping to the ctf_funchash in the ctf_dict. ctf_add_objt_sym: Add a symbol with a given name and type. The type kind can be anything, including function pointers. This adds to ctf_objthash. These both treat symbols as name -> type mappings: the linker associates symbol names with symbol indexes via the ctf_link_shuffle_syms callback, which sets up the ctf_dynsyms/ctf_dynsymidx/ctf_dynsymmax fields in the ctf_dict. Repeated relinks can add more symbols. Variables that are also exposed as symbols are removed from the variable section at serialization time. CTF symbol type sections which have enough pads, defined by CTF_INDEX_PAD_THRESHOLD (whether because they are in dicts with symbols where most types are unknown, or in archive where most types are defined in some child or parent dict, not in this specific dict) are sorted by name rather than symidx and accompanied by an index which associates each symbol type entry with a name: the existing ctf_lookup_by_symbol will map symbol indexes to symbol names and look the names up in the index automatically. (This is currently ELF-symbol-table-dependent, but there is almost nothing specific to ELF in here and we can add support for other symbol table formats easily). The compiler also uses index sections to communicate the contents of object file symbol tables without relying on any specific ordering of symbols: it doesn't need to sort them, and libctf will detect an unsorted index section via the absence of the new CTF_F_IDXSORTED header flag, and sort it if needed. Iteration: ctf_symbol_next: Iterator which returns the types and names of symbols one by one, either for function or data symbols. This does not require any sorting: the ctf_link machinery uses it to pull in all the compiler-provided symbols cheaply, but it is not restricted to that use. (Compatible) changes in API: ctf_lookup_by_symbol: can now be called for object and function symbols: never returns ECTF_NOTDATA (which is now not thrown by anything, but is kept for compatibility and because it is a plausible error that we might start throwing again at some later date). Internally we also have changes to the ctf-string functionality so that "external" strings (those where we track a string -> offset mapping, but only write out an offset) can be consulted via the usual means (ctf_strptr) before the strtab is written out. This is important because ctf_link_add_linker_symbol can now be handed symbols named via strtab offsets, and ctf_link_shuffle_syms must figure out their actual names by looking in the external symtab we have just been fed by the ctf_link_add_strtab callback, long before that strtab is written out. include/ChangeLog 2020-11-20 Nick Alcock <nick.alcock@oracle.com> * ctf-api.h (ctf_symbol_next): New. (ctf_add_objt_sym): Likewise. (ctf_add_func_sym): Likewise. * ctf.h: Document new function info section format. (CTF_F_NEWFUNCINFO): New. (CTF_F_IDXSORTED): New. (CTF_F_MAX): Adjust accordingly. libctf/ChangeLog 2020-11-20 Nick Alcock <nick.alcock@oracle.com> * ctf-impl.h (CTF_INDEX_PAD_THRESHOLD): New. (_libctf_nonnull_): Likewise. (ctf_in_flight_dynsym_t): New. (ctf_dict_t) <ctf_funcidx_names>: Likewise. <ctf_objtidx_names>: Likewise. <ctf_nfuncidx>: Likewise. <ctf_nobjtidx>: Likewise. <ctf_funcidx_sxlate>: Likewise. <ctf_objtidx_sxlate>: Likewise. <ctf_objthash>: Likewise. <ctf_funchash>: Likewise. <ctf_dynsyms>: Likewise. <ctf_dynsymidx>: Likewise. <ctf_dynsymmax>: Likewise. <ctf_in_flight_dynsym>: Likewise. (struct ctf_next) <u.ctn_next>: Likewise. (ctf_symtab_skippable): New prototype. (ctf_add_funcobjt_sym): Likewise. (ctf_dynhash_sort_by_name): Likewise. (ctf_sym_to_elf64): Rename to... (ctf_elf32_to_link_sym): ... this, and... (ctf_elf64_to_link_sym): ... this. * ctf-open.c (init_symtab): Check for lack of CTF_F_NEWFUNCINFO flag, and presence of index sections. Refactor out ctf_symtab_skippable and ctf_elf*_to_link_sym, and use them. Use ctf_link_sym_t, not Elf64_Sym. Skip initializing objt or func sxlate sections if corresponding index section is present. Adjust for new func info section format. (ctf_bufopen_internal): Add ctf_err_warn to corrupt-file error handling. Report incorrect-length index sections. Always do an init_symtab, even if there is no symtab section (there may be index sections still). (flip_objts): Adjust comment: func and objt sections are actually identical in structure now, no need to caveat. (ctf_dict_close): Free newly-added data structures. * ctf-create.c (ctf_create): Initialize them. (ctf_symtab_skippable): New, refactored out of init_symtab, with st_nameidx_set check added. (ctf_add_funcobjt_sym): New, add a function or object symbol to the ctf_objthash or ctf_funchash, by name. (ctf_add_objt_sym): Call it. (ctf_add_func_sym): Likewise. (symtypetab_delete_nonstatic_vars): New, delete vars also present as data objects. (CTF_SYMTYPETAB_EMIT_FUNCTION): New flag to symtypetab emitters: this is a function emission, not a data object emission. (CTF_SYMTYPETAB_EMIT_PAD): New flag to symtypetab emitters: emit pads for symbols with no type (only set for unindexed sections). (CTF_SYMTYPETAB_FORCE_INDEXED): New flag to symtypetab emitters: always emit indexed. (symtypetab_density): New, figure out section sizes. (emit_symtypetab): New, emit a symtypetab. (emit_symtypetab_index): New, emit a symtypetab index. (ctf_serialize): Call them, emitting suitably sorted symtypetab sections and indexes. Set suitable header flags. Copy over new fields. * ctf-hash.c (ctf_dynhash_sort_by_name): New, used to impose an order on symtypetab index sections. * ctf-link.c (ctf_add_type_mapping): Delete erroneous comment relating to code that was never committed. (ctf_link_one_variable): Improve variable name. (check_sym): New, symtypetab analogue of check_variable. (ctf_link_deduplicating_one_symtypetab): New. (ctf_link_deduplicating_syms): Likewise. (ctf_link_deduplicating): Call them. (ctf_link_deduplicating_per_cu): Note that we don't call them in this case (yet). (ctf_link_add_strtab): Set the error on the fp correctly. (ctf_link_add_linker_symbol): New (no longer a do-nothing stub), add a linker symbol to the in-flight list. (ctf_link_shuffle_syms): New (no longer a do-nothing stub), turn the in-flight list into a mapping we can use, now its names are resolvable in the external strtab. * ctf-string.c (ctf_str_rollback_atom): Don't roll back atoms with external strtab offsets. (ctf_str_rollback): Adjust comment. (ctf_str_write_strtab): Migrate ctf_syn_ext_strtab population from writeout time... (ctf_str_add_external): ... to string addition time. * ctf-lookup.c (ctf_lookup_var_key_t): Rename to... (ctf_lookup_idx_key_t): ... this, now we use it for syms too. <clik_names>: New member, a name table. (ctf_lookup_var): Adjust accordingly. (ctf_lookup_variable): Likewise. (ctf_lookup_by_id): Shuffle further up in the file. (ctf_symidx_sort_arg_cb): New, callback for... (sort_symidx_by_name): ... this new function to sort a symidx found to be unsorted (likely originating from the compiler). (ctf_symidx_sort): New, sort a symidx. (ctf_lookup_symbol_name): Support dynamic symbols with indexes provided by the linker. Use ctf_link_sym_t, not Elf64_Sym. Check the parent if a child lookup fails. (ctf_lookup_by_symbol): Likewise. Work for function symbols too. (ctf_symbol_next): New, iterate over symbols with types (without sorting). (ctf_lookup_idx_name): New, bsearch for symbol names in indexes. (ctf_try_lookup_indexed): New, attempt an indexed lookup. (ctf_func_info): Reimplement in terms of ctf_lookup_by_symbol. (ctf_func_args): Likewise. (ctf_get_dict): Move... * ctf-types.c (ctf_get_dict): ... here. * ctf-util.c (ctf_sym_to_elf64): Re-express as... (ctf_elf64_to_link_sym): ... this. Add new st_symidx field, and st_nameidx_set (always 0, so st_nameidx can be ignored). Look in the ELF strtab for names. (ctf_elf32_to_link_sym): Likewise, for Elf32_Sym. (ctf_next_destroy): Destroy ctf_next_t.u.ctn_next if need be. * libctf.ver: Add ctf_symbol_next, ctf_add_objt_sym and ctf_add_func_sym.
2020-11-20 21:34:04 +08:00
#define CTF_F_NEWFUNCINFO 0x2 /* New v3 func info section format. */
#define CTF_F_IDXSORTED 0x4 /* Index sections already sorted. */
bfd, include, ld, binutils, libctf: CTF should use the dynstr/sym This is embarrassing. The whole point of CTF is that it remains intact even after a binary is stripped, providing a compact mapping from symbols to types for everything in the externally-visible interface of an ELF object: it has connections to the symbol table for that purpose, and to the string table to avoid duplicating symbol names. So it's a shame that the hooks I implemented last year served to hook it up to the .symtab and .strtab, which obviously disappear on strip, leaving any accompanying the CTF dict containing references to strings (and, soon, symbols) which don't exist any more because their containing strtab has been vaporized. The original Solaris design used .dynsym and .dynstr (well, actually, .ldynsym, which has more symbols) which do not disappear. So should we. Thankfully the work we did before serves as guide rails, and adjusting things to use the .dynstr and .dynsym was fast and easy. The only annoyance is that the dynsym is assembled inside elflink.c in a fairly piecemeal fashion, so that the easiest way to get the symbols out was to hook in before every call to swap_symbol_out (we also leave in a hook in front of symbol additions to the .symtab because it seems plausible that we might want to hook them in future too: for now that hook is unused). We adjust things so that rather than being offered a whole hash table of symbols at once, libctf is now given symbols one at a time, with st_name indexes already resolved and pointing at their final .dynstr offsets: it's now up to libctf to resolve these to names as needed using the strtab info we pass it separately. Some bits might be contentious. The ctf_new_dynstr callback takes an elf_internal_sym, and this remains an elf_internal_sym right down through the generic emulation layers into ldelfgen. This is no worse than the elf_sym_strtab we used to pass down, but in the future when we gain non-ELF CTF symtab support we might want to lower the elf_internal_sym to some other representation (perhaps a ctf_link_symbol) in bfd or in ldlang_ctf_new_dynsym. We rename the 'apply_strsym' hooks to 'acquire_strings' instead, becuse they no longer have anything to do with symbols. There are some API changes to pieces of API which are technically public but actually totally unused by anything and/or unused by anything but ld so they can change freely: the ctf_link_symbol gains new fields to allow symbol names to be given as strtab offsets as well as strings, and a symidx so that the symbol index can be passed in. ctf_link_shuffle_syms loses its callback parameter: the idea now is that linkers call the new ctf_link_add_linker_symbol for every symbol in .dynsym, feed in all the strtab entries with ctf_link_add_strtab, and then a call to ctf_link_shuffle_syms will apply both and arrange to use them to reorder the CTF symtab at CTF serialization time (which is coming in the next commit). Inside libctf we have a new preamble flag CTF_F_DYNSTR which is always set in v3-format CTF dicts from this commit forwards: CTF dicts without this flag are associated with .strtab like they used to be, so that old dicts' external strings don't turn to garbage when loaded by new libctf. Dicts with this flag are associated with .dynstr and .dynsym instead. (The flag is not the next in sequence because this commit was written quite late: the missing flags will be filled in by the next commit.) Tests forthcoming in a later commit in this series. bfd/ChangeLog 2020-11-20 Nick Alcock <nick.alcock@oracle.com> * elflink.c (elf_finalize_dynstr): Call examine_strtab after dynstr finalization. (elf_link_swap_symbols_out): Don't call it here. Call ctf_new_symbol before swap_symbol_out. (elf_link_output_extsym): Call ctf_new_dynsym before swap_symbol_out. (bfd_elf_final_link): Likewise. * elf.c (swap_out_syms): Pass in bfd_link_info. Call ctf_new_symbol before swap_symbol_out. (_bfd_elf_compute_section_file_positions): Adjust. binutils/ChangeLog 2020-11-20 Nick Alcock <nick.alcock@oracle.com> * readelf.c (dump_section_as_ctf): Use .dynsym and .dynstr, not .symtab and .strtab. include/ChangeLog 2020-11-20 Nick Alcock <nick.alcock@oracle.com> * bfdlink.h (struct elf_sym_strtab): Replace with... (struct elf_internal_sym): ... this. (struct bfd_link_callbacks) <examine_strtab>: Take only a symstrtab argument. <ctf_new_symbol>: New. <ctf_new_dynsym>: Likewise. * ctf-api.h (struct ctf_link_sym) <st_symidx>: New. <st_nameidx>: Likewise. <st_nameidx_set>: Likewise. (ctf_link_iter_symbol_f): Removed. (ctf_link_shuffle_syms): Remove most parameters, just takes a ctf_dict_t now. (ctf_link_add_linker_symbol): New, split from ctf_link_shuffle_syms. * ctf.h (CTF_F_DYNSTR): New. (CTF_F_MAX): Adjust. ld/ChangeLog 2020-11-20 Nick Alcock <nick.alcock@oracle.com> * ldelfgen.c (struct ctf_strsym_iter_cb_arg): Rename to... (struct ctf_strtab_iter_cb_arg): ... this, changing fields: <syms>: Remove. <symcount>: Remove. <symstrtab>: Rename to... <strtab>: ... this. (ldelf_ctf_strtab_iter_cb): Adjust. (ldelf_ctf_symbols_iter_cb): Remove. (ldelf_new_dynsym_for_ctf): New, tell libctf about a single symbol. (ldelf_examine_strtab_for_ctf): Rename to... (ldelf_acquire_strings_for_ctf): ... this, only doing the strtab portion and not symbols. * ldelfgen.h: Adjust declarations accordingly. * ldemul.c (ldemul_examine_strtab_for_ctf): Rename to... (ldemul_acquire_strings_for_ctf): ... this. (ldemul_new_dynsym_for_ctf): New. * ldemul.h: Adjust declarations accordingly. * ldlang.c (ldlang_ctf_apply_strsym): Rename to... (ldlang_ctf_acquire_strings): ... this. (ldlang_ctf_new_dynsym): New. (lang_write_ctf): Call ldemul_new_dynsym_for_ctf with NULL to do the actual symbol shuffle. * ldlang.h (struct elf_strtab_hash): Adjust accordingly. * ldmain.c (bfd_link_callbacks): Wire up new/renamed callbacks. libctf/ChangeLog 2020-11-20 Nick Alcock <nick.alcock@oracle.com> * ctf-link.c (ctf_link_shuffle_syms): Adjust. (ctf_link_add_linker_symbol): New, unimplemented stub. * libctf.ver: Add it. * ctf-create.c (ctf_serialize): Set CTF_F_DYNSTR on newly-serialized dicts. * ctf-open-bfd.c (ctf_bfdopen_ctfsect): Check for the flag: open the symtab/strtab if not present, dynsym/dynstr otherwise. * ctf-archive.c (ctf_arc_bufpreamble): New, get the preamble from some arbitrary member of a CTF archive. * ctf-impl.h (ctf_arc_bufpreamble): Declare it.
2020-11-20 21:34:04 +08:00
#define CTF_F_DYNSTR 0x8 /* Strings come from .dynstr. */
libctf: symbol type linking support This adds facilities to write out the function info and data object sections, which efficiently map from entries in the symbol table to types. The write-side code is entirely new: the read-side code was merely significantly changed and support for indexed tables added (pointed to by the no-longer-unused cth_objtidxoff and cth_funcidxoff header fields). With this in place, you can use ctf_lookup_by_symbol to look up the types of symbols of function and object type (and, as before, you can use ctf_lookup_variable to look up types of file-scope variables not present in the symbol table, as long as you know their name: but variables that are also data objects are now found in the data object section instead.) (Compatible) file format change: The CTF spec has always said that the function info section looks much like the CTF_K_FUNCTIONs in the type section: an info word (including an argument count) followed by a return type and N argument types. This format is suboptimal: it means function symbols cannot be deduplicated and it causes a lot of ugly code duplication in libctf. But conveniently the compiler has never emitted this! Because it has always emitted a rather different format that libctf has never accepted, we can be sure that there are no instances of this function info section in the wild, and can freely change its format without compatibility concerns or a file format version bump. (And since it has never been emitted in any code that generated any older file format version, either, we need keep no code to read the format as specified at all!) So the function info section is now specified as an array of uint32_t, exactly like the object data section: each entry is a type ID in the type section which must be of kind CTF_K_FUNCTION, the prototype of this function. This allows function types to be deduplicated and also correctly encodes the fact that all functions declared in C really are types available to the program: so they should be stored in the type section like all other types. (In format v4, we will be able to represent the types of static functions as well, but that really does require a file format change.) We introduce a new header flag, CTF_F_NEWFUNCINFO, which is set if the new function info format is in use. A sufficiently new compiler will always set this flag. New libctf will always set this flag: old libctf will refuse to open any CTF dicts that have this flag set. If the flag is not set on a dict being read in, new libctf will disregard the function info section. Format v4 will remove this flag (or, rather, the flag has no meaning there and the bit position may be recycled for some other purpose). New API: Symbol addition: ctf_add_func_sym: Add a symbol with a given name and type. The type must be of kind CTF_K_FUNCTION (a function pointer). Internally this adds a name -> type mapping to the ctf_funchash in the ctf_dict. ctf_add_objt_sym: Add a symbol with a given name and type. The type kind can be anything, including function pointers. This adds to ctf_objthash. These both treat symbols as name -> type mappings: the linker associates symbol names with symbol indexes via the ctf_link_shuffle_syms callback, which sets up the ctf_dynsyms/ctf_dynsymidx/ctf_dynsymmax fields in the ctf_dict. Repeated relinks can add more symbols. Variables that are also exposed as symbols are removed from the variable section at serialization time. CTF symbol type sections which have enough pads, defined by CTF_INDEX_PAD_THRESHOLD (whether because they are in dicts with symbols where most types are unknown, or in archive where most types are defined in some child or parent dict, not in this specific dict) are sorted by name rather than symidx and accompanied by an index which associates each symbol type entry with a name: the existing ctf_lookup_by_symbol will map symbol indexes to symbol names and look the names up in the index automatically. (This is currently ELF-symbol-table-dependent, but there is almost nothing specific to ELF in here and we can add support for other symbol table formats easily). The compiler also uses index sections to communicate the contents of object file symbol tables without relying on any specific ordering of symbols: it doesn't need to sort them, and libctf will detect an unsorted index section via the absence of the new CTF_F_IDXSORTED header flag, and sort it if needed. Iteration: ctf_symbol_next: Iterator which returns the types and names of symbols one by one, either for function or data symbols. This does not require any sorting: the ctf_link machinery uses it to pull in all the compiler-provided symbols cheaply, but it is not restricted to that use. (Compatible) changes in API: ctf_lookup_by_symbol: can now be called for object and function symbols: never returns ECTF_NOTDATA (which is now not thrown by anything, but is kept for compatibility and because it is a plausible error that we might start throwing again at some later date). Internally we also have changes to the ctf-string functionality so that "external" strings (those where we track a string -> offset mapping, but only write out an offset) can be consulted via the usual means (ctf_strptr) before the strtab is written out. This is important because ctf_link_add_linker_symbol can now be handed symbols named via strtab offsets, and ctf_link_shuffle_syms must figure out their actual names by looking in the external symtab we have just been fed by the ctf_link_add_strtab callback, long before that strtab is written out. include/ChangeLog 2020-11-20 Nick Alcock <nick.alcock@oracle.com> * ctf-api.h (ctf_symbol_next): New. (ctf_add_objt_sym): Likewise. (ctf_add_func_sym): Likewise. * ctf.h: Document new function info section format. (CTF_F_NEWFUNCINFO): New. (CTF_F_IDXSORTED): New. (CTF_F_MAX): Adjust accordingly. libctf/ChangeLog 2020-11-20 Nick Alcock <nick.alcock@oracle.com> * ctf-impl.h (CTF_INDEX_PAD_THRESHOLD): New. (_libctf_nonnull_): Likewise. (ctf_in_flight_dynsym_t): New. (ctf_dict_t) <ctf_funcidx_names>: Likewise. <ctf_objtidx_names>: Likewise. <ctf_nfuncidx>: Likewise. <ctf_nobjtidx>: Likewise. <ctf_funcidx_sxlate>: Likewise. <ctf_objtidx_sxlate>: Likewise. <ctf_objthash>: Likewise. <ctf_funchash>: Likewise. <ctf_dynsyms>: Likewise. <ctf_dynsymidx>: Likewise. <ctf_dynsymmax>: Likewise. <ctf_in_flight_dynsym>: Likewise. (struct ctf_next) <u.ctn_next>: Likewise. (ctf_symtab_skippable): New prototype. (ctf_add_funcobjt_sym): Likewise. (ctf_dynhash_sort_by_name): Likewise. (ctf_sym_to_elf64): Rename to... (ctf_elf32_to_link_sym): ... this, and... (ctf_elf64_to_link_sym): ... this. * ctf-open.c (init_symtab): Check for lack of CTF_F_NEWFUNCINFO flag, and presence of index sections. Refactor out ctf_symtab_skippable and ctf_elf*_to_link_sym, and use them. Use ctf_link_sym_t, not Elf64_Sym. Skip initializing objt or func sxlate sections if corresponding index section is present. Adjust for new func info section format. (ctf_bufopen_internal): Add ctf_err_warn to corrupt-file error handling. Report incorrect-length index sections. Always do an init_symtab, even if there is no symtab section (there may be index sections still). (flip_objts): Adjust comment: func and objt sections are actually identical in structure now, no need to caveat. (ctf_dict_close): Free newly-added data structures. * ctf-create.c (ctf_create): Initialize them. (ctf_symtab_skippable): New, refactored out of init_symtab, with st_nameidx_set check added. (ctf_add_funcobjt_sym): New, add a function or object symbol to the ctf_objthash or ctf_funchash, by name. (ctf_add_objt_sym): Call it. (ctf_add_func_sym): Likewise. (symtypetab_delete_nonstatic_vars): New, delete vars also present as data objects. (CTF_SYMTYPETAB_EMIT_FUNCTION): New flag to symtypetab emitters: this is a function emission, not a data object emission. (CTF_SYMTYPETAB_EMIT_PAD): New flag to symtypetab emitters: emit pads for symbols with no type (only set for unindexed sections). (CTF_SYMTYPETAB_FORCE_INDEXED): New flag to symtypetab emitters: always emit indexed. (symtypetab_density): New, figure out section sizes. (emit_symtypetab): New, emit a symtypetab. (emit_symtypetab_index): New, emit a symtypetab index. (ctf_serialize): Call them, emitting suitably sorted symtypetab sections and indexes. Set suitable header flags. Copy over new fields. * ctf-hash.c (ctf_dynhash_sort_by_name): New, used to impose an order on symtypetab index sections. * ctf-link.c (ctf_add_type_mapping): Delete erroneous comment relating to code that was never committed. (ctf_link_one_variable): Improve variable name. (check_sym): New, symtypetab analogue of check_variable. (ctf_link_deduplicating_one_symtypetab): New. (ctf_link_deduplicating_syms): Likewise. (ctf_link_deduplicating): Call them. (ctf_link_deduplicating_per_cu): Note that we don't call them in this case (yet). (ctf_link_add_strtab): Set the error on the fp correctly. (ctf_link_add_linker_symbol): New (no longer a do-nothing stub), add a linker symbol to the in-flight list. (ctf_link_shuffle_syms): New (no longer a do-nothing stub), turn the in-flight list into a mapping we can use, now its names are resolvable in the external strtab. * ctf-string.c (ctf_str_rollback_atom): Don't roll back atoms with external strtab offsets. (ctf_str_rollback): Adjust comment. (ctf_str_write_strtab): Migrate ctf_syn_ext_strtab population from writeout time... (ctf_str_add_external): ... to string addition time. * ctf-lookup.c (ctf_lookup_var_key_t): Rename to... (ctf_lookup_idx_key_t): ... this, now we use it for syms too. <clik_names>: New member, a name table. (ctf_lookup_var): Adjust accordingly. (ctf_lookup_variable): Likewise. (ctf_lookup_by_id): Shuffle further up in the file. (ctf_symidx_sort_arg_cb): New, callback for... (sort_symidx_by_name): ... this new function to sort a symidx found to be unsorted (likely originating from the compiler). (ctf_symidx_sort): New, sort a symidx. (ctf_lookup_symbol_name): Support dynamic symbols with indexes provided by the linker. Use ctf_link_sym_t, not Elf64_Sym. Check the parent if a child lookup fails. (ctf_lookup_by_symbol): Likewise. Work for function symbols too. (ctf_symbol_next): New, iterate over symbols with types (without sorting). (ctf_lookup_idx_name): New, bsearch for symbol names in indexes. (ctf_try_lookup_indexed): New, attempt an indexed lookup. (ctf_func_info): Reimplement in terms of ctf_lookup_by_symbol. (ctf_func_args): Likewise. (ctf_get_dict): Move... * ctf-types.c (ctf_get_dict): ... here. * ctf-util.c (ctf_sym_to_elf64): Re-express as... (ctf_elf64_to_link_sym): ... this. Add new st_symidx field, and st_nameidx_set (always 0, so st_nameidx can be ignored). Look in the ELF strtab for names. (ctf_elf32_to_link_sym): Likewise, for Elf32_Sym. (ctf_next_destroy): Destroy ctf_next_t.u.ctn_next if need be. * libctf.ver: Add ctf_symbol_next, ctf_add_objt_sym and ctf_add_func_sym.
2020-11-20 21:34:04 +08:00
#define CTF_F_MAX (CTF_F_COMPRESS | CTF_F_NEWFUNCINFO | CTF_F_IDXSORTED \
| CTF_F_DYNSTR)
typedef struct ctf_lblent
{
uint32_t ctl_label; /* Ref to name of label. */
uint32_t ctl_type; /* Last type associated with this label. */
} ctf_lblent_t;
typedef struct ctf_varent
{
uint32_t ctv_name; /* Reference to name in string table. */
uint32_t ctv_type; /* Index of type of this variable. */
} ctf_varent_t;
/* In format v2, type sizes, measured in bytes, come in two flavours. Nearly
all of them fit into a (UINT_MAX - 1), and thus can be stored in the ctt_size
member of a ctf_stype_t. The maximum value for these sizes is CTF_MAX_SIZE.
Types larger than this must be stored in the ctf_lsize member of a
ctf_type_t. Use of this member is indicated by the presence of
CTF_LSIZE_SENT in ctt_size. */
/* In v1, the same applies, only the limit is (USHRT_MAX - 1) and
CTF_MAX_SIZE_V1, and CTF_LSIZE_SENT_V1 is the sentinel. */
typedef struct ctf_stype_v1
{
uint32_t ctt_name; /* Reference to name in string table. */
unsigned short ctt_info; /* Encoded kind, variant length (see below). */
#ifndef __GNUC__
union
{
unsigned short _size; /* Size of entire type in bytes. */
unsigned short _type; /* Reference to another type. */
} _u;
#else
__extension__
union
{
unsigned short ctt_size; /* Size of entire type in bytes. */
unsigned short ctt_type; /* Reference to another type. */
};
#endif
} ctf_stype_v1_t;
typedef struct ctf_type_v1
{
uint32_t ctt_name; /* Reference to name in string table. */
unsigned short ctt_info; /* Encoded kind, variant length (see below). */
#ifndef __GNUC__
union
{
unsigned short _size; /* Always CTF_LSIZE_SENT_V1. */
unsigned short _type; /* Do not use. */
} _u;
#else
__extension__
union
{
unsigned short ctt_size; /* Always CTF_LSIZE_SENT_V1. */
unsigned short ctt_type; /* Do not use. */
};
#endif
uint32_t ctt_lsizehi; /* High 32 bits of type size in bytes. */
uint32_t ctt_lsizelo; /* Low 32 bits of type size in bytes. */
} ctf_type_v1_t;
typedef struct ctf_stype
{
uint32_t ctt_name; /* Reference to name in string table. */
uint32_t ctt_info; /* Encoded kind, variant length (see below). */
#ifndef __GNUC__
union
{
uint32_t _size; /* Size of entire type in bytes. */
uint32_t _type; /* Reference to another type. */
} _u;
#else
__extension__
union
{
uint32_t ctt_size; /* Size of entire type in bytes. */
uint32_t ctt_type; /* Reference to another type. */
};
#endif
} ctf_stype_t;
typedef struct ctf_type
{
uint32_t ctt_name; /* Reference to name in string table. */
uint32_t ctt_info; /* Encoded kind, variant length (see below). */
#ifndef __GNUC__
union
{
uint32_t _size; /* Always CTF_LSIZE_SENT. */
uint32_t _type; /* Do not use. */
} _u;
#else
__extension__
union
{
uint32_t ctt_size; /* Always CTF_LSIZE_SENT. */
uint32_t ctt_type; /* Do not use. */
};
#endif
uint32_t ctt_lsizehi; /* High 32 bits of type size in bytes. */
uint32_t ctt_lsizelo; /* Low 32 bits of type size in bytes. */
} ctf_type_t;
#ifndef __GNUC__
#define ctt_size _u._size /* For fundamental types that have a size. */
#define ctt_type _u._type /* For types that reference another type. */
#endif
/* The following macros and inline functions compose and decompose values for
ctt_info and ctt_name, as well as other structures that contain name
references. Use outside libdtrace-ctf itself is explicitly for access to CTF
files directly: types returned from the library will always appear to be
CTF_V2.
v1: (transparently upgraded to v2 at open time: may be compiled out of the
library)
------------------------
ctt_info: | kind | isroot | vlen |
------------------------
15 11 10 9 0
v2:
------------------------
ctt_info: | kind | isroot | vlen |
------------------------
31 26 25 24 0
CTF_V1 and V2 _INFO_VLEN have the same interface:
kind = CTF_*_INFO_KIND(c.ctt_info); <-- CTF_K_* value (see below)
vlen = CTF_*_INFO_VLEN(fp, c.ctt_info); <-- length of variable data list
stid = CTF_NAME_STID(c.ctt_name); <-- string table id number (0 or 1)
offset = CTF_NAME_OFFSET(c.ctt_name); <-- string table byte offset
c.ctt_info = CTF_TYPE_INFO(kind, vlen);
c.ctt_name = CTF_TYPE_NAME(stid, offset); */
#define CTF_V1_INFO_KIND(info) (((info) & 0xf800) >> 11)
#define CTF_V1_INFO_ISROOT(info) (((info) & 0x0400) >> 10)
#define CTF_V1_INFO_VLEN(info) (((info) & CTF_MAX_VLEN_V1))
#define CTF_V2_INFO_KIND(info) (((info) & 0xfc000000) >> 26)
#define CTF_V2_INFO_ISROOT(info) (((info) & 0x2000000) >> 25)
#define CTF_V2_INFO_VLEN(info) (((info) & CTF_MAX_VLEN))
#define CTF_NAME_STID(name) ((name) >> 31)
#define CTF_NAME_OFFSET(name) ((name) & CTF_MAX_NAME)
#define CTF_SET_STID(name, stid) ((name) | ((unsigned int) stid) << 31)
/* V2 only. */
#define CTF_TYPE_INFO(kind, isroot, vlen) \
(((kind) << 26) | (((isroot) ? 1 : 0) << 25) | ((vlen) & CTF_MAX_VLEN))
#define CTF_TYPE_NAME(stid, offset) \
(((stid) << 31) | ((offset) & CTF_MAX_NAME))
/* The next set of macros are for public consumption only. Not used internally,
since the relevant type boundary is dependent upon the version of the file at
*opening* time, not the version after transparent upgrade. Use
ctf_type_isparent() / ctf_type_ischild() for that. */
#define CTF_V2_TYPE_ISPARENT(fp, id) ((id) <= CTF_MAX_PTYPE)
#define CTF_V2_TYPE_ISCHILD(fp, id) ((id) > CTF_MAX_PTYPE)
#define CTF_V2_TYPE_TO_INDEX(id) ((id) & CTF_MAX_PTYPE)
#define CTF_V2_INDEX_TO_TYPE(id, child) ((child) ? ((id) | (CTF_MAX_PTYPE+1)) : (id))
#define CTF_V1_TYPE_ISPARENT(fp, id) ((id) <= CTF_MAX_PTYPE_V1)
#define CTF_V1_TYPE_ISCHILD(fp, id) ((id) > CTF_MAX_PTYPE_V1)
#define CTF_V1_TYPE_TO_INDEX(id) ((id) & CTF_MAX_PTYPE_V1)
#define CTF_V1_INDEX_TO_TYPE(id, child) ((child) ? ((id) | (CTF_MAX_PTYPE_V1+1)) : (id))
/* Valid for both V1 and V2. */
#define CTF_TYPE_LSIZE(cttp) \
(((uint64_t)(cttp)->ctt_lsizehi) << 32 | (cttp)->ctt_lsizelo)
#define CTF_SIZE_TO_LSIZE_HI(size) ((uint32_t)((uint64_t)(size) >> 32))
#define CTF_SIZE_TO_LSIZE_LO(size) ((uint32_t)(size))
#define CTF_STRTAB_0 0 /* String table id 0 (in-CTF). */
#define CTF_STRTAB_1 1 /* String table id 1 (ELF strtab). */
/* Values for CTF_TYPE_KIND(). If the kind has an associated data list,
CTF_INFO_VLEN() will extract the number of elements in the list, and
the type of each element is shown in the comments below. */
libctf, include: support an alternative encoding for nonrepresentable types Before now, types that could not be encoded in CTF were represented as references to type ID 0, which does not itself appear in the dictionary. This choice is annoying in several ways, principally that it forces generators and consumers of CTF to grow special cases for types that are referenced in valid dicts but don't appear. Allow an alternative representation (which will become the only representation in format v4) whereby nonrepresentable types are encoded as actual types with kind CTF_K_UNKNOWN (an already-existing kind theoretically but not in practice used for padding, with value 0). This is backward-compatible, because CTF_K_UNKNOWN was not used anywhere before now: it was used in old-format function symtypetabs, but these were never emitted by any compiler and the code to handle them in libctf likely never worked and was removed last year, in favour of new-format symtypetabs that contain only type IDs, not type kinds. In order to link this type, we need an API addition to let us add types of unknown kind to the dict: we let them optionally have names so that GCC can emit many different unknown types and those types with identical names will be deduplicated together. There are also small tweaks to the deduplicator to actually dedup such types, to let opening of dicts with unknown types with names work, to return the ECTF_NONREPRESENTABLE error on resolution of such types (like ID 0), and to print their names as something useful but not a valid C identifier, mostly for the sake of the dumper. Tests added in the next commit. include/ChangeLog 2021-05-06 Nick Alcock <nick.alcock@oracle.com> * ctf.h (CTF_K_UNKNOWN): Document that it can be used for nonrepresentable types, not just padding. * ctf-api.h (ctf_add_unknown): New. libctf/ChangeLog 2021-05-06 Nick Alcock <nick.alcock@oracle.com> * ctf-open.c (init_types): Unknown types may have names. * ctf-types.c (ctf_type_resolve): CTF_K_UNKNOWN is as non-representable as type ID 0. (ctf_type_aname): Print unknown types. * ctf-dedup.c (ctf_dedup_hash_type): Do not early-exit for CTF_K_UNKNOWN types: they have real hash values now. (ctf_dedup_rwalk_one_output_mapping): Treat CTF_K_UNKNOWN types like other types with no referents: call the callback and do not skip them. (ctf_dedup_emit_type): Emit via... * ctf-create.c (ctf_add_unknown): ... this new function. * libctf.ver (LIBCTF_1.2): Add it.
2021-05-06 16:30:58 +08:00
#define CTF_K_UNKNOWN 0 /* Unknown type (used for padding and
unrepresentable types). */
#define CTF_K_INTEGER 1 /* Variant data is CTF_INT_DATA (see below). */
#define CTF_K_FLOAT 2 /* Variant data is CTF_FP_DATA (see below). */
#define CTF_K_POINTER 3 /* ctt_type is referenced type. */
#define CTF_K_ARRAY 4 /* Variant data is single ctf_array_t. */
#define CTF_K_FUNCTION 5 /* ctt_type is return type, variant data is
list of argument types (unsigned short's for v1,
uint32_t's for v2). */
#define CTF_K_STRUCT 6 /* Variant data is list of ctf_member_t's. */
#define CTF_K_UNION 7 /* Variant data is list of ctf_member_t's. */
#define CTF_K_ENUM 8 /* Variant data is list of ctf_enum_t's. */
#define CTF_K_FORWARD 9 /* No additional data; ctt_name is tag. */
#define CTF_K_TYPEDEF 10 /* ctt_type is referenced type. */
#define CTF_K_VOLATILE 11 /* ctt_type is base type. */
#define CTF_K_CONST 12 /* ctt_type is base type. */
#define CTF_K_RESTRICT 13 /* ctt_type is base type. */
#define CTF_K_SLICE 14 /* Variant data is a ctf_slice_t. */
#define CTF_K_MAX 63 /* Maximum possible (V2) CTF_K_* value. */
/* Values for ctt_type when kind is CTF_K_INTEGER. The flags, offset in bits,
and size in bits are encoded as a single word using the following macros.
(However, you can also encode the offset and bitness in a slice.) */
#define CTF_INT_ENCODING(data) (((data) & 0xff000000) >> 24)
#define CTF_INT_OFFSET(data) (((data) & 0x00ff0000) >> 16)
#define CTF_INT_BITS(data) (((data) & 0x0000ffff))
#define CTF_INT_DATA(encoding, offset, bits) \
(((encoding) << 24) | ((offset) << 16) | (bits))
#define CTF_INT_SIGNED 0x01 /* Integer is signed (otherwise unsigned). */
#define CTF_INT_CHAR 0x02 /* Character display format. */
#define CTF_INT_BOOL 0x04 /* Boolean display format. */
#define CTF_INT_VARARGS 0x08 /* Varargs display format. */
/* Use CTF_CHAR to produce a char that agrees with the system's native
char signedness. */
#if CHAR_MIN == 0
# define CTF_CHAR (CTF_INT_CHAR)
#else
# define CTF_CHAR (CTF_INT_CHAR | CTF_INT_SIGNED)
#endif
/* Values for ctt_type when kind is CTF_K_FLOAT. The encoding, offset in bits,
and size in bits are encoded as a single word using the following macros.
(However, you can also encode the offset and bitness in a slice.) */
#define CTF_FP_ENCODING(data) (((data) & 0xff000000) >> 24)
#define CTF_FP_OFFSET(data) (((data) & 0x00ff0000) >> 16)
#define CTF_FP_BITS(data) (((data) & 0x0000ffff))
#define CTF_FP_DATA(encoding, offset, bits) \
(((encoding) << 24) | ((offset) << 16) | (bits))
/* Variant data when kind is CTF_K_FLOAT is an encoding in the top eight bits. */
#define CTF_FP_ENCODING(data) (((data) & 0xff000000) >> 24)
#define CTF_FP_SINGLE 1 /* IEEE 32-bit float encoding. */
#define CTF_FP_DOUBLE 2 /* IEEE 64-bit float encoding. */
#define CTF_FP_CPLX 3 /* Complex encoding. */
#define CTF_FP_DCPLX 4 /* Double complex encoding. */
#define CTF_FP_LDCPLX 5 /* Long double complex encoding. */
#define CTF_FP_LDOUBLE 6 /* Long double encoding. */
#define CTF_FP_INTRVL 7 /* Interval (2x32-bit) encoding. */
#define CTF_FP_DINTRVL 8 /* Double interval (2x64-bit) encoding. */
#define CTF_FP_LDINTRVL 9 /* Long double interval (2x128-bit) encoding. */
#define CTF_FP_IMAGRY 10 /* Imaginary (32-bit) encoding. */
#define CTF_FP_DIMAGRY 11 /* Long imaginary (64-bit) encoding. */
#define CTF_FP_LDIMAGRY 12 /* Long double imaginary (128-bit) encoding. */
#define CTF_FP_MAX 12 /* Maximum possible CTF_FP_* value */
/* A slice increases the offset and reduces the bitness of the referenced
ctt_type, which must be a type which has an encoding (fp, int, or enum). We
also store the referenced type in here, because it is easier to keep the
ctt_size correct for the slice than to shuffle the size into here and keep
libctf: endianness fixes Testing of the first code to generate CTF_K_SLICEs on big-endian revealed a bunch of new problems in this area. Most importantly, the trick we did earlier to avoid wasting two bytes on padding in the ctf_slice_t is best avoided: because it leads to the whole file after that point no longer being naturally aligned, all multibyte accesses from then on must use memmove() to avoid unaligned access on platforms where that is fatal. In future, this is planned, but for now we are still doing direct access in many places, so we must revert to making ctf_slice_t properly aligned for storage in an array. Rather than wasting bytes on padding, we boost the size of cts_offset and cts_bits. This is still a waste of space (we cannot have offsets or bits in bitfields > 256) but it cannot be avoided for now, and slices are not so common that this will be a serious problem. A possibly-worse endianness problem fixed at the same time involves a codepath used only for foreign-endian, uncompressed CTF files, where we were not copying the actual CTF data into the buffer, leading to libctf reading only zeroes (or, possibly, uninitialized garbage). Finally, when we read in a CTF file, we copy the header and work from the copy. We were flipping the endianness of the header copy, and of the body of the file buffer, but not of the header in the file buffer itself: so if we write the file back out again we end up with an unreadable frankenfile with header and body of different endiannesses. Fix by flipping both copies of the header. include/ * ctf.h (ctf_slice_t): Make cts_offset and cts_bits unsigned short, so following structures are properly aligned. libctf/ * ctf-open.c (get_vbytes_common): Return the new slice size. (ctf_bufopen): Flip the endianness of the CTF-section header copy. Remember to copy in the CTF data when opening an uncompressed foreign-endian CTF file. Prune useless variable manipulation.
2019-06-19 19:34:56 +08:00
the ctt_type where it is for other types.
In a future version, where we loosen requirements on alignment in the CTF
file, the cts_offset and cts_bits will be chars: but for now they must be
shorts or everything after a slice will become unaligned. */
typedef struct ctf_slice
{
uint32_t cts_type;
libctf: endianness fixes Testing of the first code to generate CTF_K_SLICEs on big-endian revealed a bunch of new problems in this area. Most importantly, the trick we did earlier to avoid wasting two bytes on padding in the ctf_slice_t is best avoided: because it leads to the whole file after that point no longer being naturally aligned, all multibyte accesses from then on must use memmove() to avoid unaligned access on platforms where that is fatal. In future, this is planned, but for now we are still doing direct access in many places, so we must revert to making ctf_slice_t properly aligned for storage in an array. Rather than wasting bytes on padding, we boost the size of cts_offset and cts_bits. This is still a waste of space (we cannot have offsets or bits in bitfields > 256) but it cannot be avoided for now, and slices are not so common that this will be a serious problem. A possibly-worse endianness problem fixed at the same time involves a codepath used only for foreign-endian, uncompressed CTF files, where we were not copying the actual CTF data into the buffer, leading to libctf reading only zeroes (or, possibly, uninitialized garbage). Finally, when we read in a CTF file, we copy the header and work from the copy. We were flipping the endianness of the header copy, and of the body of the file buffer, but not of the header in the file buffer itself: so if we write the file back out again we end up with an unreadable frankenfile with header and body of different endiannesses. Fix by flipping both copies of the header. include/ * ctf.h (ctf_slice_t): Make cts_offset and cts_bits unsigned short, so following structures are properly aligned. libctf/ * ctf-open.c (get_vbytes_common): Return the new slice size. (ctf_bufopen): Flip the endianness of the CTF-section header copy. Remember to copy in the CTF data when opening an uncompressed foreign-endian CTF file. Prune useless variable manipulation.
2019-06-19 19:34:56 +08:00
unsigned short cts_offset;
unsigned short cts_bits;
} ctf_slice_t;
typedef struct ctf_array_v1
{
unsigned short cta_contents; /* Reference to type of array contents. */
unsigned short cta_index; /* Reference to type of array index. */
uint32_t cta_nelems; /* Number of elements. */
} ctf_array_v1_t;
typedef struct ctf_array
{
uint32_t cta_contents; /* Reference to type of array contents. */
uint32_t cta_index; /* Reference to type of array index. */
uint32_t cta_nelems; /* Number of elements. */
} ctf_array_t;
/* Most structure members have bit offsets that can be expressed using a short.
Some don't. ctf_member_t is used for structs which cannot contain any of
these large offsets, whereas ctf_lmember_t is used in the latter case. If
any member of a given struct has an offset that cannot be expressed using a
uint32_t, all members will be stored as type ctf_lmember_t. This is expected
to be very rare (but nonetheless possible). */
#define CTF_LSTRUCT_THRESH 536870912
/* In v1, the same is true, except that lmembers are used for structs >= 8192
bytes in size. (The ordering of members in the ctf_member_* structures is
different to improve padding.) */
#define CTF_LSTRUCT_THRESH_V1 8192
typedef struct ctf_member_v1
{
uint32_t ctm_name; /* Reference to name in string table. */
unsigned short ctm_type; /* Reference to type of member. */
unsigned short ctm_offset; /* Offset of this member in bits. */
} ctf_member_v1_t;
typedef struct ctf_lmember_v1
{
uint32_t ctlm_name; /* Reference to name in string table. */
unsigned short ctlm_type; /* Reference to type of member. */
unsigned short ctlm_pad; /* Padding. */
uint32_t ctlm_offsethi; /* High 32 bits of member offset in bits. */
uint32_t ctlm_offsetlo; /* Low 32 bits of member offset in bits. */
} ctf_lmember_v1_t;
typedef struct ctf_member_v2
{
uint32_t ctm_name; /* Reference to name in string table. */
uint32_t ctm_offset; /* Offset of this member in bits. */
uint32_t ctm_type; /* Reference to type of member. */
} ctf_member_t;
typedef struct ctf_lmember_v2
{
uint32_t ctlm_name; /* Reference to name in string table. */
uint32_t ctlm_offsethi; /* High 32 bits of member offset in bits. */
uint32_t ctlm_type; /* Reference to type of member. */
uint32_t ctlm_offsetlo; /* Low 32 bits of member offset in bits. */
} ctf_lmember_t;
#define CTF_LMEM_OFFSET(ctlmp) \
(((uint64_t)(ctlmp)->ctlm_offsethi) << 32 | (ctlmp)->ctlm_offsetlo)
#define CTF_OFFSET_TO_LMEMHI(offset) ((uint32_t)((uint64_t)(offset) >> 32))
#define CTF_OFFSET_TO_LMEMLO(offset) ((uint32_t)(offset))
typedef struct ctf_enum
{
uint32_t cte_name; /* Reference to name in string table. */
int32_t cte_value; /* Value associated with this name. */
} ctf_enum_t;
libctf, include, binutils, gdb, ld: rename ctf_file_t to ctf_dict_t The naming of the ctf_file_t type in libctf is a historical curiosity. Back in the Solaris days, CTF dictionaries were originally generated as a separate file and then (sometimes) merged into objects: hence the datatype was named ctf_file_t, and known as a "CTF file". Nowadays, raw CTF is essentially never written to a file on its own, and the datatype changed name to a "CTF dictionary" years ago. So the term "CTF file" refers to something that is never a file! This is at best confusing. The type has also historically been known as a 'CTF container", which is even more confusing now that we have CTF archives which are *also* a sort of container (they contain CTF dictionaries), but which are never referred to as containers in the source code. So fix this by completing the renaming, renaming ctf_file_t to ctf_dict_t throughout, and renaming those few functions that refer to CTF files by name (keeping compatibility aliases) to refer to dicts instead. Old users who still refer to ctf_file_t will see (harmless) pointer-compatibility warnings at compile time, but the ABI is unchanged (since C doesn't mangle names, and ctf_file_t was always an opaque type) and things will still compile fine as long as -Werror is not specified. All references to CTF containers and CTF files in the source code are fixed to refer to CTF dicts instead. Further (smaller) renamings of annoyingly-named functions to come, as part of the process of souping up queries across whole archives at once (needed for the function info and data object sections). binutils/ChangeLog 2020-11-20 Nick Alcock <nick.alcock@oracle.com> * objdump.c (dump_ctf_errs): Rename ctf_file_t to ctf_dict_t. (dump_ctf_archive_member): Likewise. (dump_ctf): Likewise. Use ctf_dict_close, not ctf_file_close. * readelf.c (dump_ctf_errs): Rename ctf_file_t to ctf_dict_t. (dump_ctf_archive_member): Likewise. (dump_section_as_ctf): Likewise. Use ctf_dict_close, not ctf_file_close. gdb/ChangeLog 2020-11-20 Nick Alcock <nick.alcock@oracle.com> * ctfread.c: Change uses of ctf_file_t to ctf_dict_t. (ctf_fp_info::~ctf_fp_info): Call ctf_dict_close, not ctf_file_close. include/ChangeLog 2020-11-20 Nick Alcock <nick.alcock@oracle.com> * ctf-api.h (ctf_file_t): Rename to... (ctf_dict_t): ... this. Keep ctf_file_t around for compatibility. (struct ctf_file): Likewise rename to... (struct ctf_dict): ... this. (ctf_file_close): Rename to... (ctf_dict_close): ... this, keeping compatibility function. (ctf_parent_file): Rename to... (ctf_parent_dict): ... this, keeping compatibility function. All callers adjusted. * ctf.h: Rename references to ctf_file_t to ctf_dict_t. (struct ctf_archive) <ctfa_nfiles>: Rename to... <ctfa_ndicts>: ... this. ld/ChangeLog 2020-11-20 Nick Alcock <nick.alcock@oracle.com> * ldlang.c (ctf_output): This is a ctf_dict_t now. (lang_ctf_errs_warnings): Rename ctf_file_t to ctf_dict_t. (ldlang_open_ctf): Adjust comment. (lang_merge_ctf): Use ctf_dict_close, not ctf_file_close. * ldelfgen.h (ldelf_examine_strtab_for_ctf): Rename ctf_file_t to ctf_dict_t. Change opaque declaration accordingly. * ldelfgen.c (ldelf_examine_strtab_for_ctf): Adjust. * ldemul.h (examine_strtab_for_ctf): Likewise. (ldemul_examine_strtab_for_ctf): Likewise. * ldeuml.c (ldemul_examine_strtab_for_ctf): Likewise. libctf/ChangeLog 2020-11-20 Nick Alcock <nick.alcock@oracle.com> * ctf-impl.h: Rename ctf_file_t to ctf_dict_t: all declarations adjusted. (ctf_fileops): Rename to... (ctf_dictops): ... this. (ctf_dedup_t) <cd_id_to_file_t>: Rename to... <cd_id_to_dict_t>: ... this. (ctf_file_t): Fix outdated comment. <ctf_fileops>: Rename to... <ctf_dictops>: ... this. (struct ctf_archive_internal) <ctfi_file>: Rename to... <ctfi_dict>: ... this. * ctf-archive.c: Rename ctf_file_t to ctf_dict_t. Rename ctf_archive.ctfa_nfiles to ctfa_ndicts. Rename ctf_file_close to ctf_dict_close. All users adjusted. * ctf-create.c: Likewise. Refer to CTF dicts, not CTF containers. (ctf_bundle_t) <ctb_file>: Rename to... <ctb_dict): ... this. * ctf-decl.c: Rename ctf_file_t to ctf_dict_t. * ctf-dedup.c: Likewise. Rename ctf_file_close to ctf_dict_close. Refer to CTF dicts, not CTF containers. * ctf-dump.c: Likewise. * ctf-error.c: Likewise. * ctf-hash.c: Likewise. * ctf-inlines.h: Likewise. * ctf-labels.c: Likewise. * ctf-link.c: Likewise. * ctf-lookup.c: Likewise. * ctf-open-bfd.c: Likewise. * ctf-string.c: Likewise. * ctf-subr.c: Likewise. * ctf-types.c: Likewise. * ctf-util.c: Likewise. * ctf-open.c: Likewise. (ctf_file_close): Rename to... (ctf_dict_close): ...this. (ctf_file_close): New trivial wrapper around ctf_dict_close, for compatibility. (ctf_parent_file): Rename to... (ctf_parent_dict): ... this. (ctf_parent_file): New trivial wrapper around ctf_parent_dict, for compatibility. * libctf.ver: Add ctf_dict_close and ctf_parent_dict.
2020-11-20 21:34:04 +08:00
/* The ctf_archive is a collection of ctf_dict_t's stored together. The format
libctf: mmappable archives If you need to store a large number of CTF containers somewhere, this provides a dedicated facility for doing so: an mmappable archive format like a very simple tar or ar without all the system-dependent format horrors or need for heavy file copying, with built-in compression of files above a particular size threshold. libctf automatically mmap()s uncompressed elements of these archives, or uncompresses them, as needed. (If the platform does not support mmap(), copying into dynamically-allocated buffers is used.) Archive iteration operations are partitioned into raw and non-raw forms. Raw operations pass thhe raw archive contents to the callback: non-raw forms open each member with ctf_bufopen() and pass the resulting ctf_file_t to the iterator instead. This lets you manipulate the raw data in the archive, or the contents interpreted as a CTF file, as needed. It is not yet known whether we will store CTF archives in a linked ELF object in one of these (akin to debugdata) or whether they'll get one section per TU plus one parent container for types shared between them. (In the case of ELF objects with very large numbers of TUs, an archive of all of them would seem preferable, so we might just use an archive, and add lzma support so you can assume that .gnu_debugdata and .ctf are compressed using the same algorithm if both are present.) To make usage easier, the ctf_archive_t is not the on-disk representation but an abstraction over both ctf_file_t's and archives of many ctf_file_t's: users see both CTF archives and raw CTF files as ctf_archive_t's upon opening, the only difference being that a raw CTF file has only a single "archive member", named ".ctf" (the default if a null pointer is passed in as the name). The next commit will make use of this facility, in addition to providing the public interface to actually open archives. (In the future, it should be possible to have all CTF sections in an ELF file appear as an "archive" in the same fashion.) This machinery is also used to allow library-internal creators of ctf_archive_t's (such as the next commit) to stash away an ELF string and symbol table, so that all opens of members in a given archive will use them. This lets CTF archives exploit the ELF string and symbol table just like raw CTF files can. (All this leads to somewhat confusing type naming. The ctf_archive_t is a typedef for the opaque internal type, struct ctf_archive_internal: the non-internal "struct ctf_archive" is the on-disk structure meant for other libraries manipulating CTF files. It is probably clearest to use the struct name for struct ctf_archive_internal inside the program, and the typedef names outside.) libctf/ * ctf-archive.c: New. * ctf-impl.h (ctf_archive_internal): New type. (ctf_arc_open_internal): New declaration. (ctf_arc_bufopen): Likewise. (ctf_arc_close_internal): Likewise. include/ * ctf.h (CTFA_MAGIC): New. (struct ctf_archive): New. (struct ctf_archive_modent): Likewise. * ctf-api.h (ctf_archive_member_f): New. (ctf_archive_raw_member_f): Likewise. (ctf_arc_write): Likewise. (ctf_arc_close): Likewise. (ctf_arc_open_by_name): Likewise. (ctf_archive_iter): Likewise. (ctf_archive_raw_iter): Likewise. (ctf_get_arc): Likewise.
2019-04-24 18:30:17 +08:00
is suitable for mmap()ing: this control structure merely describes the
mmap()ed archive (and overlaps the first few bytes of it), hence the
greater care taken with integral types. All CTF files in an archive
must have the same data model. (This is not validated.)
All integers in this structure are stored in little-endian byte order.
The code relies on the fact that everything in this header is a uint64_t
and thus the header needs no padding (in particular, that no padding is
needed between ctfa_ctfs and the unnamed ctfa_archive_modent array
that follows it).
This is *not* the same as the data structure returned by the ctf_arc_*()
functions: this is the low-level on-disk representation. */
#define CTFA_MAGIC 0x8b47f2a4d7623eeb /* Random. */
struct ctf_archive
{
/* Magic number. (In loaded files, overwritten with the file size
so ctf_arc_close() knows how much to munmap()). */
uint64_t ctfa_magic;
/* CTF data model. */
uint64_t ctfa_model;
libctf, include, binutils, gdb, ld: rename ctf_file_t to ctf_dict_t The naming of the ctf_file_t type in libctf is a historical curiosity. Back in the Solaris days, CTF dictionaries were originally generated as a separate file and then (sometimes) merged into objects: hence the datatype was named ctf_file_t, and known as a "CTF file". Nowadays, raw CTF is essentially never written to a file on its own, and the datatype changed name to a "CTF dictionary" years ago. So the term "CTF file" refers to something that is never a file! This is at best confusing. The type has also historically been known as a 'CTF container", which is even more confusing now that we have CTF archives which are *also* a sort of container (they contain CTF dictionaries), but which are never referred to as containers in the source code. So fix this by completing the renaming, renaming ctf_file_t to ctf_dict_t throughout, and renaming those few functions that refer to CTF files by name (keeping compatibility aliases) to refer to dicts instead. Old users who still refer to ctf_file_t will see (harmless) pointer-compatibility warnings at compile time, but the ABI is unchanged (since C doesn't mangle names, and ctf_file_t was always an opaque type) and things will still compile fine as long as -Werror is not specified. All references to CTF containers and CTF files in the source code are fixed to refer to CTF dicts instead. Further (smaller) renamings of annoyingly-named functions to come, as part of the process of souping up queries across whole archives at once (needed for the function info and data object sections). binutils/ChangeLog 2020-11-20 Nick Alcock <nick.alcock@oracle.com> * objdump.c (dump_ctf_errs): Rename ctf_file_t to ctf_dict_t. (dump_ctf_archive_member): Likewise. (dump_ctf): Likewise. Use ctf_dict_close, not ctf_file_close. * readelf.c (dump_ctf_errs): Rename ctf_file_t to ctf_dict_t. (dump_ctf_archive_member): Likewise. (dump_section_as_ctf): Likewise. Use ctf_dict_close, not ctf_file_close. gdb/ChangeLog 2020-11-20 Nick Alcock <nick.alcock@oracle.com> * ctfread.c: Change uses of ctf_file_t to ctf_dict_t. (ctf_fp_info::~ctf_fp_info): Call ctf_dict_close, not ctf_file_close. include/ChangeLog 2020-11-20 Nick Alcock <nick.alcock@oracle.com> * ctf-api.h (ctf_file_t): Rename to... (ctf_dict_t): ... this. Keep ctf_file_t around for compatibility. (struct ctf_file): Likewise rename to... (struct ctf_dict): ... this. (ctf_file_close): Rename to... (ctf_dict_close): ... this, keeping compatibility function. (ctf_parent_file): Rename to... (ctf_parent_dict): ... this, keeping compatibility function. All callers adjusted. * ctf.h: Rename references to ctf_file_t to ctf_dict_t. (struct ctf_archive) <ctfa_nfiles>: Rename to... <ctfa_ndicts>: ... this. ld/ChangeLog 2020-11-20 Nick Alcock <nick.alcock@oracle.com> * ldlang.c (ctf_output): This is a ctf_dict_t now. (lang_ctf_errs_warnings): Rename ctf_file_t to ctf_dict_t. (ldlang_open_ctf): Adjust comment. (lang_merge_ctf): Use ctf_dict_close, not ctf_file_close. * ldelfgen.h (ldelf_examine_strtab_for_ctf): Rename ctf_file_t to ctf_dict_t. Change opaque declaration accordingly. * ldelfgen.c (ldelf_examine_strtab_for_ctf): Adjust. * ldemul.h (examine_strtab_for_ctf): Likewise. (ldemul_examine_strtab_for_ctf): Likewise. * ldeuml.c (ldemul_examine_strtab_for_ctf): Likewise. libctf/ChangeLog 2020-11-20 Nick Alcock <nick.alcock@oracle.com> * ctf-impl.h: Rename ctf_file_t to ctf_dict_t: all declarations adjusted. (ctf_fileops): Rename to... (ctf_dictops): ... this. (ctf_dedup_t) <cd_id_to_file_t>: Rename to... <cd_id_to_dict_t>: ... this. (ctf_file_t): Fix outdated comment. <ctf_fileops>: Rename to... <ctf_dictops>: ... this. (struct ctf_archive_internal) <ctfi_file>: Rename to... <ctfi_dict>: ... this. * ctf-archive.c: Rename ctf_file_t to ctf_dict_t. Rename ctf_archive.ctfa_nfiles to ctfa_ndicts. Rename ctf_file_close to ctf_dict_close. All users adjusted. * ctf-create.c: Likewise. Refer to CTF dicts, not CTF containers. (ctf_bundle_t) <ctb_file>: Rename to... <ctb_dict): ... this. * ctf-decl.c: Rename ctf_file_t to ctf_dict_t. * ctf-dedup.c: Likewise. Rename ctf_file_close to ctf_dict_close. Refer to CTF dicts, not CTF containers. * ctf-dump.c: Likewise. * ctf-error.c: Likewise. * ctf-hash.c: Likewise. * ctf-inlines.h: Likewise. * ctf-labels.c: Likewise. * ctf-link.c: Likewise. * ctf-lookup.c: Likewise. * ctf-open-bfd.c: Likewise. * ctf-string.c: Likewise. * ctf-subr.c: Likewise. * ctf-types.c: Likewise. * ctf-util.c: Likewise. * ctf-open.c: Likewise. (ctf_file_close): Rename to... (ctf_dict_close): ...this. (ctf_file_close): New trivial wrapper around ctf_dict_close, for compatibility. (ctf_parent_file): Rename to... (ctf_parent_dict): ... this. (ctf_parent_file): New trivial wrapper around ctf_parent_dict, for compatibility. * libctf.ver: Add ctf_dict_close and ctf_parent_dict.
2020-11-20 21:34:04 +08:00
/* Number of CTF dicts in the archive. */
uint64_t ctfa_ndicts;
libctf: mmappable archives If you need to store a large number of CTF containers somewhere, this provides a dedicated facility for doing so: an mmappable archive format like a very simple tar or ar without all the system-dependent format horrors or need for heavy file copying, with built-in compression of files above a particular size threshold. libctf automatically mmap()s uncompressed elements of these archives, or uncompresses them, as needed. (If the platform does not support mmap(), copying into dynamically-allocated buffers is used.) Archive iteration operations are partitioned into raw and non-raw forms. Raw operations pass thhe raw archive contents to the callback: non-raw forms open each member with ctf_bufopen() and pass the resulting ctf_file_t to the iterator instead. This lets you manipulate the raw data in the archive, or the contents interpreted as a CTF file, as needed. It is not yet known whether we will store CTF archives in a linked ELF object in one of these (akin to debugdata) or whether they'll get one section per TU plus one parent container for types shared between them. (In the case of ELF objects with very large numbers of TUs, an archive of all of them would seem preferable, so we might just use an archive, and add lzma support so you can assume that .gnu_debugdata and .ctf are compressed using the same algorithm if both are present.) To make usage easier, the ctf_archive_t is not the on-disk representation but an abstraction over both ctf_file_t's and archives of many ctf_file_t's: users see both CTF archives and raw CTF files as ctf_archive_t's upon opening, the only difference being that a raw CTF file has only a single "archive member", named ".ctf" (the default if a null pointer is passed in as the name). The next commit will make use of this facility, in addition to providing the public interface to actually open archives. (In the future, it should be possible to have all CTF sections in an ELF file appear as an "archive" in the same fashion.) This machinery is also used to allow library-internal creators of ctf_archive_t's (such as the next commit) to stash away an ELF string and symbol table, so that all opens of members in a given archive will use them. This lets CTF archives exploit the ELF string and symbol table just like raw CTF files can. (All this leads to somewhat confusing type naming. The ctf_archive_t is a typedef for the opaque internal type, struct ctf_archive_internal: the non-internal "struct ctf_archive" is the on-disk structure meant for other libraries manipulating CTF files. It is probably clearest to use the struct name for struct ctf_archive_internal inside the program, and the typedef names outside.) libctf/ * ctf-archive.c: New. * ctf-impl.h (ctf_archive_internal): New type. (ctf_arc_open_internal): New declaration. (ctf_arc_bufopen): Likewise. (ctf_arc_close_internal): Likewise. include/ * ctf.h (CTFA_MAGIC): New. (struct ctf_archive): New. (struct ctf_archive_modent): Likewise. * ctf-api.h (ctf_archive_member_f): New. (ctf_archive_raw_member_f): Likewise. (ctf_arc_write): Likewise. (ctf_arc_close): Likewise. (ctf_arc_open_by_name): Likewise. (ctf_archive_iter): Likewise. (ctf_archive_raw_iter): Likewise. (ctf_get_arc): Likewise.
2019-04-24 18:30:17 +08:00
/* Offset of the name table. */
uint64_t ctfa_names;
/* Offset of the CTF table. Each element starts with a size (a uint64_t
libctf, include, binutils, gdb, ld: rename ctf_file_t to ctf_dict_t The naming of the ctf_file_t type in libctf is a historical curiosity. Back in the Solaris days, CTF dictionaries were originally generated as a separate file and then (sometimes) merged into objects: hence the datatype was named ctf_file_t, and known as a "CTF file". Nowadays, raw CTF is essentially never written to a file on its own, and the datatype changed name to a "CTF dictionary" years ago. So the term "CTF file" refers to something that is never a file! This is at best confusing. The type has also historically been known as a 'CTF container", which is even more confusing now that we have CTF archives which are *also* a sort of container (they contain CTF dictionaries), but which are never referred to as containers in the source code. So fix this by completing the renaming, renaming ctf_file_t to ctf_dict_t throughout, and renaming those few functions that refer to CTF files by name (keeping compatibility aliases) to refer to dicts instead. Old users who still refer to ctf_file_t will see (harmless) pointer-compatibility warnings at compile time, but the ABI is unchanged (since C doesn't mangle names, and ctf_file_t was always an opaque type) and things will still compile fine as long as -Werror is not specified. All references to CTF containers and CTF files in the source code are fixed to refer to CTF dicts instead. Further (smaller) renamings of annoyingly-named functions to come, as part of the process of souping up queries across whole archives at once (needed for the function info and data object sections). binutils/ChangeLog 2020-11-20 Nick Alcock <nick.alcock@oracle.com> * objdump.c (dump_ctf_errs): Rename ctf_file_t to ctf_dict_t. (dump_ctf_archive_member): Likewise. (dump_ctf): Likewise. Use ctf_dict_close, not ctf_file_close. * readelf.c (dump_ctf_errs): Rename ctf_file_t to ctf_dict_t. (dump_ctf_archive_member): Likewise. (dump_section_as_ctf): Likewise. Use ctf_dict_close, not ctf_file_close. gdb/ChangeLog 2020-11-20 Nick Alcock <nick.alcock@oracle.com> * ctfread.c: Change uses of ctf_file_t to ctf_dict_t. (ctf_fp_info::~ctf_fp_info): Call ctf_dict_close, not ctf_file_close. include/ChangeLog 2020-11-20 Nick Alcock <nick.alcock@oracle.com> * ctf-api.h (ctf_file_t): Rename to... (ctf_dict_t): ... this. Keep ctf_file_t around for compatibility. (struct ctf_file): Likewise rename to... (struct ctf_dict): ... this. (ctf_file_close): Rename to... (ctf_dict_close): ... this, keeping compatibility function. (ctf_parent_file): Rename to... (ctf_parent_dict): ... this, keeping compatibility function. All callers adjusted. * ctf.h: Rename references to ctf_file_t to ctf_dict_t. (struct ctf_archive) <ctfa_nfiles>: Rename to... <ctfa_ndicts>: ... this. ld/ChangeLog 2020-11-20 Nick Alcock <nick.alcock@oracle.com> * ldlang.c (ctf_output): This is a ctf_dict_t now. (lang_ctf_errs_warnings): Rename ctf_file_t to ctf_dict_t. (ldlang_open_ctf): Adjust comment. (lang_merge_ctf): Use ctf_dict_close, not ctf_file_close. * ldelfgen.h (ldelf_examine_strtab_for_ctf): Rename ctf_file_t to ctf_dict_t. Change opaque declaration accordingly. * ldelfgen.c (ldelf_examine_strtab_for_ctf): Adjust. * ldemul.h (examine_strtab_for_ctf): Likewise. (ldemul_examine_strtab_for_ctf): Likewise. * ldeuml.c (ldemul_examine_strtab_for_ctf): Likewise. libctf/ChangeLog 2020-11-20 Nick Alcock <nick.alcock@oracle.com> * ctf-impl.h: Rename ctf_file_t to ctf_dict_t: all declarations adjusted. (ctf_fileops): Rename to... (ctf_dictops): ... this. (ctf_dedup_t) <cd_id_to_file_t>: Rename to... <cd_id_to_dict_t>: ... this. (ctf_file_t): Fix outdated comment. <ctf_fileops>: Rename to... <ctf_dictops>: ... this. (struct ctf_archive_internal) <ctfi_file>: Rename to... <ctfi_dict>: ... this. * ctf-archive.c: Rename ctf_file_t to ctf_dict_t. Rename ctf_archive.ctfa_nfiles to ctfa_ndicts. Rename ctf_file_close to ctf_dict_close. All users adjusted. * ctf-create.c: Likewise. Refer to CTF dicts, not CTF containers. (ctf_bundle_t) <ctb_file>: Rename to... <ctb_dict): ... this. * ctf-decl.c: Rename ctf_file_t to ctf_dict_t. * ctf-dedup.c: Likewise. Rename ctf_file_close to ctf_dict_close. Refer to CTF dicts, not CTF containers. * ctf-dump.c: Likewise. * ctf-error.c: Likewise. * ctf-hash.c: Likewise. * ctf-inlines.h: Likewise. * ctf-labels.c: Likewise. * ctf-link.c: Likewise. * ctf-lookup.c: Likewise. * ctf-open-bfd.c: Likewise. * ctf-string.c: Likewise. * ctf-subr.c: Likewise. * ctf-types.c: Likewise. * ctf-util.c: Likewise. * ctf-open.c: Likewise. (ctf_file_close): Rename to... (ctf_dict_close): ...this. (ctf_file_close): New trivial wrapper around ctf_dict_close, for compatibility. (ctf_parent_file): Rename to... (ctf_parent_dict): ... this. (ctf_parent_file): New trivial wrapper around ctf_parent_dict, for compatibility. * libctf.ver: Add ctf_dict_close and ctf_parent_dict.
2020-11-20 21:34:04 +08:00
in network byte order) then a ctf_dict_t of that size. */
libctf: mmappable archives If you need to store a large number of CTF containers somewhere, this provides a dedicated facility for doing so: an mmappable archive format like a very simple tar or ar without all the system-dependent format horrors or need for heavy file copying, with built-in compression of files above a particular size threshold. libctf automatically mmap()s uncompressed elements of these archives, or uncompresses them, as needed. (If the platform does not support mmap(), copying into dynamically-allocated buffers is used.) Archive iteration operations are partitioned into raw and non-raw forms. Raw operations pass thhe raw archive contents to the callback: non-raw forms open each member with ctf_bufopen() and pass the resulting ctf_file_t to the iterator instead. This lets you manipulate the raw data in the archive, or the contents interpreted as a CTF file, as needed. It is not yet known whether we will store CTF archives in a linked ELF object in one of these (akin to debugdata) or whether they'll get one section per TU plus one parent container for types shared between them. (In the case of ELF objects with very large numbers of TUs, an archive of all of them would seem preferable, so we might just use an archive, and add lzma support so you can assume that .gnu_debugdata and .ctf are compressed using the same algorithm if both are present.) To make usage easier, the ctf_archive_t is not the on-disk representation but an abstraction over both ctf_file_t's and archives of many ctf_file_t's: users see both CTF archives and raw CTF files as ctf_archive_t's upon opening, the only difference being that a raw CTF file has only a single "archive member", named ".ctf" (the default if a null pointer is passed in as the name). The next commit will make use of this facility, in addition to providing the public interface to actually open archives. (In the future, it should be possible to have all CTF sections in an ELF file appear as an "archive" in the same fashion.) This machinery is also used to allow library-internal creators of ctf_archive_t's (such as the next commit) to stash away an ELF string and symbol table, so that all opens of members in a given archive will use them. This lets CTF archives exploit the ELF string and symbol table just like raw CTF files can. (All this leads to somewhat confusing type naming. The ctf_archive_t is a typedef for the opaque internal type, struct ctf_archive_internal: the non-internal "struct ctf_archive" is the on-disk structure meant for other libraries manipulating CTF files. It is probably clearest to use the struct name for struct ctf_archive_internal inside the program, and the typedef names outside.) libctf/ * ctf-archive.c: New. * ctf-impl.h (ctf_archive_internal): New type. (ctf_arc_open_internal): New declaration. (ctf_arc_bufopen): Likewise. (ctf_arc_close_internal): Likewise. include/ * ctf.h (CTFA_MAGIC): New. (struct ctf_archive): New. (struct ctf_archive_modent): Likewise. * ctf-api.h (ctf_archive_member_f): New. (ctf_archive_raw_member_f): Likewise. (ctf_arc_write): Likewise. (ctf_arc_close): Likewise. (ctf_arc_open_by_name): Likewise. (ctf_archive_iter): Likewise. (ctf_archive_raw_iter): Likewise. (ctf_get_arc): Likewise.
2019-04-24 18:30:17 +08:00
uint64_t ctfa_ctfs;
};
/* An array of ctfa_nnamed of this structure lies at
ctf_archive[ctf_archive->ctfa_modents] and gives the ctfa_ctfs or
libctf, include, binutils, gdb, ld: rename ctf_file_t to ctf_dict_t The naming of the ctf_file_t type in libctf is a historical curiosity. Back in the Solaris days, CTF dictionaries were originally generated as a separate file and then (sometimes) merged into objects: hence the datatype was named ctf_file_t, and known as a "CTF file". Nowadays, raw CTF is essentially never written to a file on its own, and the datatype changed name to a "CTF dictionary" years ago. So the term "CTF file" refers to something that is never a file! This is at best confusing. The type has also historically been known as a 'CTF container", which is even more confusing now that we have CTF archives which are *also* a sort of container (they contain CTF dictionaries), but which are never referred to as containers in the source code. So fix this by completing the renaming, renaming ctf_file_t to ctf_dict_t throughout, and renaming those few functions that refer to CTF files by name (keeping compatibility aliases) to refer to dicts instead. Old users who still refer to ctf_file_t will see (harmless) pointer-compatibility warnings at compile time, but the ABI is unchanged (since C doesn't mangle names, and ctf_file_t was always an opaque type) and things will still compile fine as long as -Werror is not specified. All references to CTF containers and CTF files in the source code are fixed to refer to CTF dicts instead. Further (smaller) renamings of annoyingly-named functions to come, as part of the process of souping up queries across whole archives at once (needed for the function info and data object sections). binutils/ChangeLog 2020-11-20 Nick Alcock <nick.alcock@oracle.com> * objdump.c (dump_ctf_errs): Rename ctf_file_t to ctf_dict_t. (dump_ctf_archive_member): Likewise. (dump_ctf): Likewise. Use ctf_dict_close, not ctf_file_close. * readelf.c (dump_ctf_errs): Rename ctf_file_t to ctf_dict_t. (dump_ctf_archive_member): Likewise. (dump_section_as_ctf): Likewise. Use ctf_dict_close, not ctf_file_close. gdb/ChangeLog 2020-11-20 Nick Alcock <nick.alcock@oracle.com> * ctfread.c: Change uses of ctf_file_t to ctf_dict_t. (ctf_fp_info::~ctf_fp_info): Call ctf_dict_close, not ctf_file_close. include/ChangeLog 2020-11-20 Nick Alcock <nick.alcock@oracle.com> * ctf-api.h (ctf_file_t): Rename to... (ctf_dict_t): ... this. Keep ctf_file_t around for compatibility. (struct ctf_file): Likewise rename to... (struct ctf_dict): ... this. (ctf_file_close): Rename to... (ctf_dict_close): ... this, keeping compatibility function. (ctf_parent_file): Rename to... (ctf_parent_dict): ... this, keeping compatibility function. All callers adjusted. * ctf.h: Rename references to ctf_file_t to ctf_dict_t. (struct ctf_archive) <ctfa_nfiles>: Rename to... <ctfa_ndicts>: ... this. ld/ChangeLog 2020-11-20 Nick Alcock <nick.alcock@oracle.com> * ldlang.c (ctf_output): This is a ctf_dict_t now. (lang_ctf_errs_warnings): Rename ctf_file_t to ctf_dict_t. (ldlang_open_ctf): Adjust comment. (lang_merge_ctf): Use ctf_dict_close, not ctf_file_close. * ldelfgen.h (ldelf_examine_strtab_for_ctf): Rename ctf_file_t to ctf_dict_t. Change opaque declaration accordingly. * ldelfgen.c (ldelf_examine_strtab_for_ctf): Adjust. * ldemul.h (examine_strtab_for_ctf): Likewise. (ldemul_examine_strtab_for_ctf): Likewise. * ldeuml.c (ldemul_examine_strtab_for_ctf): Likewise. libctf/ChangeLog 2020-11-20 Nick Alcock <nick.alcock@oracle.com> * ctf-impl.h: Rename ctf_file_t to ctf_dict_t: all declarations adjusted. (ctf_fileops): Rename to... (ctf_dictops): ... this. (ctf_dedup_t) <cd_id_to_file_t>: Rename to... <cd_id_to_dict_t>: ... this. (ctf_file_t): Fix outdated comment. <ctf_fileops>: Rename to... <ctf_dictops>: ... this. (struct ctf_archive_internal) <ctfi_file>: Rename to... <ctfi_dict>: ... this. * ctf-archive.c: Rename ctf_file_t to ctf_dict_t. Rename ctf_archive.ctfa_nfiles to ctfa_ndicts. Rename ctf_file_close to ctf_dict_close. All users adjusted. * ctf-create.c: Likewise. Refer to CTF dicts, not CTF containers. (ctf_bundle_t) <ctb_file>: Rename to... <ctb_dict): ... this. * ctf-decl.c: Rename ctf_file_t to ctf_dict_t. * ctf-dedup.c: Likewise. Rename ctf_file_close to ctf_dict_close. Refer to CTF dicts, not CTF containers. * ctf-dump.c: Likewise. * ctf-error.c: Likewise. * ctf-hash.c: Likewise. * ctf-inlines.h: Likewise. * ctf-labels.c: Likewise. * ctf-link.c: Likewise. * ctf-lookup.c: Likewise. * ctf-open-bfd.c: Likewise. * ctf-string.c: Likewise. * ctf-subr.c: Likewise. * ctf-types.c: Likewise. * ctf-util.c: Likewise. * ctf-open.c: Likewise. (ctf_file_close): Rename to... (ctf_dict_close): ...this. (ctf_file_close): New trivial wrapper around ctf_dict_close, for compatibility. (ctf_parent_file): Rename to... (ctf_parent_dict): ... this. (ctf_parent_file): New trivial wrapper around ctf_parent_dict, for compatibility. * libctf.ver: Add ctf_dict_close and ctf_parent_dict.
2020-11-20 21:34:04 +08:00
ctfa_names-relative offsets of each name or ctf_dict_t. */
libctf: mmappable archives If you need to store a large number of CTF containers somewhere, this provides a dedicated facility for doing so: an mmappable archive format like a very simple tar or ar without all the system-dependent format horrors or need for heavy file copying, with built-in compression of files above a particular size threshold. libctf automatically mmap()s uncompressed elements of these archives, or uncompresses them, as needed. (If the platform does not support mmap(), copying into dynamically-allocated buffers is used.) Archive iteration operations are partitioned into raw and non-raw forms. Raw operations pass thhe raw archive contents to the callback: non-raw forms open each member with ctf_bufopen() and pass the resulting ctf_file_t to the iterator instead. This lets you manipulate the raw data in the archive, or the contents interpreted as a CTF file, as needed. It is not yet known whether we will store CTF archives in a linked ELF object in one of these (akin to debugdata) or whether they'll get one section per TU plus one parent container for types shared between them. (In the case of ELF objects with very large numbers of TUs, an archive of all of them would seem preferable, so we might just use an archive, and add lzma support so you can assume that .gnu_debugdata and .ctf are compressed using the same algorithm if both are present.) To make usage easier, the ctf_archive_t is not the on-disk representation but an abstraction over both ctf_file_t's and archives of many ctf_file_t's: users see both CTF archives and raw CTF files as ctf_archive_t's upon opening, the only difference being that a raw CTF file has only a single "archive member", named ".ctf" (the default if a null pointer is passed in as the name). The next commit will make use of this facility, in addition to providing the public interface to actually open archives. (In the future, it should be possible to have all CTF sections in an ELF file appear as an "archive" in the same fashion.) This machinery is also used to allow library-internal creators of ctf_archive_t's (such as the next commit) to stash away an ELF string and symbol table, so that all opens of members in a given archive will use them. This lets CTF archives exploit the ELF string and symbol table just like raw CTF files can. (All this leads to somewhat confusing type naming. The ctf_archive_t is a typedef for the opaque internal type, struct ctf_archive_internal: the non-internal "struct ctf_archive" is the on-disk structure meant for other libraries manipulating CTF files. It is probably clearest to use the struct name for struct ctf_archive_internal inside the program, and the typedef names outside.) libctf/ * ctf-archive.c: New. * ctf-impl.h (ctf_archive_internal): New type. (ctf_arc_open_internal): New declaration. (ctf_arc_bufopen): Likewise. (ctf_arc_close_internal): Likewise. include/ * ctf.h (CTFA_MAGIC): New. (struct ctf_archive): New. (struct ctf_archive_modent): Likewise. * ctf-api.h (ctf_archive_member_f): New. (ctf_archive_raw_member_f): Likewise. (ctf_arc_write): Likewise. (ctf_arc_close): Likewise. (ctf_arc_open_by_name): Likewise. (ctf_archive_iter): Likewise. (ctf_archive_raw_iter): Likewise. (ctf_get_arc): Likewise.
2019-04-24 18:30:17 +08:00
typedef struct ctf_archive_modent
{
uint64_t name_offset;
uint64_t ctf_offset;
} ctf_archive_modent_t;
#ifdef __cplusplus
}
#endif
#endif /* _CTF_H */