This commit moves the two Python functions that are used for styling
into a new module, gdb.styling, there's then a small update in
python.c so GDB can find the functions in their new location.
The motivation for this change is purely to try and reduce the clutter
in the top-level gdb module, and encapsulate related functions into
modules. I did ponder documenting these functions as part of the
Python API, however, doing so would effectively "fix" the API, and I'm
still wondering if there's improvements that could be made, also, the
colorize function is only called in some cases now that GDB prefers
libsource-highlight, so it's not entirely sure how this would work as
part of a user facing API.
Still, despite these functions never having been part of a documented
API, it is possible that a user out there has overridden these to, in
some way, customize how GDB performs styling. Moving the function as
I propose in this patch could break things for that user, however,
fixing this breakage is trivial, and, as these functions were never
documented, I don't think we should be obliged to not break user code
that relies on them.
This commit adds styling support to the disassembler output, as such
two new commands are added to GDB:
set style disassembler enabled on|off
show style disassembler enabled
In this commit I make use of the Python Pygments package to provide
the styling. I did investigate making use of libsource-highlight,
however, I found the highlighting results to be inferior to those of
Pygments; only some mnemonics were highlighted, and highlighting of
register names such as r9d and r8d (on x86-64) was incorrect.
To enable disassembler highlighting via Pygments, I've added a new
extension language hook, which is then implemented for Python. This
hook is very similar to the existing hook for source code
colorization.
One possibly odd choice I made with the new hook is to pass a
gdb.Architecture through, even though this is currently unused. The
reason this argument is not used is that, currently, styling is
performed identically for all architectures.
However, even though the Python function used to perform styling of
disassembly output is not part of any documented API, I don't want
to close the door on a user overriding this function to provide
architecture specific styling. To do this, the user would inevitably
require access to the gdb.Architecture, and so I decided to add this
field now.
The styling is applied within gdb_disassembler::print_insn, to achieve
this, gdb_disassembler now writes its output into a temporary buffer,
styling is then applied to the contents of this buffer. Finally the
gdb_disassembler buffer is copied out to its final destination stream.
There's a new test to check that the disassembler output includes some
escape sequences, though I don't check for specific colours; the
precise colors will depend on which instructions are in the
disassembler output, and, I guess, how pygments is configured.
The only negative change with this commit is how we currently style
addresses in GDB.
Currently, when the disassembler wants to print an address, we call
back into GDB, and GDB prints the address value using the `address`
styling, and the symbol name using `function` styling. After this
commit, if pygments is used, then all disassembler styling is done
through pygments, and this include the address and symbol name parts
of the disassembler output.
I don't know how much of an issue this will be for people. There's
already some precedent for this in GDB when we look at source styling.
For example, function names in styled source listings are not styled
using the `function` style, but instead, either GNU Source Highlight,
or pygments gets to decide how the function name should be styled.
If the Python pygments library is not present then GDB will continue
to behave as it always has, the disassembler output is mostly
unstyled, but the address and symbols are styled using the `address`
and `function` styles, as they are today.
However, if the user does `set style disassembler enabled off`, then
all disassembler styling is switched off. This obviously covers the
use of pygments, but also includes the minimal styling done by GDB
when pygments is not available.
Don't change indirect symbol defined in IR to undefined if it is
referenced from shared object.
bfd/
PR ld/28879
* elflink.c (_bfd_elf_merge_symbol): Don't change indirect
symbol defined in IR to undefined if it is referenced from
shared object.
ld/
PR ld/28879
* testsuite/ld-plugin/lto.exp: Run PR ld/28879 tests.
* testsuite/ld-plugin/pr28879a.cc: New file.
* testsuite/ld-plugin/pr28879b.cc: Likewise.
The better to see any code that accesses expld.dataseg.
* ldexp.c (fold_segment_end): Remove seg parameter. Adjust calls.
(fold_segment_align, fold_segment_relro_end): Likewise.
* ldlang.c (lang_size_segment): Likewise.
(lang_size_relro_segment_1, lang_find_relro_sections_1): Likewise.
Now that ld properly aligns the end of the relro segment, the hack to
make relro work on powerpc can disappear.
bfd/
* bfd.c (bfd_emul_get_commonpagesize): Remove relro param.
Don't return bed->relropagesize.
* elf-bfd.h (struct elf_backend_data): Remove relropagesize.
* elfxx-target.h (ELF_RELROPAGESIZE): Remove.
* elf32-ppc.c (ELF_RELROPAGESIZE): Don't define.
* elf64-ppc.c: Likewise.
* bfd-in2.h: Regenerate.
ld/
* ldemul.c (after_parse_default): Adjust
bfd_emul_get_commonpagesize call.
x86 treats MAXPAGESIZE as a memory optimisation parameter, actual
hardware paging is always COMMPAGESIZE of 4k. Use COMMONPAGESIZE for
the end of the relro segment alignment.
The previous patch regresses pr18176, increasing the testcase file
size from 322208 to 2099872 bytes. Fixing this on x86 will require
introducing a gap after the end of the relro segment (of up to
relropagesize-1 bytes).
PR 28824
PR 18176
* ld.h (ld_config_type): Add relro_use_commonpagesize field.
* ldexp.c (fold_segment_align): Set relropagesize depending on
relro_use_commonpagesize.
* emultempl/elf-x86.em (elf_x86_create_output_section_statements):
Set relro_use_commonpagesize.
* testsuite/ld-x86-64/pr18176.d: xfail.
Background
==========
There are constraints on layout of binaries to meet demand paging and
memory protection requirements. Demand paged binaries must have file
offset mod pagesize equal to vma mod pagesize. Memory protection
(executable, read, write status) can only change at page boundaries.
The linker's MAXPAGESIZE variable gives the page size for these layout
constraints.
In a typical basic executable with two memory segments, text (RE) and
data (RW), the data segment must start on a different page to the
last text segment page. For example, with 64k pages and a small
executable of 48k text and 1k data, the text segment might start at
address 0x10000 and data at 0x20000 for a total of two 64k memory
pages. Demand paging would require the image on disk to be 64k+1k
in size. We can do better than that. If the data segment instead
starts at 0x2c000 (the end of the text segment plus one 64k page) then
there are still only two memory pages, but the disk image is now
smaller, 48k+1k in size. This is why the linker normally starts the
data segment at the end of the text segment plus one page. That
simple heuristic isn't ideal in all cases. Changing our simple
example to one with 64k-1 text size, following that heuristic would
result in data starting at 0x2ffff. Now we have two 64k memory data
pages for a data segment of 1k! If the data segment instead started
at 0x30000 we'd get a single data segment page at the cost of 1 byte
extra in the disk image, which is likely a good trade-off. So the
linker does adjust the simple heuristic. Just how much disk image
size increase is allowed is controlled by the linker's COMMONPAGESIZE
variable.
A PT_GNU_RELRO segment overlays the initial part of the data segment,
saying that those pages should be made read-only after relocation by
the dynamic loader. Page granularity for memory protection means that
the end of the relro segment must be at a page boundary.
The problem
===========
Unfortunately most targets currently only align the end of the relro
segment to COMMONPAGESIZE. That results in only partial relro
protection if an executable is running with MAXPAGESIZE pages, since
any part of the relro segment past the last MAXPAGESIZE boundary can't
be made read-only without also affecting sections past the end of the
relro segment. I believe this problem arose because x86 always runs
with 4k (COMMPAGESIZE) memory pages, and therefore using a larger
MAXPAGESIZE on x86 is for reasons other than the demand paging and
memory page protection boundary requirements.
The solution
============
Always end the relro segment on a MAXPAGESIZE boundary, except for
x86. Note that the relro segment, comprising of sections at the start
of the data segment, is sized according to how those sections are laid
out. That means the start of the relro segment is fixed relative to
its end. Which also means the start of the data segment must be at a
fixed address mod MAXPAGESIZE. So for relro the linker can't play
games with the start of the data segment to save disk space. At
least, not without introducing gaps between the relro sections. In
fact, because the linker was starting layout using its simple
heuristic of starting the data segment at the end of the text segment
plus one page, it was sometimes introducing page gaps for no reason.
See pr28743.
PR 28824
PR 28734
* ldexp.c (fold_segment_align): When relro, don't adjust up by
offset within page. Set relropagesize.
(fold_segment_relro_end): Align to relropagesize.
* ldexp.h (seg_align_type): Rename pagesize to commonpagesize.
Add relropagesize. Comment.
* ldlang.c (lang_size_segment): Adjust to suit field renaming.
(lang_size_relro_segment_1): Align relro_end using relropagesize.
I am checking this into master and will backport it to 2.38 branch.
H.J
----
On x86, GCC 12 supports -mno-direct-extern-access to enable canonical
reference to protected function and disable copy relocation. With
-mno-direct-extern-access, the canonical protected function symbols must
be accessed via canonical reference and the protected data symbols in
shared libraries are non-copyable. Under glibc 2.35, non-canonical
reference to the canonical protected function will get the run-time error:
./y: internal_f: ./libfoo.so: non-canonical reference to canonical protected function
and copy relocations against the non-copyable protected symbols will get
the run-time error:
./x: internal_i: ./libfoo.so: copy relocation against non-copyable protected symbol
Update x86 linker to disallow non-canonical reference to the canonical
protected function:
ld: plt.o: non-canonical reference to canonical protected function `internal_f' in libfoo.so
ld: failed to set dynamic section sizes: bad value
and copy relocation against the non-copyable protected symbol:
ld: main.o: copy relocation against non-copyable protected symbol `internal_i' in libfoo.so
at link-time.
bfd/
PR ld/28875
* elf-properties.c (_bfd_elf_parse_gnu_properties): Don't skip
shared libraries for GNU_PROPERTY_1_NEEDED_INDIRECT_EXTERN_ACCESS.
* elf32-i386.c (elf_i386_scan_relocs): Disallow non-canonical
reference to canonical protected function.
* elf64-x86-64.c (elf_x86_64_scan_relocs): Likewise.
* elfxx-x86.c (elf_x86_allocate_dynrelocs): Don't allow copy
relocation against non-copyable protected symbol.
ld/
PR ld/28875
* testsuite/ld-i386/i386.exp: Check non-canonical reference to
canonical protected function and check copy relocation against
non-copyable protected symbol.
* testsuite/ld-i386/pr21997-1.err: New file.
* testsuite/ld-i386/pr28875.err: Likewise.
* testsuite/ld-i386/pr28875a.c: Likewise.
* testsuite/ld-i386/pr28875b.c: Likewise.
* testsuite/ld-x86-64/pr21997-1a.err: Updated.
* testsuite/ld-x86-64/pr21997-1b.err: Likewise.
* testsuite/ld-x86-64/pr28875-data.err: New file.
* testsuite/ld-x86-64/pr28875-func.err: Likewise.
* testsuite/ld-x86-64/x86-64.exp: Check non-canonical reference
to canonical protected function and check copy relocation against
non-copyable protected symbol.
Gfortran supports namelists (a Fortran feature); it emits
DW_TAG_namelist and DW_TAG_namelist_item dies. But gdb does not
process these dies and does not support 'print' or 'ptype' commands on
namelist variables.
An attempt to print namelist variables results in gdb bailing out with
the error message as shown below.
(gdb) print nml
No symbol "nml" in current context.
This commit is to make the print and ptype commands work for namelist
variables and its items. Sample output of these commands is shared
below, with fixed gdb.
(gdb) ptype nml
type = Type nml
integer(kind=4) :: a
integer(kind=4) :: b
End Type nml
(gdb) print nml
$1 = ( a = 10, b = 20 )
When using the command "until", it is expected that GDB will exit a
loop if the current instruction is the last one related to that loop.
However, if there were trailing non-statement instructions, "until"
would just behave as "next". This was noticeable in clang-compiled
code, but might happen with gcc-compiled as well. PR gdb/17315 relates
to this problem, as running gdb.base/watchpoint.exp with clang
would fail for this reason.
To better understand this issue, consider the following source code,
with line numbers marked on the left:
10: for (i = 0; i < 10; ++i)
11: loop_body ();
12: other_stuff ();
If we transform this to pseudo-assembler, and generate a line table,
we could end up with something like this:
Address | Pseudo-Assembler | Line | Is-Statement?
0x100 | i = 0 | 10 | Yes
0x104 | loop_body () | 11 | Yes
0x108 | i = i + 1 | 10 | Yes
0x10c | if (i < 10): | 10 | No
0x110 | goto 0x104 | 10 | No
0x114 | other_stuff () | 12 | Yes
Notice the two non-statement instructions at the end of the loop.
The problem is that when we reach address 0x108 and use 'until',
hoping to leave the loop, GDB sets up a stepping range that runs from
the start of the function (0x100 in our example) to the end of the
current line table entry, that is 0x10c in our example. GDB then
starts stepping forward.
When 0x10c is reached GDB spots that we have left the stepping range,
that the new location is not a statement, and that the new location is
associated with the same source line number as the previous stepping
range. GDB then sets up a new stepping range that runs from 0x10c to
0x114, and continues stepping forward.
Within that stepping range the inferior hits the goto (at 0x110) and
loops back to address 0x104.
At 0x104 GDB spots that we have left the previous stepping range, that
the new address is marked as a statement, and that the new address is
for a different source line. As a result, GDB stops and returns
control to the user. This is not what the user was expecting, they
expected GDB to exit the loop.
The fix proposed in this patch, is that, when the user issues the
'until' command, and GDB sets up the initial stepping range, GDB will
check subsequent SALs (symtab_and_lines) to see if they are
non-statements associated with the same line number. If they are then
the end of the initial stepping range is extended to the end of the
non-statement SALs.
In our example above, the user is at 0x108 and uses 'until', GDB now
sets up a stepping range from the start of the function 0x100 to
0x114, the first address associated with a different line.
Now as GDB steps around the loop it never leaves the initial stepping
range. It is only when GDB exits the loop that we leave the stepping
range, and the stepping finishes at address 0x114.
This patch also adds a test case that can be run with gcc to test that
this functionality is not broken in the future.
Bug: https://sourceware.org/bugzilla/show_bug.cgi?id=17315
This commit updates the output of 'maint info jit' to print not just
the jit_code_entry address, but also the symfile address, and the
symfile size.
The new information could be obtained by looking into target memory at
the contents of the jit_code_entry, but, by storing this information
within gdb at the time the jit object is loaded, it is now possible to
check if the jit_code_entry has been modified in target memory behind
gdb's back.
Additionally, the symfile address is the same address that is now used
in the objfile names after commit 4a620b7e.
One test that relies on the output of 'maint info jit' was updated to
allow for the new output format.
This commit adds Makefile, configure and NEWS for LoongArch.
Signed-off-by: Zhensong Liu <liuzhensong@loongson.cn>
Signed-off-by: Qing zhang <zhangqing@loongson.cn>
Signed-off-by: Youling Tang <tangyouling@loongson.cn>
Signed-off-by: Tiezhu Yang <yangtiezhu@loongson.cn>
This commit adds initial native Linux support for LoongArch.
Signed-off-by: Zhensong Liu <liuzhensong@loongson.cn>
Signed-off-by: Qing zhang <zhangqing@loongson.cn>
Signed-off-by: Youling Tang <tangyouling@loongson.cn>
Signed-off-by: Tiezhu Yang <yangtiezhu@loongson.cn>
This commit adds initial Linux target support for LoongArch.
Signed-off-by: Zhensong Liu <liuzhensong@loongson.cn>
Signed-off-by: Qing zhang <zhangqing@loongson.cn>
Signed-off-by: Youling Tang <tangyouling@loongson.cn>
Signed-off-by: Tiezhu Yang <yangtiezhu@loongson.cn>
This commit adds initial baremetal support for LoongArch.
Signed-off-by: Zhensong Liu <liuzhensong@loongson.cn>
Signed-off-by: Qing zhang <zhangqing@loongson.cn>
Signed-off-by: Youling Tang <tangyouling@loongson.cn>
Signed-off-by: Tiezhu Yang <yangtiezhu@loongson.cn>
This commit adds initial target description support for LoongArch.
Signed-off-by: Zhensong Liu <liuzhensong@loongson.cn>
Signed-off-by: Qing zhang <zhangqing@loongson.cn>
Signed-off-by: Youling Tang <tangyouling@loongson.cn>
Signed-off-by: Tiezhu Yang <yangtiezhu@loongson.cn>
While reviewing a different patch I wanted to know more about what was
going on during GDB's stepping. I added some extra infrun debug print
calls, and I thought these might be useful to others.
elf64-ppc.c: In function 'ppc64_elf_size_dynamic_sections':
elf64-ppc.c:10309:45: error: value computed is not used [-Werror=unused-value]
++lgot_ents, ++lgot_masks, isym != NULL && isym++)
It is of course a silly warning, fixed in later versions of gcc. I
wrote "isym != NULL && isym++" rather than the simpler "isym++" to
stop sanitisers complaining about incrementing a NULL pointer. isym
is of course unused in any code path where it might start off as
NULL. Sometimes you can't win. So don't try to be clever in reading
local symbols only when needed. 99 times out of 100 they will be
cached anyway.
* elf64-ppc.c (ppc64_elf_size_dynamic_sections): Avoid annoying
warnings by always reading local syms.
(ppc64_elf_layout_multitoc): Likewise.
Add a test for commit 7c4643efe7, which fixed --only-keep-debug for ELF
relocatables.
* testsuite/binutils-all/objcopy.exp
(keep_debug_symbols_for_elf_relocatable): New test.
The extension version checking logic is really just too complicated to
encode into the linker, trying to do so causes more harm than good.
This removes the checks and the associated tests, leaving the logic to
keep the largest version of each extension linked into the target.
bfd/
* elfnn-riscv.c (riscv_version_mismatch): Rename to
riscv_update_subset_version, and stop reporting warnings on
version mismatches.
(riscv_merge_std_ext): Adjust calls to riscv_version_mismatch.
(riscv_merge_multi_letter_ext): Likewise.
ld/
* testsuite/ld-riscv-elf/attr-merge-arch-failed-01.d: Remove
* testsuite/ld-riscv-elf/attr-merge-arch-failed-01a.s: Likewise
* testsuite/ld-riscv-elf/attr-merge-arch-failed-01b.s: Likewise
* testsuite/ld-riscv-elf/attr-merge-arch-failed-02.d: Likewise
* testsuite/ld-riscv-elf/attr-merge-arch-failed-02a.s: Likewise
* testsuite/ld-riscv-elf/attr-merge-arch-failed-02b.s: Likewise
* testsuite/ld-riscv-elf/attr-merge-arch-failed-02c.s: Likewise
* testsuite/ld-riscv-elf/attr-merge-arch-failed-02d.s: Likewise
* testsuite/ld-riscv-elf/attr-merge-user-ext-01.d: New test.
* testsuite/ld-riscv-elf/attr-merge-user-ext-rv32i21_m2p0.s:
Likewise.
* testsuite/ld-riscv-elf/attr-merge-user-ext-rv32i21_m2p1.s:
Likewise.
* testsuite/ld-riscv-elf/ld-riscv-elf.exp: Remove obselete
attr-merge-arch-failed-{01,02}, replace with
attr-merge-user-ext-01.
Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
I have no info on the format of a "SUNPRO C++ Namespace" stab, so am
relying on the previous code being correct in parsing these stabs.
Just don't allow NULs anywhere in the stab.
PR 28862
* stabs.c (parse_stab_string): Don't overrun buffer when parsing
'Y' stab.
I noticed that most of the calls to index_cache_debug include a
trailing newline. As the new debug mechanism already adds a newline,
that means all of these debug calls result in a blank line being
printed, which I think is a mistake.
Remove all the trailing newlines.
I also reformatted one of the index_cache_debug where a string will
now fit onto a single line.
Unless 'set debug index-cache on' is used, there should be no visible
change in output after this commit.
GOT32 relocations are allowed since absolute value + addend is stored in
the GOT slot.
Tested on glibc 2.35 build with GCC 11.2 and -Os.
bfd/
PR ld/28870
* elfxx-x86.c (_bfd_elf_x86_valid_reloc_p): Also allow GOT32
relocations.
ld/
PR ld/28870
* testsuite/ld-i386/i386.exp: Run pr28870.
* testsuite/ld-i386/pr28870.d: New file.
* testsuite/ld-i386/pr28870.s: Likewise.
Add a new argument to the gdb.Value.format_string method, 'styling'.
This argument is False by default.
When this argument is True, then the returned string can contain output
styling escape sequences.
When this argument is False, then the returned string will not contain
any styling escape sequences.
If the returned string is going to be printed to the user, then it is
often nice to retain the GDB styling.
For the testing, we need to adjust the TERM environment variable, as
we do for all the styling tests. I'm now running all of the C tests
in gdb.python/py-format-string.exp in an environment where styling
could be generated, but only my new test should actually produce
styled output, hopefully this will catch the case where a bug might
cause format_string to always produce styled output.
While working on function calls, I realized that the thread_fsm member
of struct thread_info is a raw pointer to a resource it owns. This
commit changes the type of the thread_fsm member to a std::unique_ptr in
order to signify this ownership relationship and slightly ease resource
management (no need to manually call delete).
To ensure consistent use, the field is made a private member
(m_thread_fsm). The setter method (set_thread_fsm) can then check
that it is incorrect to associate a FSM to a thread_info object if
another one is already in place. This is ensured by an assertion.
The function run_inferior_call takes an argument as a pointer to a
call_thread_fsm and installs it in it in a thread_info instance. Also
change this function's signature to accept a unique_ptr in order to
signify that the ownership of the call_thread_fsm is transferred during
the call.
No user visible change expected after this commit.
Tested on x86_64-linux with no regression observed.
Change-Id: Ia1224f72a4afa247801ce6650ce82f90224a9ae8
This commit should fix PR gdb/28711. What's actually going on is
pretty involved, and there's still a bit of the story that I don't
understand completely, however, from my observed results, I think that
the change I propose making here (or something very similar) is going
to be needed.
The original bug report involves using eclipse to drive gdb using mi
commands. A separate tty is spun off in which to send gdb the mi
commands, this tty is created using the new-ui command.
The behaviour observed is that, given a particular set of mi commands
being sent to gdb, we sometimes see an ESPIPE error from a lseek
call, which ultimately results in gdb terminating.
The problems all originate from gdb_readline_no_editing_callback in
gdb/event-top.c, where we can (sometimes) perform calls to fgetc, and
allow glibc to perform buffering on the FILE object being used.
I say sometime, because, gdb_readline_no_editing_callback already
includes a call to disable the glibc buffering, but this is only done
if the input stream is not a tty. In our case the input stream is a
tty, so the buffering is left in place.
The first step to understanding why this problem occurs is to
understand that eclipse sends multiple commands to gdb very quickly
without waiting for and answer to each command, eclipse plans to
collect all of the command results after sending all the commands to
gdb. In fact, eclipse sends the commands to gdb that they appear to
arrive in the gdb process as a single block of data. When reproducing
this issue within the testsuite I find it necessary to send multiple
commands using a single write call.
The next bit of the story gets a little involved, and this is where my
understanding is not complete. I can describe the behaviour that I
observe, and (for me at least) I'm happy that what I'm seeing, if a
little strange, is consistent. In order to fully understand what's
going on I think I would likely need to dive into kernel code, which
currently seems unnecessary given that I'm happy with the solution I'm
proposing.
The following description all relates to input from a tty in which I'm
not using readline. I see the same problems either when using a
new-ui tty, or with gdb's standard, non-readline, mi tty.
Here's what I observe happening when I send multiple commands to gdb
using a single write, if I send gdb this:
command_1\ncommand_2\ncommand_3
then gdb's event loop will wake up (from its select) as it sees there
is input available. We call into gdb_readline_no_editing_callback,
where we call fgetc, glibc will do a single big read, and get back
just:
command_1\n
that is, despite there being multiple lines of input available, I
consistently get just a single line. From glibc a single character is
returned from the fgetc call, and within gdb we accumulate characters,
one at a time, into our own buffer. Eventually gdb sees the '\n'
character, and dispatches the whole 'command_1' into gdb's command
handler, which processes the command and prints the result. We then
return to gdb_readline_no_editing_callback, which in turn returns to
gdb's event loop where we re-enter the select.
Inside the select we immediately see that there is more input waiting
on the input stream, drop out of the select, and call back into
gdb_readline_no_editing_callback. In this function we again call
fgetc where glibc performs another big read. This time glibc gets:
command_2\n
that is, we once again get just a single line, despite there being a
third line available. Just like the first command we copy the whole
string, character by character into gdb's buffer, then handle the
command. After handling the command we go to the event loop, enter,
and then exit the select, and call back to the function
gdb_readline_no_editing_callback.
In gdb_readline_no_editing_callback we again call fgetc, this time
glibc gets the string:
command_3\n
like before, we copy this to gdb's buffer and handle the command, then
we return to the event loop. At this point the select blocks while we
wait for more input to arrive.
The important bit of this is that someone, somewhere is, it appears,
taking care to split the incoming write into lines.
My next experiment is to try something like:
this_is_a_very_long_command\nshort_command\n
However, I actually make 'this_is_a_very_long_command' very long, as
in many hundreds of characters long. One way to do this is:
echo xxxxxx.....xxxxx
and just adding more and more 'x' characters as needed. What I'm
aiming for is to have the first command be longer than glibc's
internal read buffer, which, on my machine, is 1024 characters.
However, for this discussion, lets imagine that glibc's buffer is just
8 characters (we can create just this situation by adding a suitable
setbuf call into gdb_readline_no_editing_callback).
Now, if I send gdb this data:
abcdefghij\nkl\n
The first read from glibc will get 'abcdefgh', that is a full 8
character buffer. Once gdb has copied these to its buffer we call
fgetc again, and now glibc will get 'ij\n', that is, just like before,
multiple lines are split at the '\n' character. The full command,
which is now in gdb's buffer can be handled 'abcdefghij', after which
we go (via the event loop) back to gdb_readline_no_editing_callback.
Now we call fgetc, and glibc will get 'kl\n', which is then handled in
the normal way.
So far, so good. However, there is, apparently, one edge case where
the above rules don't apply.
If the '\n' character is the first character read from the kernel,
then the incoming lines are not split up. So, given glibc's 8
character buffer, if I send gdb this:
abcdefgh\nkl\n
that is the first command is 8 characters plus a newline, then, on the
first read (from within glibc) we get 'abcdefgh' in a single buffer.
As there's no newline gdb calls fgetc again, and glibc does another
large read, now we get:
\nkl\n
which doesn't follow the above pattern - the lines are not split into
separate buffers!
So, gdb reads the first character from glibc using fgetc, this is the
newline. Now gdb has a complete command, and so the command is
handled. We then return to the event loop and enter the select.
The problem is that, as far as the kernel is concerned, there is no
more input pending, it's all been read into glibc's buffer, and so the
select doesn't return. The second command is basically stuck in
glibc's buffer.
If I send another command to gdb, or even just send an empty
command (a lone newline) then the select returns, we call into
gdb_readline_no_editing_callback, and now gdb sees the second
command.
OK, so the above is interesting, but it doesn't explain the ESPIPE
error.
Well, that's a slightly different, but related issue. The ESPIPE
case will _only_ show up when using new-ui to create the separate tty
for mi commands, and is a consequence of this commit:
commit afe09f0b63
Date: Thu Jul 18 17:20:04 2019 +0100
Fix for using named pipes on Windows
Prior to this commit, the new-ui command would open the tty three
times, once each for stdin, stderr, and stdout. After this commit we
open the tty just once and reuse the FILE object for all three roles.
Consider the problem case, where glibc has (unexpectedly) read the
second command into its internal buffer. When we handle the first
command we usually end up having to write something to the mi output
stream.
After the above commit the same FILE object represents both the input
and output streams, so, when gdb tries to write to the FILE object,
glibc spots that there is input pending within the input buffer, and
so assumes that we have read ahead of where we should be in the input
file. To correct for this glibc tries to do an lseek call to
reposition the file offset of the output stream prior to writing to
it. However, as the output stream is a tty, and seeking is not
supported on a tty, this lseek call fails, this results in the ESPIPE,
which ultimately causes gdb to terminate.
So, now we understand why the ESPIPE triggers (which was what caused
the gdb crash in the original bug report), and we also understand that
sometime gdb will not handle the second command in a timely
fashion (if the first command is just the wrong length). So, what to
do about all this?
We could revert the commit mentioned above (and implement its
functionality another way). This would certainly resolve the ESPIPE
issue, the buffered input would now only be on the input stream, the
output stream would have no buffered input, and so glibc would never
try to lseek, and so we'd never get the ESPIPE error.
However, this only solves one of the two problems. We would still
suffer from the problem where, if the first command is just the wrong
length, the second command will not (immediately) get handled.
The only solution I can see to this problem is to unbuffer the input
stream. If glibc is not buffering the input, but instead, we read
incoming data character by character from the kernel, then everything
will be fine. As soon as we see the newline at the end of the first
command we will handle the first command. As glibc will have no
buffered input it will not be tempted to lseek, so no ESPIPE error.
When we go have to the event loop there will be more data pending in
the kernel, so the select will immediately return, and the second
command will be processed.
I'm tempted to suggest that we should move the unbuffering of the
input stream out of gdb_readline_no_editing_callback and do it
somewhere earlier, more like when we create the input streams.
However, I've not done that in this commit for a couple of reasons:
1. By keeping the unbuffering in gdb_readline_no_editing_callback
I'm making the smallest possible change that fixes the bug. Moving
the unbuffering somewhere better can be done as a refactor later, if
that 's felt to be important,
2. I don't think making repeated calls to unbuffer the input will
have that much performance impact. We only make the unbuffer call
once per call to gdb_readline_no_editing_callback, and, if the input
stream is already unbuffered we'll return pretty quickly, so I don't
see this as being massively costly,
3. Tom is currently doing lots of gdb stream management changes and
I want to minimise the chances we'll conflict.
So, this commit just changes gdb_readline_no_editing_callback to
always unbuffer the input stream.
The test for this issue sends two commands in a loop, with the first
command growing bigger each time around the loop. I actually make the
first command bigger by just adding whitespace to the front, as gdb
still has to read the complete command (including whitespace) via
glibc, so this is enough to trigger the bug.
The original bug was reported when using a virtual machine, and in
this situation we see this in the strace output:
read(9, "70-var-info-path-expression var1.aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", 1024) = 64
read(9, "\n71-var-info-path-expression var1.aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa\n", 1024) = 67
I'm not completely sure what's going on here, but it appears that the
kernel on the virtual machine is delivering the input to glibc slower
than I see on my real hardware; glibc asks for 1024 bytes, but only
gets 64 bytes the first time. In the second read we see the problem
case, the first character is the newline, but then the entire second
command is included.
If I run this exact example on my real hardware then the first command
would not be truncated at 64 bytes, instead, I'd expect to see the
newline included in the first read, with the second command split into
a second read.
So, for testing, I check cases where the first command is just a few
characters (starting at 8 character), all the way up to 2048
characters. Hopefully, this should mean we hit the problem case for
most machine setups.
The only last question relates to commit afe09f0b63 that I
mentioned earlier. That commit was intended to provide support for
Microsoft named pipes:
https://docs.microsoft.com/en-us/windows/win32/ipc/named-pipes
I know next to nothing about this topic beyond a brief scan of the
above link, but I think these windows named pipe are closer in
behaviour to unix sockets than to unix named fifos.
I am a little nervous that, after the above commit, we now use the
same FILE for in, err, and out streams. In contrast, in a vanilla C
program, I would expect different FILE objects for each stream.
Still, I'm reluctant to revert the above commit (and provide the same
functionality a different way) without a specific bug to point at,
and, now that the streams are unbuffered, I expect a lot of the read
and write calls are going straight to the kernel with minimal glibc
involvement, so maybe it doesn't really matter. Anyway, I haven't
touched the above patch, but it is something to keep in mind when
working in this area.
Bug: https://sourceware.org/bugzilla/show_bug.cgi?id=28711
We have three places in gdb where we initialise a disassembler that
will not print anything (used for figuring out the length of
instructions, or collecting other information from the disassembler).
Each of these places has its own stub function to act as a print like
callback, the stub function is identical in each case, and just does
nothing.
In this commit I create a new function to initialise a disassembler
that doesn't print anything, and have all three locations use this new
function. There's now only one non-printing stub function.
There should be no user visible changes after this commit.
GDB already has a flag to suppress printing notification events, such
as thread and inferior context switches, on the CLI. This is used
internally when executing commands. Make the flag available to the
user via a new command. This is expected to be useful in scripts.
For instance, suppose that when Inferior 1 gets to a certain state,
you want to add and set up a new inferior using the commands below,
but you also want to have a reduced/clean output.
define do-setup
printf "Setting up Inferior 2...\n"
add-inferior -exec a.out
inferior 2
break file.c:3
run
inferior 1
printf "Done\n"
end
Currently, GDB prints
(gdb) do-setup
Setting up Inferior 2...
[New inferior 2]
Added inferior 2 on connection 1 (native)
[Switching to inferior 2 [<null>] (/tmp/a.out)]
Breakpoint 2 at 0x1155: file file.c, line 3.
Thread 2.1 "a.out" hit Breakpoint 2, main () at file.c:3
3 return 0;
[Switching to inferior 1 [process 7670] (/tmp/test)]
[Switching to thread 1.1 (process 7670)]
#0 main () at test.c:2
2 int a = 1;
Done
GDB's Python API make it possible to capture and return GDB's output,
but this does not work for all the streams. In particular, CLI
notification events are not captured:
(gdb) python gdb.execute("do-setup", False, True)
[Switching to inferior 2 [<null>] (/tmp/a.out)]
Thread 2.1 "a.out" hit Breakpoint 2, main () at file.c:3
3 return 0;
[Switching to inferior 1 [process 8263] (/tmp/test)]
[Switching to thread 1.1 (process 8263)]
#0 main () at test.c:2
2 int a = 1;
You can use the new "set suppress-cli-notifications" command to
suppress the output:
(gdb) set suppress-cli-notifications on
(gdb) do-setup
Setting up Inferior 2...
[New inferior 2]
Added inferior 2 on connection 1 (native)
Breakpoint 2 at 0x1155: file file.c, line 3.
Done
Extend the 'cli_suppress_notification' struct with a new field,
'normal_stop', that can be used for checking if printing normal stop
events on the CLI should be suppressed.
This patch only introduces the flag. The subsequent patch adds a user
command to turn the flag off/on.
