binutils-gdb/gdbserver
Pedro Alves 7e9cf1fe36 gdbserver: spurious SIGTRAP w/ detach while step-over in progress
A following patch will add a new testcase that has two processes, each
with a number of threads constantly tripping a breakpoint and stepping
over it, because the breakpoint has a condition that evals false.
Then GDB detaches from one of the processes, while both processes are
running.  And then the testcase sends a SIGUSR1 to the other process.

When run against gdbserver, that would occasionaly fail like this:

 (gdb) PASS: gdb.threads/detach-step-over.exp: iter 1: detach
 Executing on target: kill -SIGUSR1 208303    (timeout = 300)
 spawn -ignore SIGHUP kill -SIGUSR1 208303

 Thread 2.5 "detach-step-ove" received signal SIGTRAP, Trace/breakpoint trap.
 [Switching to Thread 208303.208305]
 0x000055555555522a in thread_func (arg=0x0) at /home/pedro/gdb/binutils-gdb/src/gdb/testsuite/gdb.threads/detach-step-over.c:54
 54            counter++; /* Set breakpoint here.  */

What happened was that GDBserver is doing a step-over for process A
when a detach request for process B arrives.  And that generates a
spurious SIGTRAP report for process A, as seen above.

The GDBserver logs reveal what happened:

 - GDB manages to detach while a step over is in progress.  That reaches
   linux_process_target::complete_ongoing_step_over(), which does:

      /* Passing NULL_PTID as filter indicates we want all events to
	 be left pending.  Eventually this returns when there are no
	 unwaited-for children left.  */
      ret = wait_for_event_filtered (minus_one_ptid, null_ptid, &wstat,
				     __WALL);

   As the comment say, this leaves all events pending, _including_ the
   just finished step SIGTRAP.  We never discard that SIGTRAP.  So
   GDBserver reports the SIGTRAP to GDB.  GDB can't explain the
   SIGTRAP, so it reports it to the user.

The GDBserver log looks like this.  The LWP of interest is 208305:

 Need step over [LWP 208305]? yes, found breakpoint at 0x555555555227
 proceed_all_lwps: found thread 208305 needing a step-over
 Starting step-over on LWP 208305.  Stopping all threads

208305 starts a step-over.

 >>>> entering void linux_process_target::stop_all_lwps(int, lwp_info*)
 stop_all_lwps (stop-and-suspend, except=LWP 208303.208305)
 Sending sigstop to lwp 208303
 Sending sigstop to lwp 207755
 wait_for_sigstop: pulling events
 LWFE: waitpid(-1, ...) returned 207755, ERRNO-OK
 LLW: waitpid 207755 received Stopped (signal) (stopped)
 pc is 0x7f7e045593bf
 Expected stop.
 LLW: SIGSTOP caught for LWP 207755.207755 while stopping threads.
 LWFE: waitpid(-1, ...) returned 208303, ERRNO-OK
 LLW: waitpid 208303 received Stopped (signal) (stopped)
 pc is 0x7ffff7e743bf
 Expected stop.
 LLW: SIGSTOP caught for LWP 208303.208303 while stopping threads.
 LWFE: waitpid(-1, ...) returned 0, ERRNO-OK
 leader_pid=208303, leader_lp!=NULL=1, num_lwps=11, zombie=0
 leader_pid=207755, leader_lp!=NULL=1, num_lwps=11, zombie=0
 LLW: exit (no unwaited-for LWP)
 stop_all_lwps done, setting stopping_threads back to !stopping
 <<<< exiting void linux_process_target::stop_all_lwps(int, lwp_info*)
 Done stopping all threads for step-over.
 pc is 0x555555555227
 Writing 8b to 0x555555555227 in process 208305
 Could not findsigchld_handler
  fast tracepoint jump at 0x555555555227 in list (uninserting).
   pending reinsert at 0x555555555227
   step from pc 0x555555555227
 Resuming lwp 208305 (step, signal 0, stop expected)
 <<<< exiting ptid_t linux_process_target::wait_1(ptid_t, target_waitstatus*, target_wait_flags)
 handling possible serial event
 getpkt ("D;32b8b");  [no ack sent]

The detach request arrives.

 sigchld_handler
 Tracing is already off, ignoring
 detach: step over in progress, finish it first

GDBserver realizes a step over for 208305 was in progress, let's it
finish.

 LWFE: waitpid(-1, ...) returned 208305, ERRNO-OK
 LLW: waitpid 208305 received Stopped (signal) (stopped)
 pc is 0x555555555227
 Expected stop.
 LLW: step LWP 208303.208305, 0, 0 (discard delayed SIGSTOP)
   pending reinsert at 0x555555555227
   step from pc 0x555555555227
 Resuming lwp 208305 (step, signal 0, stop not expected)
 LWFE: waitpid(-1, ...) returned 0, ERRNO-OK
 leader_pid=208303, leader_lp!=NULL=1, num_lwps=11, zombie=0
 leader_pid=207755, leader_lp!=NULL=1, num_lwps=11, zombie=0
 sigsuspend'ing
 LWFE: waitpid(-1, ...) returned 208305, ERRNO-OK
 LLW: waitpid 208305 received Trace/breakpoint trap (stopped)
 pc is 0x55555555522a
 CSBB: LWP 208303.208305 stopped by trace
 LWFE: waitpid(-1, ...) returned 0, ERRNO-OK
 leader_pid=208303, leader_lp!=NULL=1, num_lwps=11, zombie=0
 leader_pid=207755, leader_lp!=NULL=1, num_lwps=11, zombie=0
 LLW: exit (no unwaited-for LWP)
 Finished step over.

The step-over for 208305 finishes.

 Writing cc to 0x555555555227 in process 208305
 Could not find fast tracepoint jump at 0x555555555227 in list (reinserting).
 >>>> entering void linux_process_target::stop_all_lwps(int, lwp_info*)
 stop_all_lwps (stop, except=none)
 wait_for_sigstop: pulling events

The detach proceeds (snipped).

...

 proceed_one_lwp: lwp 208305
    LWP 208305 has pending status, leaving stopped

Later on, 208305 has a pending status (the step SIGTRAP from the
step-over), so GDBserver starts the process of reporting it.

...

 wait_1 ret = LWP 208303.208305, 1, 5
 <<<< exiting ptid_t linux_process_target::wait_1(ptid_t, target_waitstatus*, target_wait_flags)

...

and eventually GDB receives the stop notification (T05 == SIGTRAP):

 getpkt ("vStopped");  [no ack sent]
 sigchld_handler
 vStopped: acking 3
 Writing resume reply for LWP 208303.208305:1
 putpkt ("$T0506:f0ee58f7ff7f0* ;07:f0ee58f7ff7f0* ;10:2a525*"550* ;thread:p32daf.32db1;core:c;#37"); [noack mode]

From the GDB side, we see:

 [infrun] fetch_inferior_event: enter
   [infrun] fetch_inferior_event: fetch_inferior_event enter
   [infrun] do_target_wait: Found 2 inferiors, starting at #1
   [infrun] print_target_wait_results: target_wait (-1.0.0 [process -1], status) =
   [infrun] print_target_wait_results:   208303.208305.0 [Thread 208303.208305],
   [infrun] print_target_wait_results:   status->kind = stopped, signal = GDB_SIGNAL_TRAP
   [infrun] handle_inferior_event: status->kind = stopped, signal = GDB_SIGNAL_TRAP
   [infrun] start_step_over: enter
     [infrun] start_step_over: stealing global queue of threads to step, length = 6
     [infrun] operator(): putting back 6 threads to step in global queue
   [infrun] start_step_over: exit
   [infrun] handle_signal_stop: context switch
   [infrun] context_switch: Switching context from process 0 to Thread 208303.208305
   [infrun] handle_signal_stop: stop_pc=0x55555555522a
   [infrun] handle_signal_stop: random signal (GDB_SIGNAL_TRAP)
   [infrun] stop_waiting: stop_waiting
   [infrun] stop_all_threads: starting

The fix is to discard the step SIGTRAP, unless GDB wanted the thread
to step.

gdbserver/ChangeLog:

	* linux-low.cc (linux_process_target::complete_ongoing_step_over):
	Discard step SIGTRAP, unless GDB wanted the thread to step.
2021-02-03 01:14:53 +00:00
..
.dir-locals.el Update copyright year range in all GDB files 2021-01-01 12:12:21 +04:00
.gitattributes
.gitignore
acinclude.m4 gdbserver: replace AC_TRY_COMPILE in acinclude.m4 2020-10-31 08:31:00 -04:00
aclocal.m4
ax.cc gdb: make some variables static 2021-01-20 20:55:05 -05:00
ax.h Update copyright year range in all GDB files 2021-01-01 12:12:21 +04:00
ChangeLog gdbserver: spurious SIGTRAP w/ detach while step-over in progress 2021-02-03 01:14:53 +00:00
config.in gdbserver: re-generate configure 2020-10-24 12:44:19 -04:00
configure gdbserver: replace AC_TRY_COMPILE in acinclude.m4 2020-10-31 08:31:00 -04:00
configure.ac Update copyright year range in all GDB files 2021-01-01 12:12:21 +04:00
configure.srv gdbserver: Add GNU/Linux support for ARC 2020-10-07 17:46:23 +02:00
debug.cc gdb: make some variables static 2021-01-20 20:55:05 -05:00
debug.h Update copyright year range in all GDB files 2021-01-01 12:12:21 +04:00
dll.cc Update copyright year range in all GDB files 2021-01-01 12:12:21 +04:00
dll.h Update copyright year range in all GDB files 2021-01-01 12:12:21 +04:00
fork-child.cc Update copyright year range in all GDB files 2021-01-01 12:12:21 +04:00
gdb_proc_service.h Update copyright year range in all GDB files 2021-01-01 12:12:21 +04:00
gdbreplay.cc Update copyright year range in all GDB files 2021-01-01 12:12:21 +04:00
gdbthread.h Update copyright year range in all GDB files 2021-01-01 12:12:21 +04:00
hostio-errno.cc Update copyright year range in all GDB files 2021-01-01 12:12:21 +04:00
hostio.cc Update copyright year range in all GDB files 2021-01-01 12:12:21 +04:00
hostio.h Update copyright year range in all GDB files 2021-01-01 12:12:21 +04:00
i387-fp.cc Update copyright year range in all GDB files 2021-01-01 12:12:21 +04:00
i387-fp.h Update copyright year range in all GDB files 2021-01-01 12:12:21 +04:00
inferiors.cc Update copyright year range in all GDB files 2021-01-01 12:12:21 +04:00
inferiors.h Update copyright year range in all GDB files 2021-01-01 12:12:21 +04:00
linux-aarch32-low.cc Update copyright year range in all GDB files 2021-01-01 12:12:21 +04:00
linux-aarch32-low.h Update copyright year range in all GDB files 2021-01-01 12:12:21 +04:00
linux-aarch32-tdesc.cc Update copyright year range in all GDB files 2021-01-01 12:12:21 +04:00
linux-aarch32-tdesc.h Update copyright year range in all GDB files 2021-01-01 12:12:21 +04:00
linux-aarch64-ipa.cc Update copyright year range in all GDB files 2021-01-01 12:12:21 +04:00
linux-aarch64-low.cc Update copyright year range in all GDB files 2021-01-01 12:12:21 +04:00
linux-aarch64-tdesc.cc Update copyright year range in all GDB files 2021-01-01 12:12:21 +04:00
linux-aarch64-tdesc.h Update copyright year range in all GDB files 2021-01-01 12:12:21 +04:00
linux-amd64-ipa.cc Update copyright year range in all GDB files 2021-01-01 12:12:21 +04:00
linux-arc-low.cc Update copyright year range in all GDB files 2021-01-01 12:12:21 +04:00
linux-arm-low.cc Update copyright year range in all GDB files 2021-01-01 12:12:21 +04:00
linux-arm-tdesc.cc Update copyright year range in all GDB files 2021-01-01 12:12:21 +04:00
linux-arm-tdesc.h Update copyright year range in all GDB files 2021-01-01 12:12:21 +04:00
linux-i386-ipa.cc Update copyright year range in all GDB files 2021-01-01 12:12:21 +04:00
linux-ia64-low.cc Update copyright year range in all GDB files 2021-01-01 12:12:21 +04:00
linux-low.cc gdbserver: spurious SIGTRAP w/ detach while step-over in progress 2021-02-03 01:14:53 +00:00
linux-low.h Update copyright year range in all GDB files 2021-01-01 12:12:21 +04:00
linux-m68k-low.cc Update copyright year range in all GDB files 2021-01-01 12:12:21 +04:00
linux-mips-low.cc Update copyright year range in all GDB files 2021-01-01 12:12:21 +04:00
linux-nios2-low.cc Update copyright year range in all GDB files 2021-01-01 12:12:21 +04:00
linux-ppc-ipa.cc Update copyright year range in all GDB files 2021-01-01 12:12:21 +04:00
linux-ppc-low.cc Update copyright year range in all GDB files 2021-01-01 12:12:21 +04:00
linux-ppc-tdesc-init.h Update copyright year range in all GDB files 2021-01-01 12:12:21 +04:00
linux-riscv-low.cc Update copyright year range in all GDB files 2021-01-01 12:12:21 +04:00
linux-s390-ipa.cc Update copyright year range in all GDB files 2021-01-01 12:12:21 +04:00
linux-s390-low.cc Update copyright year range in all GDB files 2021-01-01 12:12:21 +04:00
linux-s390-tdesc.h Update copyright year range in all GDB files 2021-01-01 12:12:21 +04:00
linux-sh-low.cc Update copyright year range in all GDB files 2021-01-01 12:12:21 +04:00
linux-sparc-low.cc Update copyright year range in all GDB files 2021-01-01 12:12:21 +04:00
linux-tic6x-low.cc Update copyright year range in all GDB files 2021-01-01 12:12:21 +04:00
linux-x86-low.cc gdb: make some variables static 2021-01-20 20:55:05 -05:00
linux-x86-tdesc.cc Update copyright year range in all GDB files 2021-01-01 12:12:21 +04:00
linux-x86-tdesc.h Update copyright year range in all GDB files 2021-01-01 12:12:21 +04:00
linux-xtensa-low.cc Update copyright year range in all GDB files 2021-01-01 12:12:21 +04:00
Makefile.in [gdb/build] Fix gdbserver build with -fsanitize=address 2021-01-07 10:37:51 +01:00
mem-break.cc Update copyright year range in all GDB files 2021-01-01 12:12:21 +04:00
mem-break.h Update copyright year range in all GDB files 2021-01-01 12:12:21 +04:00
netbsd-aarch64-low.cc Update copyright year range in all GDB files 2021-01-01 12:12:21 +04:00
netbsd-amd64-low.cc Update copyright year range in all GDB files 2021-01-01 12:12:21 +04:00
netbsd-i386-low.cc Update copyright year range in all GDB files 2021-01-01 12:12:21 +04:00
netbsd-low.cc Update copyright year range in all GDB files 2021-01-01 12:12:21 +04:00
netbsd-low.h Update copyright year range in all GDB files 2021-01-01 12:12:21 +04:00
notif.cc Update copyright year range in all GDB files 2021-01-01 12:12:21 +04:00
notif.h Update copyright year range in all GDB files 2021-01-01 12:12:21 +04:00
proc-service.cc Update copyright year range in all GDB files 2021-01-01 12:12:21 +04:00
proc-service.list Update copyright year range in all GDB files 2021-01-01 12:12:21 +04:00
README
regcache.cc Update copyright year range in all GDB files 2021-01-01 12:12:21 +04:00
regcache.h Update copyright year range in all GDB files 2021-01-01 12:12:21 +04:00
remote-utils.cc Update copyright year range in all GDB files 2021-01-01 12:12:21 +04:00
remote-utils.h Update copyright year range in all GDB files 2021-01-01 12:12:21 +04:00
server.cc Fix a couple vStopped pending ack bugs 2021-02-03 01:14:46 +00:00
server.h Update copyright year range in all GDB files 2021-01-01 12:12:21 +04:00
symbol.cc Update copyright year range in all GDB files 2021-01-01 12:12:21 +04:00
target.cc Update copyright year range in all GDB files 2021-01-01 12:12:21 +04:00
target.h Update copyright year range in all GDB files 2021-01-01 12:12:21 +04:00
tdesc.cc Update copyright year range in all GDB files 2021-01-01 12:12:21 +04:00
tdesc.h Update copyright year range in all GDB files 2021-01-01 12:12:21 +04:00
thread-db.cc Update copyright year range in all GDB files 2021-01-01 12:12:21 +04:00
tracepoint.cc gdb: make some variables static 2021-01-20 20:55:05 -05:00
tracepoint.h Update copyright year range in all GDB files 2021-01-01 12:12:21 +04:00
utils.cc Update copyright year range in all GDB files 2021-01-01 12:12:21 +04:00
utils.h Update copyright year range in all GDB files 2021-01-01 12:12:21 +04:00
win32-i386-low.cc Update copyright year range in all GDB files 2021-01-01 12:12:21 +04:00
win32-low.cc Update copyright year range in all GDB files 2021-01-01 12:12:21 +04:00
win32-low.h Update copyright year range in all GDB files 2021-01-01 12:12:21 +04:00
x86-low.cc Update copyright year range in all GDB files 2021-01-01 12:12:21 +04:00
x86-low.h Update copyright year range in all GDB files 2021-01-01 12:12:21 +04:00
x86-tdesc.h Update copyright year range in all GDB files 2021-01-01 12:12:21 +04:00
xtensa-xtregs.cc Update copyright year range in all GDB files 2021-01-01 12:12:21 +04:00

		   README for GDBserver & GDBreplay
		    by Stu Grossman and Fred Fish

Introduction:

This is GDBserver, a remote server for Un*x-like systems.  It can be used to
control the execution of a program on a target system from a GDB on a different
host.  GDB and GDBserver communicate using the standard remote serial protocol.
They communicate via either a serial line or a TCP connection.

For more information about GDBserver, see the GDB manual:

    https://sourceware.org/gdb/current/onlinedocs/gdb/Remote-Protocol.html

Usage (server (target) side):

First, you need to have a copy of the program you want to debug put onto
the target system.  The program can be stripped to save space if needed, as
GDBserver doesn't care about symbols.  All symbol handling is taken care of by
the GDB running on the host system.

To use the server, you log on to the target system, and run the `gdbserver'
program.  You must tell it (a) how to communicate with GDB, (b) the name of
your program, and (c) its arguments.  The general syntax is:

	target> gdbserver COMM PROGRAM [ARGS ...]

For example, using a serial port, you might say:

	target> gdbserver /dev/com1 emacs foo.txt

This tells GDBserver to debug emacs with an argument of foo.txt, and to
communicate with GDB via /dev/com1.  GDBserver now waits patiently for the
host GDB to communicate with it.

To use a TCP connection, you could say:

	target> gdbserver host:2345 emacs foo.txt

This says pretty much the same thing as the last example, except that we are
going to communicate with the host GDB via TCP.  The `host:2345' argument means
that we are expecting to see a TCP connection to local TCP port 2345.
(Currently, the `host' part is ignored.)  You can choose any number you want for
the port number as long as it does not conflict with any existing TCP ports on
the target system.  This same port number must be used in the host GDB's
`target remote' command, which will be described shortly. Note that if you chose
a port number that conflicts with another service, GDBserver will print an error
message and exit.

On some targets, GDBserver can also attach to running programs.  This is
accomplished via the --attach argument.  The syntax is:

	target> gdbserver --attach COMM PID

PID is the process ID of a currently running process.  It isn't necessary
to point GDBserver at a binary for the running process.

Usage (host side):

You need an unstripped copy of the target program on your host system, since
GDB needs to examine it's symbol tables and such.  Start up GDB as you normally
would, with the target program as the first argument.  (You may need to use the
--baud option if the serial line is running at anything except 9600 baud.)
Ie: `gdb TARGET-PROG', or `gdb --baud BAUD TARGET-PROG'.  After that, the only
new command you need to know about is `target remote'.  It's argument is either
a device name (usually a serial device, like `/dev/ttyb'), or a HOST:PORT
descriptor.  For example:

	(gdb) target remote /dev/ttyb

communicates with the server via serial line /dev/ttyb, and:

	(gdb) target remote the-target:2345

communicates via a TCP connection to port 2345 on host `the-target', where
you previously started up GDBserver with the same port number.  Note that for
TCP connections, you must start up GDBserver prior to using the `target remote'
command, otherwise you may get an error that looks something like
`Connection refused'.

Building GDBserver:

See the `configure.srv` file for the list of host triplets you can build
GDBserver for.

Building GDBserver for your host is very straightforward.  If you build
GDB natively on a host which GDBserver supports, it will be built
automatically when you build GDB.  You can also build just GDBserver:

	% mkdir obj
	% cd obj
	% path-to-toplevel-sources/configure --disable-gdb
	% make all-gdbserver

(If you have a combined binutils+gdb tree, you may want to also
disable other directories when configuring, e.g., binutils, gas, gold,
gprof, and ld.)

If you prefer to cross-compile to your target, then you can also build
GDBserver that way.  For example:

	% export CC=your-cross-compiler
	% path-to-topevel-sources/configure --disable-gdb
	% make all-gdbserver

Using GDBreplay:

A special hacked down version of GDBserver can be used to replay remote
debug log files created by GDB.  Before using the GDB "target" command to
initiate a remote debug session, use "set remotelogfile <filename>" to tell
GDB that you want to make a recording of the serial or tcp session.  Note
that when replaying the session, GDB communicates with GDBreplay via tcp,
regardless of whether the original session was via a serial link or tcp.

Once you are done with the remote debug session, start GDBreplay and
tell it the name of the log file and the host and port number that GDB
should connect to (typically the same as the host running GDB):

	$ gdbreplay logfile host:port

Then start GDB (preferably in a different screen or window) and use the
"target" command to connect to GDBreplay:

	(gdb) target remote host:port

Repeat the same sequence of user commands to GDB that you gave in the
original debug session.  GDB should not be able to tell that it is talking
to GDBreplay rather than a real target, all other things being equal.  Note
that GDBreplay echos the command lines to stderr, as well as the contents of
the packets it sends and receives.  The last command echoed by GDBreplay is
the next command that needs to be typed to GDB to continue the session in
sync with the original session.