binutils-gdb/gdb/dwarf-index-cache.c
Simon Marchi 87d6a7aa93 Add DWARF index cache
New in v3:

- Remove things related to the dwarf-5 format.
- Fix compilation on mingw (scoped_mmap.c).

GDB can generate indexes for DWARF debug information, which, when
integrated in the original binary, can speed up loading object files.
This can be done using the gdb-add-index script or directly by the
linker itself.  However, not many people know about this.  And even
among those who do, because it requires additional steps, I don't know a
lot of people who actually go through that trouble.

To help make using the DWARF index more transparent, this patch
introduces a DWARF index cache.  When enabled, loading an index-less
binary in GDB will automatically save an index file in ~/.cache/gdb.
When loading that same object file again, the index file will be looked
up and used to load the DWARF index.  You therefore get the benefit of
the DWARF index without having to do additional manual steps or
modifying your build system.  When an index section is already present
in the file, GDB will prefer that one over looking up the cache.

When doing my edit-compile-debug cycle, I often debug multiple times the
same build, so the cache helps reducing the load time of the debug
sessions after the first one.

- The saved index file is exactly the same as the output of the "save
  gdb-index" command.  It is therefore the exact same content that would
  be found in the .gdb_index or .debug_names section.  We just leave it
  as a standalone file instead of merging it in the binary.

- The cache is just a directory with files named after the object
  file's build-id.  It is not possible to save/load the index for an
  object file without build-id in the cache.

- The cache uses the gdb index format.  The problem with the dwarf-5
  format is that we can generate an addendum to the .debug_str section
  that you're supposed to integrate to the original binary.  This
  complicates a little bit loading the data from the cached index files,
  so I would leave this for later.

- The size taken up by ~/.cache/gdb is not limited.  I was thinking we
  could add configurable limit (like ccache does), but that would come
  after.  Also, maybe a command to flush the cache.

- The cache is disabled by default.  I think once it's been out there
  and tested for a while, it could be turned on by default, so that
  everybody can enjoy it.

- The code was made to follow the XDG specification: if the
  XDG_CACHE_HOME environment variable, it is used, otherwise it falls
  back to ~/.cache/gdb.  It is possible to change it using "set
  index-cache directory".  On other OSes than GNU/Linux, ~/.cache may
  not be the best place to put such data.  On macOS it should probably
  default to ~/Library/Caches/...  On Windows, %LocalAppData%/...  I
  don't intend to do this part, but further patches are welcome.

- I think that we need to be careful that multiple instances of GDB
  don't interfere with each other (not far fetched at all if you run GDB
  in some automated script) and the cache is always coherent (either the
  file is not found, or it is found and entirely valid).  Writing the
  file directly to its final location seems like a recipe for failure.
  One GDB could read a file in the index while it is being written by
  another GDB.  To mitigate this, I made write_psymtabs_to_index write
  to temporary files and rename them once it's done.  Two GDB instances
  writing the index for the same file should not step on each other's
  toes (the last file to be renamed will stay).  A GDB looking up a file
  will only see a complete file or no file.  Also, if GDB crashes while
  generating the index file, it will leave a work-in-progress file, but
  it won't be picked up by other instances looking up in the cache.

gdb/ChangeLog:

	* common/pathstuff.h (get_standard_cache_dir): New.
	* common/pathstuff.c (get_standard_cache_dir): New.
	* build-id.h (build_id_to_string): New.
	* dwarf-index-common.h (INDEX4_SUFFIX, INDEX5_SUFFIX,
	DEBUG_STR_SUFFIX): Move to here.
	* dwarf-index-write.c (INDEX4_SUFFIX, INDEX5_SUFFIX,
	DEBUG_STR_SUFFIX): Move from there.
	(write_psymtabs_to_index): Make non-static, add basename
	parameter.  Write to temporary files, rename when done.
	(save_gdb_index_command): Adjust call to
	write_psymtabs_to_index.
	* dwarf2read.h (dwarf2_per_objfile) <index_cache_res>: New
	field.
	* dwarf2read.c (dwz_file) <index_cache_res>: New field.
	(get_gdb_index_contents_from_cache): New.
	(get_gdb_index_contents_from_cache_dwz): New.
	(dwarf2_initialize_objfile): Read index from cache.
	(dwarf2_build_psymtabs): Save to index.
	* dwarf-index-cache.h: New file.
	* dwarf-index-cache.c: New file.
	* dwarf-index-write.h: New file.

gdb/testsuite/ChangeLog:

	* boards/index-cache-gdb.exp: New file.
	* gdb.dwarf2/index-cache.exp: New file.
	* gdb.dwarf2/index-cache.c: New file.
	* gdb.base/maint.exp: Check if we are using the index cache.
2018-08-07 18:14:20 -04:00

464 lines
13 KiB
C

/* Caching of GDB/DWARF index files.
Copyright (C) 1994-2018 Free Software Foundation, Inc.
This file is part of GDB.
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation; either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see <http://www.gnu.org/licenses/>. */
#include "defs.h"
#include "dwarf-index-cache.h"
#include "build-id.h"
#include "cli/cli-cmds.h"
#include "command.h"
#include "common/scoped_mmap.h"
#include "common/pathstuff.h"
#include "dwarf-index-write.h"
#include "dwarf2read.h"
#include "objfiles.h"
#include "selftest.h"
#include <string>
#include <stdlib.h>
/* When set to 1, show debug messages about the index cache. */
static int debug_index_cache = 0;
/* The index cache directory, used for "set/show index-cache directory". */
static char *index_cache_directory = NULL;
/* See dwarf-index.cache.h. */
index_cache global_index_cache;
/* set/show index-cache commands. */
static cmd_list_element *set_index_cache_prefix_list;
static cmd_list_element *show_index_cache_prefix_list;
/* A cheap (as in low-quality) recursive mkdir. Try to create all the parents
directories up to DIR and DIR itself. Stop if we hit an error along the way.
There is no attempt to remove created directories in case of failure. */
static void
mkdir_recursive (const char *dir)
{
gdb::unique_xmalloc_ptr<char> holder (xstrdup (dir));
char * const start = holder.get ();
char *component_start = start;
char *component_end = start;
while (1)
{
/* Find the beginning of the next component. */
while (*component_start == '/')
component_start++;
/* Are we done? */
if (*component_start == '\0')
return;
/* Find the slash or null-terminator after this component. */
component_end = component_start;
while (*component_end != '/' && *component_end != '\0')
component_end++;
/* Temporarily replace the slash with a null terminator, so we can create
the directory up to this component. */
char saved_char = *component_end;
*component_end = '\0';
/* If we get EEXIST and the existing path is a directory, then we're
happy. If it exists, but it's a regular file and this is not the last
component, we'll fail at the next component. If this is the last
component, the caller will fail with ENOTDIR when trying to
open/create a file under that path. */
if (mkdir (start, 0700) != 0)
if (errno != EEXIST)
return;
/* Restore the overwritten char. */
*component_end = saved_char;
component_start = component_end;
}
}
/* Default destructor of index_cache_resource. */
index_cache_resource::~index_cache_resource () = default;
/* See dwarf-index-cache.h. */
void
index_cache::set_directory (std::string dir)
{
gdb_assert (!dir.empty ());
m_dir = std::move (dir);
if (debug_index_cache)
printf_unfiltered ("index cache: now using directory %s\n", m_dir.c_str ());
}
/* See dwarf-index-cache.h. */
void
index_cache::enable ()
{
if (debug_index_cache)
printf_unfiltered ("index cache: enabling (%s)\n", m_dir.c_str ());
m_enabled = true;
}
/* See dwarf-index-cache.h. */
void
index_cache::disable ()
{
if (debug_index_cache)
printf_unfiltered ("index cache: disabling\n");
m_enabled = false;
}
/* See dwarf-index-cache.h. */
void
index_cache::store (struct dwarf2_per_objfile *dwarf2_per_objfile)
{
objfile *obj = dwarf2_per_objfile->objfile;
if (!enabled ())
return;
const bfd_build_id *build_id = build_id_bfd_get (obj->obfd);
if (build_id == nullptr)
{
if (debug_index_cache)
printf_unfiltered ("index cache: objfile %s has no build id\n",
objfile_name (obj));
return;
}
if (m_dir.empty ())
{
warning (_("The index cache directory name is empty, skipping store."));
return;
}
std::string build_id_str = build_id_to_string (build_id);
TRY
{
/* Try to create the containing directory. */
mkdir_recursive (m_dir.c_str ());
if (debug_index_cache)
printf_unfiltered ("index cache: writing index cache for objfile %s\n",
objfile_name (obj));
/* Write the index itself to the directory, using the build id as the
filename. */
write_psymtabs_to_index (dwarf2_per_objfile, m_dir.c_str (),
build_id_str.c_str (), dw_index_kind::GDB_INDEX);
}
CATCH (except, RETURN_MASK_ERROR)
{
if (debug_index_cache)
printf_unfiltered ("index cache: couldn't store index cache for objfile "
"%s: %s", objfile_name (obj), except.message);
}
END_CATCH
}
#if HAVE_SYS_MMAN_H
/* Hold the resources for an mmapped index file. */
struct index_cache_resource_mmap final : public index_cache_resource
{
/* Try to mmap FILENAME. Throw an exception on failure, including if the
file doesn't exist. */
index_cache_resource_mmap (const char *filename)
: mapping (mmap_file (filename))
{}
scoped_mmap mapping;
};
/* See dwarf-index-cache.h. */
gdb::array_view<const gdb_byte>
index_cache::lookup_gdb_index (const bfd_build_id *build_id,
std::unique_ptr<index_cache_resource> *resource)
{
if (!enabled ())
return {};
if (m_dir.empty ())
{
warning (_("The index cache directory name is empty, skipping cache "
"lookup."));
return {};
}
/* Compute where we would expect a gdb index file for this build id to be. */
std::string filename = make_index_filename (build_id, INDEX4_SUFFIX);
TRY
{
if (debug_index_cache)
printf_unfiltered ("index cache: trying to read %s\n",
filename.c_str ());
/* Try to map that file. */
index_cache_resource_mmap *mmap_resource
= new index_cache_resource_mmap (filename.c_str ());
/* Yay, it worked! Hand the resource to the caller. */
resource->reset (mmap_resource);
return gdb::array_view<const gdb_byte>
((const gdb_byte *) mmap_resource->mapping.get (),
mmap_resource->mapping.size ());
}
CATCH (except, RETURN_MASK_ERROR)
{
if (debug_index_cache)
printf_unfiltered ("index cache: couldn't read %s: %s\n",
filename.c_str (), except.message);
}
END_CATCH
return {};
}
#else /* !HAVE_SYS_MMAN_H */
/* See dwarf-index-cache.h. This is a no-op on unsupported systems. */
gdb::array_view<const gdb_byte>
index_cache::lookup_gdb_index (const bfd_build_id *build_id,
std::unique_ptr<index_cache_resource> *resource)
{
return {};
}
#endif
/* See dwarf-index-cache.h. */
std::string
index_cache::make_index_filename (const bfd_build_id *build_id,
const char *suffix) const
{
std::string build_id_str = build_id_to_string (build_id);
return m_dir + SLASH_STRING + build_id_str + suffix;
}
/* "set index-cache" handler. */
static void
set_index_cache_command (const char *arg, int from_tty)
{
printf_unfiltered (_("\
Missing arguments. See \"help set index-cache\" for help.\n"));
}
/* True when we are executing "show index-cache". This is used to improve the
printout a little bit. */
static bool in_show_index_cache_command = false;
/* "show index-cache" handler. */
static void
show_index_cache_command (const char *arg, int from_tty)
{
/* Note that we are executing "show index-cache". */
auto restore_flag = make_scoped_restore (&in_show_index_cache_command, true);
/* Call all "show index-cache" subcommands. */
cmd_show_list (show_index_cache_prefix_list, from_tty, "");
printf_unfiltered ("\n");
printf_unfiltered
(_("The index cache is currently %s.\n"),
global_index_cache.enabled () ? _("enabled") : _("disabled"));
}
/* "set index-cache on" handler. */
static void
set_index_cache_on_command (const char *arg, int from_tty)
{
global_index_cache.enable ();
}
/* "set index-cache off" handler. */
static void
set_index_cache_off_command (const char *arg, int from_tty)
{
global_index_cache.disable ();
}
/* "set index-cache directory" handler. */
static void
set_index_cache_directory_command (const char *arg, int from_tty,
cmd_list_element *element)
{
/* Make sure the index cache directory is absolute and tilde-expanded. */
gdb::unique_xmalloc_ptr<char> abs (gdb_abspath (index_cache_directory));
xfree (index_cache_directory);
index_cache_directory = abs.release ();
global_index_cache.set_directory (index_cache_directory);
}
/* "show index-cache stats" handler. */
static void
show_index_cache_stats_command (const char *arg, int from_tty)
{
const char *indent = "";
/* If this command is invoked through "show index-cache", make the display a
bit nicer. */
if (in_show_index_cache_command)
{
indent = " ";
printf_unfiltered ("\n");
}
printf_unfiltered (_("%s Cache hits (this session): %u\n"),
indent, global_index_cache.n_hits ());
printf_unfiltered (_("%sCache misses (this session): %u\n"),
indent, global_index_cache.n_misses ());
}
#if GDB_SELF_TEST && defined (HAVE_MKDTEMP)
namespace selftests
{
/* Try to create DIR using mkdir_recursive and make sure it exists. */
static bool
create_dir_and_check (const char *dir)
{
mkdir_recursive (dir);
struct stat st;
if (stat (dir, &st) != 0)
perror_with_name ("stat");
return (st.st_mode & S_IFDIR) != 0;
}
/* Test mkdir_recursive. */
static void
test_mkdir_recursive ()
{
char base[] = "/tmp/gdb-selftests-XXXXXX";
if (mkdtemp (base) == NULL)
perror_with_name ("mkdtemp");
/* Try not to leave leftover directories. */
struct cleanup_dirs {
cleanup_dirs (const char *base)
: m_base (base)
{}
~cleanup_dirs () {
rmdir (string_printf ("%s/a/b/c/d/e", m_base).c_str ());
rmdir (string_printf ("%s/a/b/c/d", m_base).c_str ());
rmdir (string_printf ("%s/a/b/c", m_base).c_str ());
rmdir (string_printf ("%s/a/b", m_base).c_str ());
rmdir (string_printf ("%s/a", m_base).c_str ());
rmdir (m_base);
}
private:
const char *m_base;
} cleanup_dirs (base);
std::string dir = string_printf ("%s/a/b", base);
SELF_CHECK (create_dir_and_check (dir.c_str ()));
dir = string_printf ("%s/a/b/c//d/e/", base);
SELF_CHECK (create_dir_and_check (dir.c_str ()));
}
}
#endif /* GDB_SELF_TEST && defined (HAVE_MKDTEMP) */
void
_initialize_index_cache ()
{
/* Set the default index cache directory. */
std::string cache_dir = get_standard_cache_dir ();
if (!cache_dir.empty ())
{
index_cache_directory = xstrdup (cache_dir.c_str ());
global_index_cache.set_directory (std::move (cache_dir));
}
else
warning (_("Couldn't determine a path for the index cache directory."));
/* set index-cache */
add_prefix_cmd ("index-cache", class_files, set_index_cache_command,
_("Set index-cache options"), &set_index_cache_prefix_list,
"set index-cache ", false, &setlist);
/* show index-cache */
add_prefix_cmd ("index-cache", class_files, show_index_cache_command,
_("Show index-cache options"), &show_index_cache_prefix_list,
"show index-cache ", false, &showlist);
/* set index-cache on */
add_cmd ("on", class_files, set_index_cache_on_command,
_("Enable the index cache."), &set_index_cache_prefix_list);
/* set index-cache off */
add_cmd ("off", class_files, set_index_cache_off_command,
_("Disable the index cache."), &set_index_cache_prefix_list);
/* set index-cache directory */
add_setshow_filename_cmd ("directory", class_files, &index_cache_directory,
_("Set the directory of the index cache."),
_("Show the directory of the index cache."),
NULL,
set_index_cache_directory_command, NULL,
&set_index_cache_prefix_list,
&show_index_cache_prefix_list);
/* show index-cache stats */
add_cmd ("stats", class_files, show_index_cache_stats_command,
_("Show some stats about the index cache."),
&show_index_cache_prefix_list);
/* set debug index-cache */
add_setshow_boolean_cmd ("index-cache", class_maintenance,
&debug_index_cache,
_("Set display of index-cache debug messages."),
_("Show display of index-cache debug messages."),
_("\
When non-zero, debugging output for the index cache is displayed."),
NULL, NULL,
&setdebuglist, &showdebuglist);
#if GDB_SELF_TEST && defined (HAVE_MKDTEMP)
selftests::register_test ("mkdir_recursive", selftests::test_mkdir_recursive);
#endif
}