Commit Graph

5 Commits

Author SHA1 Message Date
Joseph Myers
5675961d15 conformtest: Support xfail markers on individual assertions.
Various conformtest tests fail because of known issues, filed in
Bugzilla, that are hard to fix (requiring new features, kernel
cooperation or involving changing types in ways that may involve care
around padding when interfacing to the kernel).  Such an issue has the
effect of making the whole test for the (standard, header) pair fail,
so hiding any other issues with that header for that standard
(possibly regressions or architecture-specific issues).

This patch adds a mechanism for individual conformtest test
expectations to start with xfail-, meaning that a failure of that
particular assertion does not cause the whole test to fail at the
makefile level and so failure at the makefile level can be used to
detect other issues that are likely to be easier to fix.  This is
similar to the whitelisted symbols in the linknamespace tests, or the
marking of particular libm tests as allowing spurious or missing
exceptions, for example.  The bugs filed in Bugzilla should still be
fixed, and the xfail- markers removed at that point, but xfail-
renders the tests more useful until that happens.

Note that there is no way to add such a marker for the assertion that
a header uses only symbols in the namespace of symbols it's meant to
use.  I don't think there's any need for a way to xfail those
namespace tests other than xfailing the whole (standard, header) pair
at the makefile level, since they are generally straightforward to fix
(add appropriate conditionals on the problem definitions).

The xfails in this patch do not necessarily cover all cases of
hard-to-fix header bugs filed in Bugzilla that currently show up in
conformtest failures; there may be more yet to add for existing open
bugs.

Tested for x86_64 and x86.

	* conform/conformtest.pl ($xerrors): New variable.
	(note_error): New function.
	(compiletest): New argument $xfail.  Use not_error.
	(runtest): Likewise.
	(top level): Handle xfail- lines.  Update calls to compiletest and
	runtest.  Handle xfail- and optional- in headers listed with
	allow-header.
	* conform/data/fcntl.h-data (O_TTY_INIT): Use xfail-.
	(O_EXEC): Likewise.
	(O_SEARCH): Likewise.
	* conform/data/stropts.h-data (ioctl): Likewise.
	* conform/data/sys/ipc.h-data (ipc_perm.mode): Likewise.
	* conform/data/sys/sem.h-data (semid_ds.sem_nsems): Likewise.
	* conform/data/sys/socket.h-data (msghdr.msg_iovlen): Likewise.
	(msghdr.msg_controllen): Likewise.
	(cmsghdr.cmsg_len): Likewise.
	* conform/data/utmpx.h-data (utmpx.ut_tv): Likewise.
	* conform/Makefile (test-xfail-XPG3/sys/ipc.h/conform): Remove
	variable.
	(test-xfail-XPG3/sys/sem.h/conform): Likewise.
	(test-xfail-XPG4/stropts.h/conform): Likewise.
	(test-xfail-XPG4/sys/ipc.h/conform): Likewise.
	(test-xfail-XPG4/sys/sem.h/conform): Likewise.
	(test-xfail-XPG4/sys/socket.h/conform): Likewise.
	(test-xfail-UNIX98/stropts.h/conform): Likewise.
	(test-xfail-UNIX98/sys/ipc.h/conform): Likewise.
	(test-xfail-UNIX98/sys/sem.h/conform): Likewise.
	(test-xfail-UNIX98/sys/socket.h/conform): Likewise.
	(test-xfail-XOPEN2K/stropts.h/conform): Likewise.
	(test-xfail-XOPEN2K/sys/ipc.h/conform): Likewise.
	(test-xfail-XOPEN2K/sys/sem.h/conform): Likewise.
	(test-xfail-XOPEN2K/sys/socket.h/conform): Likewise.
	(test-xfail-XOPEN2K/utmpx.h/conform): Likewise.
	(test-xfail-POSIX2008/fcntl.h/conform): Likewise.
	(test-xfail-POSIX2008/stropts.h/conform): Likewise.
	(test-xfail-XOPEN2K8/fcntl.h/conform): Likewise.
	(test-xfail-XOPEN2K8/stropts.h/conform): Likewise.
	(test-xfail-XOPEN2K8/sys/ipc.h/conform): Likewise.
	(test-xfail-XOPEN2K8/sys/sem.h/conform): Likewise.
	(test-xfail-XOPEN2K8/sys/socket.h/conform): Likewise.
2015-06-19 20:05:41 +00:00
Joseph Myers
2ea18461d3 conformtest: correct set of standards for which some headers are tested.
Reviewing the sets of headers for which conformtest has expectations
for various standards showed up cases where headers had expectations
for standards not including those headers, or were missing
expectations for a standard that did include the header.  This patch
fixes easy cases of this (largely disabling tests for standards for
which they are inapplicable).  Tested x86_64.

	* conform/data/aio.h-data [XPG3 || XPG4]: Disable whole file.
	* conform/data/arpa/inet.h-data [XPG3]: Likewise.
	* conform/data/dlfcn.h-data [XPG3 || XPG4]: Likewise.
	* conform/data/fmtmsg.h-data [XPG3]: Likewise.
	* conform/data/libgen.h-data [XPG3]: Likewise.
	* conform/data/mqueue.h-data [XPG3 || XPG4]: Likewise.
	* conform/data/ndbm.h-data [XPG3]: Likewise.
	* conform/data/net/if.h-data [XPG3 || XPG4 || UNIX98]: Likewise.
	* conform/data/netdb.h-data [XPG3]: Likewise.
	* conform/data/netinet/in.h-data [XPG3]: Likewise.
	* conform/data/poll.h-data [XPG3]: Likewise.
	* conform/data/spawn.h-data [XPG3 || XPG4 || UNIX98]: Likewise.
	* conform/data/strings.h-data [XPG3]: Likewise.
	* conform/data/stropts.h-data [XPG3]: Likewise.
	* conform/data/sys/mman.h-data [XPG3]: Likewise.
	* conform/data/sys/resource.h-data [XPG3]: Likewise.
	* conform/data/sys/select.h-data [XPG3 || XPG4 || UNIX98]:
	Likewise.
	* conform/data/sys/statvfs.h-data [XPG3]: Likewise.
	* conform/data/sys/time.h-data [XPG3]: Likewise.
	* conform/data/sys/timeb.h-data [XPG3]: Likewise.
	* conform/data/sys/uio.h-data [XPG3]: Likewise.
	* conform/data/sys/un.h-data [XPG3]: Likewise.
	* conform/data/syslog.h-data [XPG3]: Likewise.
	* conform/data/ucontext.h-data [XPG3]: Likewise.
	* conform/data/utmpx.h-data [XPG3]: Likewise.
	* conform/data/varargs.h-data [UNIX98]: Enable file.
2014-03-06 17:42:03 +00:00
Ulrich Drepper
3134156779 First steps to get conformtest fully working 2012-02-25 23:18:39 -05:00
Ulrich Drepper
f095bb7204 Add support for XPG7 testing.
The header conformance testing code needed extending for XPG7.  This
exposed a few bugs in the headers.  There are more changes to come.
2010-01-09 10:56:41 -08:00
Ulrich Drepper
0ed99ce46d Update.
2000-02-27  Ulrich Drepper  <drepper@redhat.com>

	* conform/conformtest.pl (@headers): Add wordexp.h, wctype.h, wchar.h,
	varargs.h, utmpx.h, utime.h, unistd.h, ulimit.h, ucontext.h, time.h,
	termios.h, tar.h, sys/wait.h, sys/uio.h, sys/types.h, sys/times.h,
	sys/timeb.h, sys/time.h, sys/statvfs.h, sys/stat.h, sys/shm.h,
	sys/sem.h, and sys/resource.h.
	Implement handling of allow-header.
	* conform/data/fcntl.h-data: Correct various bugs.
	* conform/data/fnmatch.h-data: Likewise.
	* conform/data/math.h-data: Likewise.
	* conform/data/ndbm.h-data: Likewise.
	* conform/data/nl_types.h-data: Likewise.
	* conform/data/poll.h-data: Likewise.
	* conform/data/pthread.h-data: Likewise.
	* conform/data/semaphore.h-data: Likewise.
	* conform/data/setjmp.h-data: Likewise.
	* conform/data/signal.h-data: Likewise.
	* conform/data/stdarg.h-data: Likewise.
	* conform/data/stddef.h-data: Likewise.
	* conform/data/stdio.h-data: Likewise.
	* conform/data/stdlib.h-data: Likewise.
	* conform/data/string.h-data: Likewise.
	* conform/data/strings.h-data: Likewise.
	* conform/data/stropts.h-data: Likewise.
	* conform/data/syslog.h-data: Likewise.
	* conform/data/sys/ipc.h-data: Likewise.
	* conform/data/sys/mman.h-data: Likewise.
	* conform/data/sys/msg.h-data: Likewise.
	* conform/data/tar.h-data: New file.
	* conform/data/termios.h-data: New file.
	* conform/data/time.h-data: New file.
	* conform/data/ucontext.h-data: New file.
	* conform/data/ulimit.h-data: New file.
	* conform/data/unistd.h-data: New file.
	* conform/data/utime.h-data: New file.
	* conform/data/utmpx.h-data: New file.
	* conform/data/varargs.h-data: New file.
	* conform/data/wchar.h-data: New file.
	* conform/data/wctype.h-data: New file.
	* conform/data/wordexp.h-data: New file.
	* conform/data/sys/resource.h-data: New file.
	* conform/data/sys/sem.h-data: New file.
	* conform/data/sys/shm.h-data: New file.
	* conform/data/sys/stat.h-data: New file.
	* conform/data/sys/statvfs.h-data: New file.
	* conform/data/sys/time.h-data: New file.
	* conform/data/sys/timeb.h-data: New file.
	* conform/data/sys/times.h-data: New file.
	* conform/data/sys/types.h-data: New file.
	* conform/data/sys/uio.h-data: New file.
	* conform/data/sys/wait.h-data: New file.

	* grp/grp.h (getgrent_r): Declare only if __USE_GNU.

	* include/sys/time.h (__itimes): Fix type of second parameter.

	* io/sys/stat.h (S_IFLNK): Define also if __USE_XOPEN.
	(S_ISVTX): Likewise.

	* io/sys/statvfs.h: Define fsblkcnt_t and fsfilcnt_t.

	* locale/langindo.h: Make ERA_YEAR, INT_CURR_SYMBOL, CURRENCY_SYMBOL,
	MON_DECIMAL_POINT, MON_THOUSANDS_SEP, MON_GROUPING, POSITIVE_SIGN,
	NEGATIVE_SIGN, INT_FRAC_DIGITS, FRAC_DIGITS, P_CS_PRECEDES,
	P_SEP_BY_SPACE, N_CS_PRECEDES, N_SEP_BY_SPACE, P_SIGN_POSN,
	N_SIGN_POSN, DECIMAL_POINT, THOUSANDS_SEP, GROUPING, YESEXPR,
	NOEXPR, YESSTR, and NOSTR only available if __USE_GNU.

	* math/bits/mathcalls.h: Don't declare tgamma and scalbn if only
	__USE_XOPEN.

	* posix/unistd.h: Don't define ptrdiff_t.
	(sbrk): Correct type of parameter.
	* sysdeps/generic/sbrk.c: Likewise.
	* sysdeps/mach/hurd/sbrk.c: Likewise.

	* posix/sys/types.h: Allow id_t, suseconds_t, fsblkcnt_t, and
	fsfilcnt_t being defined elsewhere.

	* resource/sys/resource.h: Define id_t.
	If __USE_XOPEN use int for type of first parameter if getrlimit,
	getrlimit64, setrlimit, setrlimit64, getrusage, getpriority, and
	setpriority.
	* sysdeps/generic/getpriority.c: Fix type of second parameter.
	* sysdeps/generic/setpriority.c: Likewise.
	* sysdeps/mach/hurd/getpriority.c: Likewise.
	* sysdeps/mach/hurd/setpriority.c: Likewise.
	* sysdeps/unix/sysv/irix4/getpriority.c: Likewise.
	* sysdeps/unix/sysv/irix4/setpriority.c: Likewise.
	* sysdeps/unix/sysv/linux/getpriority.c: Likewise.

	* sysdeps/generic/utimes.c: Fix parameter of third parameter.
	* sysdeps/mach/hurd/utimes.c: Likewise.
	* sysdeps/posix/utimes.c: Likewise.

	* sysdeps/generic/bits/shm.h: Define shmatt_t type and use it in
	struct shmid_ds definition.
	* sysdeps/gnu/bits/shm.h: Likewise.
	* sysdeps/unix/sysv/linux/alpha/bits/shm.h: Likewise.
	* sysdeps/unix/sysv/linux/bits/shm.h: Likewise.
	* sysdeps/unix/sysv/linux/sparc/bits/shm.h: Likewise.

	* sysvipc/sys/shm.h: Define time_t.

	* sysvipc/sys/sem.h (struct sembuf): Make sem_num field unsigned.

	* sysdeps/gnu/bits/sem.h: Rename anonymous type struct sem_queue and
	struct sem_undo to struct __sem_queue and struct __sem_undo resp.

	* time/time.h: Define suseconds_t if not already done.

	* sysdeps/unix/sysv/linux/bits/statvfs.h: Rename f_spare to __f_spare.
	* sysdeps/unix/sysv/linux/fstatvfs64.c: Likewise.
	* sysdeps/unix/sysv/linux/statvfs64.c: Likewise.
	* sysdeps/unix/sysv/linux/internal_statvfs.c: Likewise.

	* intl/bindtextdom.c: Use rwlock to avoid surprising results in
	multithreaded applications.
	* intl/dcigettext.c: Likewise.
	* intl/textdomain.c: Likewise.
	* intl/*.c: Update copyright.
	* sysdeps/generic/bits/libc-lock.h: Define __libc_rwlock_* stubs.

2000-02-27  Andreas Jaeger  <aj@suse.de>

	* sysdeps/unix/sysv/linux/configure.in: Treat also "/usr/" as
	prefix especially.

2000-02-27  Ulrich Drepper  <drepper@redhat.com>

	* po/de.po: Update from translation team.
	* po/ko.po: Likewise.
2000-02-28 05:08:01 +00:00