[lfs-support] Glibc-2.28 tests

[Hans Malissa]

Hi,

I'm getting unexpected test failures in glibc (8.3-systemd, chapter 6.9).
make check exits with:

...
Summary of test results:
3 FAIL
5823 PASS
31 UNSUPPORTED
17 XFAIL
2 XPASS
make[1]: *** [Makefile:347: tests] Error 1
make[1]: Leaving directory '/usr/src/glibc-2.28'
make: *** [Makefile:9: check] Error 2

The 3 FAILs are:
FAIL: inet/tst-idna_name_classify
FAIL: libio/tst-readline
FAIL: misc/tst-ttyname

The 2 XPASSes are:
XPASS: elf/tst-protected1a
XPASS: elf/tst-protected1b

inet/tst-idna_name_classify and misc/tst-ttyname are mentioned in the book, but not inet/tst-idna_name_classify. 
Is it safe to continue at this point?
Also, what is the best strategy to find out more about a specific error? Google searches mainly lead to archives of test results, but not to an explanation.
Thanks a lot,

Hans

[Ken Moffat]

Post by Hans Malissa
Hi,
I'm getting unexpected test failures in glibc (8.3-systemd, chapter 6.9).
...
3 FAIL
5823 PASS
31 UNSUPPORTED
17 XFAIL
2 XPASS
make[1]: *** [Makefile:347: tests] Error 1
make[1]: Leaving directory '/usr/src/glibc-2.28'
make: *** [Makefile:9: check] Error 2
FAIL: inet/tst-idna_name_classify
FAIL: libio/tst-readline
FAIL: misc/tst-ttyname
XPASS: elf/tst-protected1a
XPASS: elf/tst-protected1b
inet/tst-idna_name_classify and misc/tst-ttyname are mentioned in the book, but not inet/tst-idna_name_classify. 
Is it safe to continue at this point?

With one extra failure, yes. We don't usually note or care about
XPASS, only what actually fails. Having one or two more here is no
big deal. But there is no hard and fast rule - having 20 more is
bad, having 6 more - maybe, maybe not.

FWIW, I only got 1 on the machine I looked at, because one other was
'unsupported' so to an extent the results vary, probably with
micro-architecture and kernel configuration.

Post by Hans Malissa
Also, what is the best strategy to find out more about a specific error? Google searches mainly lead to archives of test results, but not to an explanation.
Thanks a lot,
Hans

Necromancy ? Sometimes an explanation will be hinted at after
reading several results across the first few pages. But other
times, nothing - and to an extent it will depend on what google
knows about your search history, but more, these days, to what other
people are commonly searching for.

ĸen
--
Is it about a bicycle ?
--
http://lists.linuxfromscratch.org/listinfo/lfs-support
FAQ: http://www.linuxfromscratch.org/blfs/faq.html
Unsubscribe: See the above information page

Do not top post on this list.

A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?
A: Top-posting.
Q: What is the most annoying thing in e-mail?

http://en.wikipedia.org/wiki/Postin

[Hans Malissa]

On Oct 22, 2018, at 09:18 AM, Ken Moffat <***@ntlworld.com> wrote:
On Mon, Oct 22, 2018 at 02:44:23PM +0000, Hans Malissa wrote:
Hi,

I'm getting unexpected test failures in glibc (8.3-systemd, chapter 6.9).
make check exits with:

...
Summary of test results:
3 FAIL
5823 PASS
31 UNSUPPORTED
17 XFAIL
2 XPASS
make[1]: *** [Makefile:347: tests] Error 1
make[1]: Leaving directory '/usr/src/glibc-2.28'
make: *** [Makefile:9: check] Error 2

The 3 FAILs are:
FAIL: inet/tst-idna_name_classify
FAIL: libio/tst-readline
FAIL: misc/tst-ttyname

The 2 XPASSes are:
XPASS: elf/tst-protected1a
XPASS: elf/tst-protected1b

inet/tst-idna_name_classify and misc/tst-ttyname are mentioned in the book, but not inet/tst-idna_name_classify. 
Is it safe to continue at this point?


With one extra failure, yes. We don't usually note or care about
XPASS, only what actually fails. Having one or two more here is no
big deal. But there is no hard and fast rule - having 20 more is
bad, having 6 more - maybe, maybe not.

FWIW, I only got 1 on the machine I looked at, because one other was
'unsupported' so to an extent the results vary, probably with
micro-architecture and kernel configuration.

Thanks. I will continue with my build as usual. I've backed up $LFS/tools anyway in chapter 5.36, so if everything goes wrong I could just go back to this point.
 


Also, what is the best strategy to find out more about a specific error? Google searches mainly lead to archives of test results, but not to an explanation.
Thanks a lot,

Hans

Necromancy ? Sometimes an explanation will be hinted at after
reading several results across the first few pages. But other
times, nothing - and to an extent it will depend on what google
knows about your search history, but more, these days, to what other
people are commonly searching for.

Äžen

[Ken Moffat]

Post by Hans Malissa
So are these tests documented in detail somewhere? The glibc website doesn't give explanation of the individual test cases as far as I can see.
Thanks a lot,
Hans 

They were probably discussed (at a guess, on the libc-alpha list)
before being committed. Those are the type of google matches I was
referring to.

But in general, I am not aware of any documentation explaining what
a test if doing. That is true for most projects, not just glibc.
Often, the test script may explain what it is doing - but for us
mere mortals the details will usually be hard to understand because
we lack context, e.g. about how the test actually runs. Some are
fairly simple, others either delete their output before the tests
have finished, or require specific knowledge to understand what they
are doing.

ĸen
--
Is it about a bicycle ?
--
http://lists.linuxfromscratch.org/listinfo/lfs-support
FAQ: http://www.linuxfromscratch.org/blfs/faq.html
Unsubscribe: See the above information page

Do not top post on this list.

A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?
A: Top-posting.
Q: What is the most annoying thing in e-mail?

http://en.wikipedia.org/wi

[Bruce Dubbs]

Post by Ken Moffat

Post by Hans Malissa
So are these tests documented in detail somewhere? The glibc website doesn't give explanation of the individual test cases as far as I can see.
Thanks a lot,
Hans

They were probably discussed (at a guess, on the libc-alpha list)
before being committed. Those are the type of google matches I was
referring to.
But in general, I am not aware of any documentation explaining what
a test if doing. That is true for most projects, not just glibc.
Often, the test script may explain what it is doing - but for us
mere mortals the details will usually be hard to understand because
we lack context, e.g. about how the test actually runs. Some are
fairly simple, others either delete their output before the tests
have finished, or require specific knowledge to understand what they
are doing.

Let me add that we build gcc in a partial environment. There are a lot
of factors that can cause a test failure including kernel configuration,
support libraries, etc. In some cases we've seen test failures because
a library is too new for the assumptions made in the test.

-- Bruce
--
http://lists.linuxfromscratch.org/listinfo/lfs-support
FAQ: http://www.linuxfromscratch.org/blfs/faq.html
Unsubscribe: See the above information page

Do not top post on this list.

A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?
A: Top-posting.
Q: What is the most annoying thing in e-mail?

http

[Hans Malissa]

On Oct 22, 2018, at 01:59 PM, Bruce Dubbs <***@gmail.com> wrote:
On 10/22/2018 01:08 PM, Ken Moffat wrote:
On Mon, Oct 22, 2018 at 05:27:59PM +0000, Hans Malissa wrote:

So are these tests documented in detail somewhere? The glibc website doesn't give explanation of the individual test cases as far as I can see.
Thanks a lot,

Hans

They were probably discussed (at a guess, on the libc-alpha list)
before being committed. Those are the type of google matches I was
referring to.

But in general, I am not aware of any documentation explaining what
a test if doing. That is true for most projects, not just glibc.
Often, the test script may explain what it is doing - but for us
mere mortals the details will usually be hard to understand because
we lack context, e.g. about how the test actually runs. Some are
fairly simple, others either delete their output before the tests
have finished, or require specific knowledge to understand what they
are doing.

Let me add that we build gcc in a partial environment. There are a lot
of factors that can cause a test failure including kernel configuration,
support libraries, etc. In some cases we've seen test failures because
a library is too new for the assumptions made in the test.

-- Bruce

Thanks for the responses, this is really helpful.
The book makes it quite clear that the test suites are critically important, at least for some packages. But from this discussion it seems as if things may be okay even with "a few" failed tests. And I guess there's a fine line between "just a few failed tests" and "a failed package build".
Thanks everyone for your help!

Hans