openssl/ssl/statem
Matt Caswell 47e81a1bfa Pass an EVP_PKEY for SSL_SECOP_TMP_DH in the security callback
The security operation SSL_SECOP_TMP_DH is defined to take an EVP_PKEY
in the "other" parameter:

 /* Temporary DH key */
 # define SSL_SECOP_TMP_DH                (7 | SSL_SECOP_OTHER_PKEY)

In most places this is what is passed. All these places occur server side.
However there is one client side call of this security operation and it
passes a DH object instead. This is incorrect according to the
definition of SSL_SECOP_TMP_DH, and is inconsistent with all of the other
locations.

Our own default security callback, and the debug callback in the apps,
never look at this value and therefore this issue was never noticed
previously. In theory a client side application could be relying on this
behaviour and could be broken by this change. This is probably fairly
unlikely but can't be ruled out.

Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
Reviewed-by: Ben Kaduk <kaduk@mit.edu>
(Merged from https://github.com/openssl/openssl/pull/13136)
2020-10-16 14:58:44 +01:00
..
extensions_clnt.c [ssl] Support ssl_decapsulate on client side 2020-10-14 18:42:59 +03:00
extensions_cust.c
extensions_srvr.c [ssl] Support ssl_encapsulate on server side 2020-10-14 18:42:59 +03:00
extensions.c Run the withlibctx.pl script 2020-10-01 09:25:20 +01:00
README.md Fix many MarkDown issues in {NOTES*,README*,HACKING,LICENSE}.md files 2020-07-05 11:29:43 +02:00
statem_clnt.c Pass an EVP_PKEY for SSL_SECOP_TMP_DH in the security callback 2020-10-16 14:58:44 +01:00
statem_dtls.c Update copyright year 2020-10-15 14:10:06 +01:00
statem_lib.c Run the withlibctx.pl script 2020-10-01 09:25:20 +01:00
statem_local.h Update copyright year 2020-06-04 14:33:57 +01:00
statem_srvr.c Run the withlibctx.pl script 2020-10-01 09:25:20 +01:00
statem.c
statem.h

State Machine Design

This file provides some guidance on the thinking behind the design of the state machine code to aid future maintenance.

The state machine code replaces an older state machine present in OpenSSL versions 1.0.2 and below. The new state machine has the following objectives:

  • Remove duplication of state code between client and server
  • Remove duplication of state code between TLS and DTLS
  • Simplify transitions and bring the logic together in a single location so that it is easier to validate
  • Remove duplication of code between each of the message handling functions
  • Receive a message first and then work out whether that is a valid transition - not the other way around (the other way causes lots of issues where we are expecting one type of message next but actually get something else)
  • Separate message flow state from handshake state (in order to better understand each)
    • message flow state = when to flush buffers; handling restarts in the event of NBIO events; handling the common flow of steps for reading a message and the common flow of steps for writing a message etc
    • handshake state = what handshake message are we working on now
  • Control complexity: only the state machine can change state: keep all the state changes local to the state machine component

The message flow state machine is divided into a reading sub-state machine and a writing sub-state machine. See the source comments in statem.c for a more detailed description of the various states and transitions possible.

Conceptually the state machine component is designed as follows:

                      libssl
                         |
-------------------------|-----statem.h------------------------------------
                         |
                  _______V____________________
                 |                            |
                 |    statem.c                |
                 |                            |
                 |    Core state machine code |
                 |____________________________|
      statem_local.h     ^          ^
               _________|          |_______
              |                            |
 _____________|____________   _____________|____________
|                          | |                          |
| statem_clnt.c            | | statem_srvr.c            |
|                          | |                          |
| TLS/DTLS client specific | | TLS/DTLS server specific |
| state machine code       | | state machine code       |
|__________________________| |__________________________|
             |        |_______________|__       |
             |        ________________|  |      |
             |       |                   |      |
 ____________V_______V________   ________V______V_______________
|                             | |                               |
| statem_both.c               | | statem_dtls.c                 |
|                             | |                               |
| Non core functions common   | | Non core functions common to  |
| to both servers and clients | | both DTLS servers and clients |
|_____________________________| |_______________________________|