Path: ...!weretis.net!feeder9.news.weretis.net!tncsrv06.tnetconsulting.net!tncsrv09.home.tnetconsulting.net!.POSTED.omega.home.tnetconsulting.net!not-for-mail
From: Grant Taylor <gtaylor@tnetconsulting.net>
Newsgroups: comp.mail.sendmail
Subject: Re: adding CA certificates (for use by sendmail)
Date: Tue, 12 Nov 2024 22:08:08 -0600
Organization: TNet Consulting
Message-ID: <vh18n8$oeh$1@tncsrv09.home.tnetconsulting.net>
References: <87ttcbly3k.fsf@example.com>
Mime-Version: 1.0
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
Injection-Date: Wed, 13 Nov 2024 04:08:08 -0000 (UTC)
Injection-Info: tncsrv09.home.tnetconsulting.net; posting-host="omega.home.tnetconsulting.net:198.18.1.11";
	logging-data="25041"; mail-complaints-to="newsmaster@tnetconsulting.net"
User-Agent: Mozilla Thunderbird
Content-Language: en-US
In-Reply-To: <87ttcbly3k.fsf@example.com>
Bytes: 8799
Lines: 215

On 11/12/24 19:30, Wolfgang Agnes wrote:
> It's not easy to decide where I should ask a FreeBSD group or 
> sendmail one.  I'm posting to both, but suggesting a follow-up-to 
> comp.mail.sendmail.

I get why you chose comp.mail.sendmail.  But my take on this is that 
it's more of an OS (lack of certs) issue than it is a Sendmail 
(configuration) issue.  But both work for me.

> I'm particularly interested---due to curiosity and my education on 
> the subject---in the following sendmail message in maillog:
> 
> --8<-------------------------------------------------------->8---
> Nov 12 08:27:39 antartida sm-mta[72775]: STARTTLS=client, cert-subject=/CN=mx.google.com, cert-issuer=/C=US/O=Google+20Trust+20Services/CN=WR2, verifymsg=unable to get issuer certificate
> --8<-------------------------------------------------------->8---
> 
> My intuition says the system doesn't have the certificate for the CN
> mx.google.com.

I interpret it a little bit differently.

As is true with most Unix errors, the error is staring you in the face. 
But you have to have seen the error before to recognize and understand it:

--> unable to get issuer certificate <--

The issuer is:

--> cert-issuer=/C=US/O=Google+20Trust+20Services/CN=WR2 <--

Or decoded:

--> cert-issuer=/C=US/O=Google Trust Services/CN=WR2 <--

The problem is related to the issuer's (Google Trust Service) 
certificate, not the subject's (mx.google.com) certificate.

> If my intuition is totally wrong, feel free to stop reading right here.

You're closer than you realize.

> The hostname connecting to my sendmail that generated that line above is
> 
>    aspmx.l.google.com

I think this was from Sendmail /sending/ an email to Google and acting 
as the client to Google's server.

> So I tried (as a guess) to connect to this host on port 25, saying 
> STARTTLS and fetching the certificates that came.

:-)

> Because sendmail is my primary concern, I installed these certificates 
> in /etc/mail/certs.  I was afraid that if they were outside of 
> /etc/mail, sendmail would refuse to read them.  And if my operation 
> were not successful in this conservative approach, then it would be 
> a waste of time to try to move these cerficates outside of /etc/mail, 
> so I did not even try that because I did not succeed in avoiding the 
> message ``unable to get issuer certificate''.

I would have to reference documentation to refresh myself on the nuances 
and minutia of /etc/mail/certs to know for sure if that was the proper 
thing to do or not.

Having fought this type of battle, and won, multiple times, I would have 
actually opted for the system wide certificate store under /etc/ssl or 
/etc/tls.

> I found got three certificates with the -showcerts option by OpenSSL.

Nice.  I suspect you're now in the 1% of people that work with TLS on 
the command line.  Welcome to the club.  And I'm sorry.

> They were GTS_Root_R1_2028.pem---this expires in 2028, but I already had
> one for GTS Root R1 that expires in 2036---, GTS_WR2_RSA.pem and
> GTS_WR2_EC.pem.

Those names don't /exactly/ match the issuers that I see when running 
tests on my side, but they are very close.

> I obtained them with
> 
>    openssl s_client -connect aspmx.l.google.com:25 \
>        -starttls smtp \
>        -showcerts </dev/null

Nice.

Here's a modified (added STARTTLS bit) version of an excerpt from a 
script I wrote today to check HTTPS chains:

    FQDN=smtp.google.com
    PORT=25

    openssl s_client -showcerts -connect ${FQDN}:${PORT} -STARTTLS smtp 
0</dev/null 2>/dev/null | sed -n '/-----BEGIN 
CERTIFICATE-----/,/-----END CERTIFICATE-----/p' > ${FQDN}.pem

    csplit -zf ${FQDN}.pem ${FQDN}.pem '/-----BEGIN CERTIFICATE-----/' 
'{*}' -b ".cert-%0d.pem" > /dev/null

    PARTS=($(ls -1r ${FQDN}.pem.cert-?.pem))

This leaves us with four files:

  - smtp.google.com.pem - the combined file extracted with s_client
  - smtp.google.com.pem.cert-0.pem - the 1st cert
  - smtp.google.com.pem.cert-1.pem - the 2nd cert
  - smtp.google.com.pem.cert-2.pem - the 3rd cert

> I obtained the hash of each certificate with, for example,
> 
>    openssl x509 -noout -hash < GTS_Root_R1_2028.pem

:-)

I tend to prefer the `-in ${FILE}` vs redirect approach.  But both work.

> Then I created the symbolic links using these hash as ls shows below.

Regarding your "I already had" comment, it looks like you used ".0" for 
all of your hash files.  The .# nomenclature is to allow for different 
files that have the same hash.  Unexpected, but not impossible either.

> But that did not change those log messages.  I don't know how to 
> debug sendmail to the point of seeing which files it is reading. 
> Thanks for any advice you might have.

I think that it's going to depend on how Sendmail is configured.  Is it 
using it's own certificate store or the system wide certificate store?

What is the CACertPath set to in your sendmail.cf file?

You will need your hash links to be in the directory that Sendmail is 
looking for root certs in.

When I look at the three smtp.google.com.pem.cert-#.pem files generated 
above, I find that there is a fourth certificate that is needed.

    for ((x=${#PARTS}; x>0; x--)); do
       echo;
       echo ${PARTS[$x]};
       openssl x509 -noout -subject -subject_hash -startdate -enddate 
-issuer -issuer_hash -in ${PARTS[$x]};
    done

    smtp.google.com.pem.cert-0.pem
    subject= /CN=mx.google.com
    6264a34a
    notBefore=Oct  7 08:25:31 2024 GMT
    notAfter=Dec 30 08:25:30 2024 GMT
    issuer= /C=US/O=Google Trust Services/CN=WR2
    3c8b39ef

    smtp.google.com.pem.cert-1.pem
    subject= /C=US/O=Google Trust Services/CN=WR2
    3c8b39ef
    notBefore=Dec 13 09:00:00 2023 GMT
    notAfter=Feb 20 14:00:00 2029 GMT
    issuer= /C=US/O=Google Trust Services LLC/CN=GTS Root R1
    1001acf7

    smtp.google.com.pem.cert-2.pem
    subject= /C=US/O=Google Trust Services LLC/CN=GTS Root R1
    1001acf7
    notBefore=Jun 19 00:00:42 2020 GMT
    notAfter=Jan 28 00:00:42 2028 GMT
    issuer= /C=BE/O=GlobalSign nv-sa/OU=Root CA/CN=GlobalSign Root CA
    5ad8a5d6

The fourth file that's needed is the 5ad8a5d6.0 file (in /etc/ssl/certs 
on my system):

    openssl x509 -noout -subject -subject_hash -startdate -enddate 
-issuer -issuer_hash -in /etc/ssl/certs/5ad8a5d6.0

    subject= /C=BE/O=GlobalSign nv-sa/OU=Root CA/CN=GlobalSign Root CA
    5ad8a5d6
    notBefore=Sep  1 12:00:00 1998 GMT
========== REMAINDER OF ARTICLE TRUNCATED ==========