Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
37 commits
Select commit Hold shift + click to select a range
92d8e39
Don't use CRLs for pool internal host-host TLS communications
minglumlu Jan 27, 2026
4299249
Add 'purpose' field in Certificate datamodel
minglumlu Jan 6, 2026
35acb7a
Add 'pinned' in type of Certificate
minglumlu Jan 6, 2026
3d01cb9
Add new APIs for trusted certificates (no impl.)
minglumlu Jan 6, 2026
7353dc1
Implement APIs in xapi_pool.ml
minglumlu Jan 6, 2026
4195da2
Check if already exists when installing a certificate
minglumlu Jan 9, 2026
982d9d2
Update host_install
minglumlu Jan 6, 2026
adb85d1
Update host_uninstall
minglumlu Jan 7, 2026
db9cc54
Make CertificateProvider.store_path support multiple paths
minglumlu Jan 20, 2026
8af92c9
Add new constructors of certificate type for trusted certs
minglumlu Jan 20, 2026
5d00572
Update copy_certs_to_host to include trusted certs
minglumlu Jan 7, 2026
d40c350
Use Cert_distrib instead of sync* in Certificates
minglumlu Jan 21, 2026
9b12db8
Update how to update bundles
minglumlu Jan 21, 2026
c73183a
Add collect_trusted_certs
minglumlu Jan 22, 2026
b7bc861
Add pool.exchange_trusted_certificates_on_join
minglumlu Jan 22, 2026
40003ac
Add collect_crls
minglumlu Jan 22, 2026
cccecfc
Add pool.exchange_crls_on_join
minglumlu Jan 23, 2026
277ab8a
Fix certificate error messages
minglumlu Jan 24, 2026
6ae6dcd
Cleanup trusted on ejected host
minglumlu Jan 24, 2026
3eef0a9
[doc] Update how to handle trusted certificates when pool.join
minglumlu Feb 4, 2026
b0f0c0e
Bump up last_known_schema_hash
minglumlu Mar 4, 2026
0bc05a2
CA-423556: print full checkout error msg in xe apply-edition
changlei-li Feb 5, 2026
2229630
CP-310090 Stunnel lib: Expose unix socket path for TLS proxy
changlei-li Feb 4, 2026
18dde79
Improve certificate_verity errors
changlei-li Feb 4, 2026
bc72990
Support check stunnel log with input channel
changlei-li Feb 4, 2026
fcce6f9
Add unit test for stunnel_log_scanner
changlei-li Feb 4, 2026
9919734
[Backport] Improvement of handling trusted certificates (#6931)
minglumlu Mar 6, 2026
7b7a701
[Merge lcm to lcm feature] For trusted certificates feature (#6940)
minglumlu Mar 10, 2026
8789d0c
[LCM to feature] Merge LCM to feature branch (#6962)
minglumlu Mar 20, 2026
09d2b62
Merge branch '26.1-lcm' into feature/26.1-lcm/trusted-certs
minglumlu Apr 15, 2026
daef7c7
Bump up schema_minor_vsn to 794
minglumlu Apr 14, 2026
e6e6dd5
Merge 26.1-lcm into feature/26.1-lcm/trusted-certs (#7012)
minglumlu Apr 15, 2026
a0f895e
Merge 26.1-lcm into feature/26.1-lcm/trusted-certs (#7027)
minglumlu Apr 22, 2026
4c48602
Update datamodel_lifecycle for feature/trusted-certs
minglumlu Apr 22, 2026
6dfe35d
[LCM] Update datamodel_lifecycle for feature/trusted-certs (#7029)
minglumlu Apr 27, 2026
33a4a31
Merge LCM into feature/trusted-certs (#7038)
minglumlu Apr 27, 2026
7989d70
Merge 26.1-lcm into feature/trusted-certs (#7044)
minglumlu Apr 29, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions doc/content/design/pool-certificates.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,8 @@ revision: 2
status: released (22.6.0)
---

This design is modified by [trusted-certificates.md](trusted-certificates.md).

## Overview

Xenserver has used TLS-encrypted communications between xapi daemons in a pool since its first release.
Expand Down Expand Up @@ -353,22 +355,30 @@ This feature needs clients to behave differently when initiating pool joins, to
Several alerts are introduced:
* POOL_CA_CERTIFICATE_EXPIRING_30, POOL_CA_CERTIFICATE_EXPIRING_14, POOL_CA_CERTIFICATE_EXPIRING_07, POOL_CA_CERTIFICATE_EXPIRED: Similar to host certificates, now the user-installable pool's CA certificates are monitored for expiry dates and alerts are generated about them. The body for this type of message is:

```
<body><message>The trusted TLS server certificate {is expiring soon|has expired}.</message><date>20210302T02:00:01Z</date></body>
```

* HOST_INTERNAL_CERTIFICATE_EXPIRING_30, HOST_INTERNAL_CERTIFICATE_EXPIRING_14, HOST_INTERNAL_CERTIFICATE_EXPIRING_07, HOST_INTERNAL_CERTIFICATE_EXPIRED: Similar to host certificates, the newly-introduced hosts' internal server certificates are monitored for expiry dates and alerts are generated about them. The body for this type of message is:

```
<body><message>The TLS server certificate for internal communications {is expiring soon|has expired}.</message><date>20210302T02:00:01Z</date></body>
```

* TLS_VERIFICATION_EMERGENCY_DISABLED: The host is in emergency mode and is not enforcing tls verification anymore, the situation that forced the disabling must be fixed and the verification enabled ASAP.

```
<body><host>HOST-UUID</host></body>
```

* FAILED_LOGIN_ATTEMPTS: An hourly alert that contains the number of failed attempts and the 3 most common origins for these failed alerts. The body for this type of message is:

```
<body>
<total>35</total>
<known><username>usr5</username><originator>origin5</originator><ip>5.4.3.2</ip><number>10</number><date>20200922T15:03:13Z</date></known>
<known><username>usr4</username><useragent>UA</useragent><number>6</number><date>20200922T15:03:13Z</date></known>
<known><useragent>UA</useragent><ip>4.3.2.1</ip><number>4</number><date>20200922T14:57:11Z</date></known>
<unknown>10</unknown>
</body>
```
127 changes: 111 additions & 16 deletions doc/content/design/trusted-certificates.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,16 +2,16 @@
title: Trusted certificates for identity validation in TLS connections
layout: default
design_doc: true
revision: 1
revision: 2
status: draft
---

# Overview

In various use cases, TLS connections are established on the host on which XAPI runs.
When establishing a TLS connection, the peer identity needs to be validated.
This is done using either a root CA certificate to perform certificate chain validation, or a known peer certificate for validation with certificate pinning.
The root CA certificates and peer certificates involved in this process are referred to as trusted certificates.
This is done using either a root CA certificate to perform certificate chain validation, or a pinned certificate for validation with certificate pinning.
The root CA certificates and pinned certificates involved in this process are referred to as trusted certificates.
When a trusted certificate is installed, the local endpoint can validate the peer identity during TLS connection establishment.
Certificate chain validation is a general-purpose, standards-based approach but requires additional steps, such as getting the peer's certificate signed by a CA.
In contrast, certificate pinning offers a quicker way to set up trust in some cases without the overhead of CA signing.
Expand All @@ -21,13 +21,13 @@ This allows the use case to start in quicker and easier way without prior CA sig

As the unified API for the whole system, XAPI also exposes interfaces for users to install and manage trusted certificates that are used by system components for different purposes.

The base design described in [pool-certificates.md](https://github.com/minglumlu/xen-api/blob/5d1ea1520825d502c57a90a02db476cd7d6a9132/doc/content/design/pool-certificates.md) defines the database, API, and trust store in the filesystem for managing trusted certificates.
The base design described in [pool-certificates.md](pool-certificates.md) defines the database, API, and trust store in the filesystem for managing trusted certificates.
This document introduces the following enhancements to that design:

* Explicit separation of root CA certificates and peer certificates:
* Explicit separation of root CA certificates and pinned certificates:
In the base design, both certificate types share the same database schema, APIs, and are stored together in a single bundle file.
This makes it difficult to determine the appropriate validation approach based on the certificate type.
The improvement introduces a type value to separate root CA certificates and peer certificates explicitly.
The improvement introduces a type value to separate root CA certificates and pinned certificates explicitly.

* Add a "purpose" attribute for trusted certificates:
According to the base design, only certificates used for internal TLS connections among XAPI processes within a pool are stored separately.
Expand All @@ -49,13 +49,13 @@ This case benefits from the improvements introduced in this design as well.
## Database schema
The *Certificate* class in database is defined to represent general certificates, including trusted certificates.
One existing class field "type" supports the following enumeration values:
* "ca": trusted certificates including both root CA and peer.
* "ca": trusted certificates including both root CA and pinned.
* "host": identity certificate of a host for communication with entities outside the pool.
* "host_internal": identity certificate of a host for communication with other pool members.

Two improvements in this design:
* A new value "peer" is introduced in this design so that the existing "ca" now represents trusted root CA only.
The new "peer" will represent trusted peer certificates.
* A new value "pinned" is introduced in this design so that the existing "ca" now represents trusted root CA only.
The new "pinned" will represent trusted pinned certificates.

* A new enumeration type "purpose" is introduced to indicate the intended usage of a trusted certificate.
A new *Certificate* class field "purpose" (a set of values of enumeration type "purpose") will be added to represent all applicable purposes of a trusted certificate.
Expand All @@ -79,27 +79,122 @@ For the same reason, "pool.uninstall_ca_certificate" will also be deprecated.
This is a new API introduced in this design with its arguments being defined as:
* session (ref session_id): reference to a valid session;
* self (ref Pool): reference to the pool;
* ca (boolean): the trusted certificate is a root CA certificate used to verify a chain (true), or a peer certificate used for certificate pinning (false);
* ca (boolean): the trusted certificate is a root CA certificate used to verify a chain (true), or a pinned certificate used for certificate pinning (false);
* cert (string): the trusted certificate in PEM format;
* purpose (string list): the purposes of the trusted certificate.

This new API is used to install trusted certificate.
When *purpose* is an empty set, it stands for a root CA certificate for general purpose.
The *purpose* can not be an empty set when the *ca* is false, because each peer certificate is specific to a single server and therefore unsuitable for a shared trusted certificate for general purpose.
The *purpose* can not be an empty set when the *ca* is false, because each pinned certificate is specific to a single server and therefore unsuitable for a shared trusted certificate for general purpose.

It returns *void* when succeed. Otherwise, return corresponding API error.

### pool.uninstall_trusted_certificate
This is a new API introduced in this design to uninstall a trusted certificate with its arguments being defined as:
* session (ref session_id): reference to a valid session;
* certificate (ref Certificate): reference to the trusted certificate;
* force (bool): remove the database entry even if the file doesn't exist.

It returns *void* when succeed. Otherwise, return corresponding API error.

### pool.join
Prior to this design, trusted certificates are exchanged between the pool and the joining host during the pre‑join phase.
This design preserves that behavior to ensure the joiner works correctly both before and after joining the pool.
According to the base design, trusted certificates are exchanged between the pool and the joining host during the pre‑join phase.
This design basically preserves that behavior to ensure the joiner works correctly both before and after joining the pool.
However, a number of modifications have been introduced compared with the base design.

~~~mermaid

sequenceDiagram
participant clnt as Client
participant join as Joiner
participant coor as Coordinator
participant memb as Member
clnt->>join: Pool.join coordinator_ip coordinator_username coordinator_password
join->>coor:login_with_password rpc_no_verify coordinator_ip coordinator_username coordinator_password
coor-->>join:

Note over join: pre_join_checks
rect rgba(0,0,0,0.05)
join->>join: assert_tls_verification_matches
alt fails
Note over join: interrupt join, raise error
end
end

Note over join: exchnage trusted intra-pool host identity certificates
rect rgba(0,0,0,0.05)
join->>coor: Pool.exchange_certificates_on_join <Joiner's trusted host identity cert>
coor->>coor: Cert_distrib.exchange_certificates_with_joiner start
coor->>memb: Host.cert_distrib_atom Write
memb-->>coor:
coor->>coor: Cert_distrib.get_local_pool_certs
coor-->>coor: Cert_distrib.exchange_certificates_with_joiner done
coor-->>join: <trusted host identity certs in pool>
join->>join: Cert_distrib.import_joining_pool_certs <trusted host identity certs in pool>
end

join->>coor:login_with_password rpc_verify coordinator_ip coordinator_username coordinator_password
coor-->>join:

Note over join: exchange legacy ca certificates
rect rgba(0,0,0,0.05)
join->>coor: Pool.exchange_ca_certificates_on_join <Joiner's legacy ca certs>
coor->>coor: Cert_distrib.exchange_ca_certificates_with_joiner
coor-->>join: <legacy ca certs in pool>
join->>join: Cert_distrib.import_joining_pool_ca_certificates <legacy ca certs in pool>
end

Note over join: exchange trusted certificates
rect rgba(0,0,0,0.05)
join->>coor: Pool.exchange_trusted_certificates_on_join <Joiner's trusted certs>
loop for every <trusted> in Joiner
coor->>coor: Pool.install_trusted_certificate start
coor->>memb: Host.cert_distrib_atom Write
memb-->>coor:
coor->>memb: Host.certificate_sync
memb-->>coor:
coor-->>coor: Pool.install_trusted_certificate done
end
coor->>coor: Cert_distrib.collect_trusted_certs
coor-->>join: <trusted certs in pool>
loop for every <trusted> in pool
join->>join: Pool.install_trusted_certificate
end
end

Note over join: exchange CRLs
rect rgba(0,0,0,0.05)
join->>coor: Pool.exchange_crls_on_join <Joiner's CRLs>
loop for every <CRL> in Joiner
coor->>coor: Pool.crl_install start
coor->>memb: Host.cert_distrib_atom Write
memb-->>coor:
coor->>memb: Host.certificate_sync
memb-->>coor:
coor-->>coor: Pool.crl_install done
end
coor->>coor: Cert_distrib.collect_crls
coor-->>join: <CLRs in pool>
loop for every <CRL> in pool
join->>join: Pool.crl_install
end
end

join->>coor: Host.add joiner
coor-->>join:

join->>join: restart_as_slave

Note over join: Copy all certificates from coordinator
rect rgba(0,0,0,0.05)
join->>coor: Host.copy_primary_host_certs
coor->>join: Host.cert_distrib_atom Write
join-->>coor:
coor->>join: Host.cert_distrib_atom GenBundle
join-->>coor:
coor-->>join: return from Host.copy_primary_host_certs
end

~~~

### pool.eject
The trusted certificates will be removed from any host which is being eject from the pool.
Expand Down Expand Up @@ -130,10 +225,10 @@ The stores for the certificates installed via "pool.install_trusted_certificate"
| Name | Filesystem location | Used for |
| ---- | ------------------- | -------- |
| Trusted General CA | /etc/trusted-certs/ca-general/ | Trusted root CA certificates that users can install to validate a peer’s identity when establishing a TLS connection for general purpose.
| Trusted Peer | /etc/trusted-certs/peer-\<PURPOSE\>/ | Trusted peer certificates that users can install to validate a peer’s identity when establishing a TLS connection for \<PURPOSE\>.
| Trusted Peer | /etc/trusted-certs/pinned-\<PURPOSE\>/ | Trusted pinned certificates that users can install to validate a peer’s identity when establishing a TLS connection for \<PURPOSE\>.
| Trusted CA | /etc/trusted-certs/ca-\<PURPOSE\>/ | Trusted root CA certificates that users can install to validate a peer’s identity when establishing a TLS connection for \<PURPOSE\>.
| General Bundle | /etc/trusted-certs/ca-bundle-general.pem | Bundle of trusted root CA certificates under /etc/trusted-certs/ca-general/ to verify a peer's identity when establishing a TLS connection for general purpose.
| Peer Bundle | /etc/trusted-certs/peer-bundle-\<PURPOSE\>.pem | Bundle of trusted peer certificates under /etc/trusted-certs/peer-\<PURPOSE\>/ to verify a peer's identity when establishing a TLS connection for \<PURPOSE\>.
| Peer Bundle | /etc/trusted-certs/pinned-bundle-\<PURPOSE\>.pem | Bundle of trusted pinned certificates under /etc/trusted-certs/pinned-\<PURPOSE\>/ to verify a peer's identity when establishing a TLS connection for \<PURPOSE\>.
| CA Bundle | /etc/trusted-certs/ca-bundle-\<PURPOSE\>.pem | Bundle of trusted root CA certificates under /etc/trusted-certs/ca-\<PURPOSE\>/ to verify a peer's identity when establishing a TLS connection for \<PURPOSE\>.

The filesystem location is derived from the \<PURPOSE\>. Each \<PURPOSE\> string corresponds to a predefined value of the "purpose" type in the database, implemented as predefined constants.
Expand Down
18 changes: 16 additions & 2 deletions ocaml/alerts/certificate/certificate_check.ml
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ type cert =
| CA of API.ref_Certificate * API.datetime
| Host of API.ref_host * API.datetime
| Internal of API.ref_host * API.datetime
| Pinned of API.ref_Certificate * API.datetime

let get_certificates rpc session_id =
XenAPI.Certificate.get_all_records ~rpc ~session_id
Expand All @@ -22,14 +23,18 @@ let get_certificates rpc session_id =
)
| `ca ->
CA (cert_ref, certificate.API.certificate_not_after)
| `pinned ->
Pinned (cert_ref, certificate.API.certificate_not_after)

let certificate_description = function
| Host _ ->
"The TLS server certificate"
| Internal _ ->
"The internal TLS server certificate"
| CA _ ->
"The CA pool certificate"
"The pool-wide trusted root CA certificate"
| Pinned _ ->
"The pool-wide trusted pinned leaf certificate"

let alert_conditions = function
| Host _ ->
Expand All @@ -53,16 +58,25 @@ let alert_conditions = function
; (14, Api_messages.pool_ca_certificate_expiring_14)
; (30, Api_messages.pool_ca_certificate_expiring_30)
]
| Pinned _ ->
[
(0, Api_messages.pool_pinned_certificate_expired)
; (7, Api_messages.pool_pinned_certificate_expiring_07)
; (14, Api_messages.pool_pinned_certificate_expiring_14)
; (30, Api_messages.pool_pinned_certificate_expiring_30)
]

let alert_message_cls_and_obj_uuid rpc session_id cert =
match cert with
| Host (host, _) | Internal (host, _) ->
(`Host, XenAPI.Host.get_uuid ~rpc ~session_id ~self:host)
| CA (cert, _) ->
(`Certificate, XenAPI.Certificate.get_uuid ~rpc ~session_id ~self:cert)
| Pinned (cert, _) ->
(`Certificate, XenAPI.Certificate.get_uuid ~rpc ~session_id ~self:cert)

let get_expiry = function
| Host (_, exp) | Internal (_, exp) | CA (_, exp) ->
| Host (_, exp) | Internal (_, exp) | CA (_, exp) | Pinned (_, exp) ->
exp

let alert rpc session_id =
Expand Down
1 change: 1 addition & 0 deletions ocaml/alerts/certificate/certificate_check.mli
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@ type cert =
| CA of API.ref_Certificate * API.datetime
| Host of API.ref_host * API.datetime
| Internal of API.ref_host * API.datetime
| Pinned of API.ref_Certificate * API.datetime

val certificate_description : cert -> string

Expand Down
10 changes: 10 additions & 0 deletions ocaml/idl/datamodel_certificate.ml
Original file line number Diff line number Diff line change
Expand Up @@ -30,9 +30,16 @@ let certificate_type =
; ( "host_internal"
, "Certificate that identifies a single host to other pool members"
)
; ("pinned", "Pinned leaf certificate that is trusted by the whole pool")
]
)

let certificate_purpose =
Enum
( "certificate_purpose"
, [("licensing", "Trusted certificates that are for licensing purpose.")]
)

let t =
create_obj ~name:_certificate
~descr:"An X509 certificate used for TLS connections" ~doccomments:[]
Expand Down Expand Up @@ -75,5 +82,8 @@ let t =
; field ~qualifier:StaticRO ~lifecycle:[] ~ty:String "fingerprint_sha1"
~default_value:(Some (VString ""))
"The certificate's SHA1 fingerprint / hash"
; field ~qualifier:StaticRO ~lifecycle:[] ~ty:(Set certificate_purpose)
"purpose" ~default_value:(Some (VSet []))
"The purposes of the certificate"
]
~messages:[] ()
2 changes: 1 addition & 1 deletion ocaml/idl/datamodel_common.ml
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ open Datamodel_roles
to leave a gap for potential hotfixes needing to increment the schema version.*)
let schema_major_vsn = 5

let schema_minor_vsn = 793
let schema_minor_vsn = 794

(* Historical schema versions just in case this is useful later *)
let rio_schema_major_vsn = 5
Expand Down
30 changes: 28 additions & 2 deletions ocaml/idl/datamodel_errors.ml
Original file line number Diff line number Diff line change
Expand Up @@ -878,6 +878,13 @@ let _ =
"The host joining the pool has different CA certificates from the pool \
coordinator while using the same name, uninstall them and try again."
() ;
error Api_errors.pool_joining_host_trusted_certificates_conflict
["ref_in_pool"; "ref_on_host"]
~doc:
"The joining host has a trusted certificate identical to one on the pool \
coordinator but with different purpose. Uninstall it then install it on \
the host again with the pool-compatible purpose, and try again."
() ;
error Api_errors.pool_joining_sm_features_incompatible
["pool_sm_ref"; "candidate_sm_ref"]
~doc:
Expand Down Expand Up @@ -1664,6 +1671,8 @@ let _ =
~doc:"The specified certificate does not exist." () ;
error Api_errors.certificate_already_exists ["name"]
~doc:"A certificate already exists with the specified name." () ;
error Api_errors.trusted_certificate_already_exists ["fingerprint"]
~doc:"A trusted certificate already exists with the same purpose." () ;
error Api_errors.certificate_name_invalid ["name"]
~doc:"The specified certificate name is invalid." () ;
error Api_errors.certificate_corrupt ["name"]
Expand All @@ -1690,7 +1699,7 @@ let _ =
() ;

error Api_errors.server_certificate_invalid []
~doc:"The provided certificate is not in a PEM-encoded X509." () ;
~doc:"The provided certificate is not in a PEM-encoded X509 format." () ;
error Api_errors.server_certificate_key_mismatch []
~doc:
"The provided key does not match the provided certificate's public key."
Expand All @@ -1706,9 +1715,26 @@ let _ =
() ;

error Api_errors.server_certificate_chain_invalid []
~doc:"The provided intermediate certificates are not in a PEM-encoded X509."
~doc:
"The provided intermediate certificates are not in a PEM-encoded X509 \
format."
() ;

error Api_errors.not_trusted_certificate ["ref"]
~doc:"The provided certificate is not a trusted certificate." () ;

error Api_errors.certificate_lacks_purpose []
~doc:"No purpose is specified for the provided certificate." () ;

error Api_errors.trusted_certificate_expired ["now"; "not_after"]
~doc:"The provided certificate has expired." () ;

error Api_errors.trusted_certificate_not_valid_yet ["now"; "not_before"]
~doc:"The provided certificate is not valid yet." () ;

error Api_errors.trusted_certificate_invalid []
~doc:"The provided certificate is not in a PEM-encoded X509 format." () ;

error Api_errors.vmpp_has_vm []
~doc:"There is at least one VM assigned to this protection policy." () ;
error Api_errors.vmpp_archive_more_frequent_than_backup []
Expand Down
Loading
Loading