-
Notifications
You must be signed in to change notification settings - Fork 24.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[CI] SSLConfigurationReloaderTests testReloadingKeyStore failing #108774
Comments
Pinging @elastic/es-security (Team:Security) |
Muting this since I've also seen this fail earlier today: https://gradle-enterprise.elastic.co/s/skv7lgzrvktq6/tests/task/:x-pack:plugin:core:test/details/org.elasticsearch.xpack.core.ssl.SSLConfigurationReloaderTests/testReloadingKeyStore?top-execution=1 |
I can reliable reproduce with Java 23, but not Java 22
To avoid some misleading warnings, rule out the security manager, and provide better logging here is a better reproduction line:
fails with Java 23, but works with Java 22. Java 22 (works)
Java 23 (fails)
I am pretty clueless to what the root cause may be. It is either an issue with out MockWebServer which delegates down to com.sun.net.httpserver.HttpsServer or an issue with apache http client. I ran another test that uses the MockWebServer with HTTPS and it worked..so I am pretty clueless but take another look soon. |
I can get the test to pass in Java23 by changing: --- a/x-pack/plugin/core/src/test/java/org/elasticsearch/xpack/core/ssl/SSLConfigurationReloaderTests.java
+++ b/x-pack/plugin/core/src/test/java/org/elasticsearch/xpack/core/ssl/SSLConfigurationReloaderTests.java
@@ -130,7 +130,7 @@ public class SSLConfigurationReloaderTests extends ESTestCase {
// Load HTTPClient only once. Client uses the same store as a truststore
try (CloseableHttpClient client = getSSLClient(keystorePath, "testnode")) {
final Consumer<SSLContext> keyMaterialPreChecks = (context) -> {
- try (MockWebServer server = new MockWebServer(context, true)) {
+ try (MockWebServer server = new MockWebServer(context, false)) {
server.enqueue(new MockResponse().setResponseCode(200).setBody("body"));
server.start();
privilegedConnect(() -> client.execute(new HttpGet("https://localhost:" + server.getPort())).close()); So, the problem with the test is that in Java23 we are wiring up an apache http client to a mock web server with mutual TLS and for some reason the mock web server does not trust the apache http client. It isn't clear if mTLS is intentional with this test since it really is not an important detail of what is being tested. It also not clear if it worked because the mock web server and apache http client are configured correctly for mTLS or just happened to work in the past. We tend to conflate key and trust stores, so maybe we aren't wiring up mTLS correctly and are relying on a Java bug (pre java 23) for this to work? I'll keep chipping away, but starting to gain more confidence this is not a production concern. |
Pretty confident that for some reason i can not find that pre Java23 mTLS via the MockWebServer-> com.sun.net.httpserver.HttpsServer simply didn't work. I am also pretty sure that asking for mTLS from this mock web server is a typo since we never give the client any private credentials. I think Java23 is behaving correctly, and Java22- is wrong. In Java 22, I was able to get the mock HTTP server to return it's mock response to a cURL call that absolutely does not present a certificate to the server. The same cURL test failed to form a TLS connection in Java 23, which is the correct behavior since the mock web server explicitly requests mTLS. To fix this, I will remove the mTLS from the test since it was never part of what the test was testing. I don't know specifically what was fixed/broken but since we don't use com.sun.net.httpserver.HttpsServer in production code, probably not a big deal. |
Seems to be related to JDK23 somehow, although I don't see any direct indication in the logs and the failure.
Build scan:
https://gradle-enterprise.elastic.co/s/ml2fwj3ywiafc/tests/:x-pack:plugin:core:test/org.elasticsearch.xpack.core.ssl.SSLConfigurationReloaderTests/testReloadingKeyStore
Reproduction line:
Applicable branches:
main
Reproduces locally?:
Yes
Failure history:
Failure dashboard for
org.elasticsearch.xpack.core.ssl.SSLConfigurationReloaderTests#testReloadingKeyStore
Failure excerpt:
The text was updated successfully, but these errors were encountered: