-
Notifications
You must be signed in to change notification settings - Fork 7
Opensearch for document queries + generic case search #810
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
marijnritense
merged 51 commits into
feature/open-search
from
poc/opensearch-for-document-queries
Jul 21, 2026
Merged
Changes from all commits
Commits
Show all changes
51 commits
Select commit
Hold shift + click to select a range
e59dc9c
form flow for building blocks
marijnritense 001e1e1
feedback
marijnritense feab67b
feedback
marijnritense 1050e68
PoC mongo for document queries
marijnritense 2e8ef50
indexes
marijnritense 2722d36
opensearch poc (wip)
marijnritense 0262312
more poc
marijnritense 38ff807
bumped up limit to see proper counts and pagination
marijnritense 00d0087
more poc
marijnritense 3b39d66
more poc
marijnritense 5b5184a
Merge branch 'next-minor' into poc/opensearch-for-document-queries
marijnritense 78c7f00
merge issue fixed
marijnritense 4b88594
added global search to the case list
marijnritense f648746
added toggle for search engine
marijnritense 66d362f
removed mongodb module
marijnritense 71a19d4
fixed tests
marijnritense b2d654b
search improvements for both opensearch and the current one. Includes…
marijnritense 2ed5e5a
added fallback for when opensearch is unavailable
marijnritense c88fae7
changed opensearch to be disabled by default
marijnritense 68aa153
Opensearch improvements (#774)
ivo-ritense 39e830f
Merge branch 'next-minor' into poc/opensearch-for-document-queries
marijnritense 9000142
- open search admin page
marijnritense c79ff9c
made search field always expanded
marijnritense d8a7c52
query fixes to respect access control
marijnritense b166d87
Merge branch 'feature/gh794-global-search-and-opensearch' into poc/op…
marijnritense d3961b8
resilience + UI changes
marijnritense bb98e31
added overview for past reindex runs
marijnritense 6ce480f
made the expanded rows consistent with carbon design
marijnritense 309390d
many reindexing page improvements
marijnritense aa86120
performance improvements for reindexing (caching, storing final count…
marijnritense e973d8f
fixed corner case where an empty table would not update when starting…
marijnritense a6ff1f1
Merge branch 'next-minor' into poc/opensearch-for-document-queries
marijnritense dee1946
PR feedback
marijnritense 3157a20
Merge branch 'refs/heads/next-minor' into poc/opensearch-for-document…
marijnritense fa6659a
PR feedback
marijnritense d30f2ed
pr feedback
marijnritense f2d6632
pr feedback
marijnritense 5c2e173
changed the changedOn update to not rely on the CurrentTimeStamp anno…
marijnritense f50634a
added opensearch configurations to the other apps + moved dependency …
marijnritense fab0808
fixed bug where the searches would be case sensitive
marijnritense 14901ab
security for opensearch
marijnritense bce4230
added progress bar for pruning
marijnritense 2fd68d5
documentation
marijnritense ed25f9c
fixed potential risk with sql injection
marijnritense 6dcd4db
improved when the opensearch beans are available
marijnritense d7cbb1e
fixed incorrect link
marijnritense a125a18
fixed corner case allowing users to search when no search fields were…
marijnritense 7ef5690
changed opensearch reindex runs to be split into two sections
marijnritense aa36520
Adjust styles
sofiaIvarsRitense c04eb42
Remove extra margin bottom
sofiaIvarsRitense 1511131
pr feedback, extra tests
marijnritense File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
74 changes: 74 additions & 0 deletions
74
...apps/dev/src/main/kotlin/com/ritense/gzac/opensearch/InsecureSslOpenSearchClientConfig.kt
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,74 @@ | ||
| /* | ||
| * Copyright 2015-2026 Ritense BV, the Netherlands. | ||
| * | ||
| * Licensed under EUPL, Version 1.2 (the "License"); | ||
| * you may not use this file except in compliance with the License. | ||
| * You may obtain a copy of the License at | ||
| * | ||
| * https://joinup.ec.europa.eu/collection/eupl/eupl-text-eupl-12 | ||
| * | ||
| * Unless required by applicable law or agreed to in writing, software | ||
| * distributed under the License is distributed on an "AS IS" basis, | ||
| * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| * See the License for the specific language governing permissions and | ||
| * limitations under the License. | ||
| */ | ||
|
|
||
| package com.ritense.gzac.opensearch | ||
|
|
||
| import org.apache.http.auth.AuthScope | ||
| import org.apache.http.auth.UsernamePasswordCredentials | ||
| import org.apache.http.conn.ssl.NoopHostnameVerifier | ||
| import org.apache.http.conn.ssl.TrustAllStrategy | ||
| import org.apache.http.impl.client.BasicCredentialsProvider | ||
| import org.apache.http.ssl.SSLContextBuilder | ||
| import org.opensearch.client.RestClientBuilder | ||
| import org.opensearch.spring.boot.autoconfigure.RestClientBuilderCustomizer | ||
| import org.springframework.beans.factory.annotation.Value | ||
| import org.springframework.context.annotation.Bean | ||
| import org.springframework.context.annotation.Configuration | ||
|
|
||
| /** | ||
| * Dev-only configuration that disables SSL certificate verification for OpenSearch | ||
| * and configures basic authentication. | ||
| * Allows connecting to OpenSearch with self-signed certificates in local development. | ||
| * | ||
| * DO NOT use this configuration in production. | ||
| */ | ||
| @Configuration | ||
| class InsecureSslOpenSearchClientConfig { | ||
|
ivo-ritense marked this conversation as resolved.
|
||
|
|
||
| @Value("\${opensearch.username:}") | ||
| private lateinit var username: String | ||
|
|
||
| @Value("\${opensearch.password:}") | ||
| private lateinit var password: String | ||
|
|
||
| @Bean | ||
| fun insecureSslRestClientBuilderCustomizer(): RestClientBuilderCustomizer { | ||
| return object : RestClientBuilderCustomizer { | ||
| override fun customize(builder: RestClientBuilder) { | ||
| builder.setHttpClientConfigCallback { httpClientBuilder -> | ||
| val sslContext = SSLContextBuilder.create() | ||
| .loadTrustMaterial(TrustAllStrategy.INSTANCE) | ||
| .build() | ||
|
|
||
| httpClientBuilder | ||
| .setSSLContext(sslContext) | ||
| .setSSLHostnameVerifier(NoopHostnameVerifier.INSTANCE) | ||
|
|
||
| if (username.isNotBlank() && password.isNotBlank()) { | ||
| val credentialsProvider = BasicCredentialsProvider() | ||
| credentialsProvider.setCredentials( | ||
| AuthScope.ANY, | ||
| UsernamePasswordCredentials(username, password) | ||
| ) | ||
| httpClientBuilder.setDefaultCredentialsProvider(credentialsProvider) | ||
| } | ||
|
|
||
| httpClientBuilder | ||
| } | ||
| } | ||
| } | ||
| } | ||
| } | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,74 @@ | ||
| /* | ||
| * Copyright 2015-2024 Ritense BV, the Netherlands. | ||
| * | ||
| * Licensed under EUPL, Version 1.2 (the "License"); | ||
| * you may not use this file except in compliance with the License. | ||
| * You may obtain a copy of the License at | ||
| * | ||
| * https://joinup.ec.europa.eu/collection/eupl/eupl-text-eupl-12 | ||
| * | ||
| * Unless required by applicable law or agreed to in writing, software | ||
| * distributed under the License is distributed on an "AS IS" basis, | ||
| * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| * See the License for the specific language governing permissions and | ||
| * limitations under the License. | ||
| */ | ||
|
|
||
| dockerCompose { | ||
| projectName = "case-opensearch" | ||
| integrationTestingPostgresql { | ||
| isRequiredBy(project.tasks.integrationTestingPostgresql) | ||
| useComposeFiles.addAll( | ||
| "../docker-resources/docker-compose-base-test-postgresql.yml", | ||
| "docker-compose-override-postgresql.yml" | ||
| ) | ||
| } | ||
| } | ||
|
|
||
| dependencies { | ||
| implementation project(":backend:admin-settings") | ||
| implementation project(":backend:authorization") | ||
| implementation project(":backend:case") | ||
| implementation project(":backend:inbox") | ||
| implementation project(":backend:outbox") | ||
|
|
||
| // ShedLock for cross-instance coordination of the re-index job. The LockProvider bean itself | ||
| // is supplied at runtime by core's SchedulerAutoConfiguration; here we only need the API. | ||
| implementation "net.javacrumbs.shedlock:shedlock-spring:${shedlockVersion}" | ||
|
|
||
| implementation "org.springframework.boot:spring-boot-starter-data-jpa" | ||
| implementation "org.springframework.boot:spring-boot-starter-web" | ||
| implementation "org.springframework.boot:spring-boot-starter-security" | ||
| implementation "org.springframework.boot:spring-boot-autoconfigure" | ||
| implementation "com.fasterxml.jackson.module:jackson-module-kotlin" | ||
| implementation "io.github.oshai:kotlin-logging:${kotlinLoggingVersion}" | ||
|
|
||
| // OpenSearch via spring-data-opensearch (Apache 2.0 licensed) | ||
| implementation "org.opensearch.client:spring-data-opensearch-starter:${springDataOpenSearchVersion}" | ||
|
|
||
| annotationProcessor "org.springframework.boot:spring-boot-autoconfigure-processor" | ||
|
|
||
| testImplementation project(':backend:test-utils-common') | ||
| testImplementation project(':backend:core') | ||
| testImplementation(project(':backend:audit')) { | ||
| exclude(group: "com.ritense.valtimo", module: "case") | ||
| } | ||
| testImplementation "org.springframework.boot:spring-boot-starter-test" | ||
| testImplementation "org.mockito.kotlin:mockito-kotlin:${mockitoKotlinVersion}" | ||
| testImplementation "org.postgresql:postgresql" | ||
| testImplementation "org.springframework.security:spring-security-test" | ||
|
|
||
| jar { | ||
| enabled = true | ||
| manifest { | ||
| attributes("Implementation-Title": "Ritense Case OpenSearch module") | ||
| attributes("Implementation-Version": projectVersion) | ||
| } | ||
| } | ||
| } | ||
|
marijnritense marked this conversation as resolved.
|
||
|
|
||
| tasks.named("integrationTestingPostgresql") { | ||
| systemProperty("liquibase.duplicateFileMode", "WARN") | ||
| } | ||
|
|
||
| apply from: "gradle/publishing.gradle" | ||
20 changes: 20 additions & 0 deletions
20
backend/case-opensearch/docker-compose-override-postgresql.yml
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,20 @@ | ||
| services: | ||
| db: | ||
| ports: | ||
| - "3365:5432" | ||
| environment: | ||
| - POSTGRES_DB=case-opensearch-test | ||
| opensearch: | ||
| image: opensearchproject/opensearch:2.19.2 | ||
| environment: | ||
| - discovery.type=single-node | ||
| - DISABLE_SECURITY_PLUGIN=true | ||
| - DISABLE_INSTALL_DEMO_CONFIG=true | ||
| ports: | ||
| - "39200:9200" | ||
| healthcheck: | ||
| test: ["CMD-SHELL", "curl -sf http://localhost:9200/_cluster/health || exit 1"] | ||
| interval: 5s | ||
| timeout: 10s | ||
| retries: 40 | ||
| start_period: 15s |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.