folio-spring-support

Copyright (C) 2020-2023 The Open Library Foundation

This software is distributed under the terms of the Apache License, Version 2.0. See the file "LICENSE" for more information.

• folio-spring-support
  • Table of Contents
  • Introduction
  • Code structure
  • Execution Context
  • Properties
  • Kafka Tenant Filtering
  • Database Connection Pool Settings
  • CQL support
  • Logging
    • Default logging format
    • Request and Response Logging
    • HTTP Service Clients connection pool
  • Custom /_/tenant Logic
    • TenantService Event Methods
    • TenantService Methods and Fields
    • Event Order
      • Upon Creation
      • Upon Deletion
    • Sample
  • Internationalization
  • Additional information
    • Issue tracker

This is a library that contains the basic functionality and main dependencies required for development of FOLIO modules using Spring framework (also known as "Spring Way").

Please find a step-by-step guide on how to create a new FOLIO Spring based module at https://github.com/folio-org/mod-spring-template

An example of the module based on folio-spring-support could be found at https://github.com/folio-org/folio-sample-modules/tree/master/mod-spring-petstore

The library comprises several submodules that are built as separate artifacts (jar files) and can be integrated into a project as distinct dependencies. This facilitates more precise dependency management depending on the requirements of each project.

The library includes the following submodules:

• folio-spring-base - provides fundamental functionality for developing FOLIO modules using the Spring framework.
• folio-spring-cql - facilitates CQL querying (refer to the CQL support section below)
• folio-spring-kafka - provides common Kafka support for Spring modules, including tenant-aware message filtering.
• folio-spring-system-user - (deprecated) provides functionality for system-user creation and utilization

FolioExecutionContext is used to store essential request headers (in thread local). Folio Spring Base populates this data using FolioExecutionScopeFilter. It is used by EnrichUrlAndHeadersClient, to provide right tenant id and other headers for outgoing REST requests. It is also used in DataSourceSchemaAdvisorBeanPostProcessor for selection of the appropriate schema for sql queries.

FolioExecutionContext is immutable. In order to start new execution context the construct

  try (var x = new FolioExecutionContextSetter(currentFolioExecutionContext)) {
    chain.doFilter(request, response);
  }

should be used (pick any of the available constructors).

Using try-with-resources is best practice. Not using try-with-resources is error-prone, may result in a wrong tenant and should be avoided. If not using try-with-resources ensure to call folioExecutionContextSetter.close() when the execution is finished. Example:

  // Not using try-with-resources is discouraged!
  var x = new FolioExecutionContextSetter(currentFolioExecutionContext);
  // do some stuff
  x.close();

CAUTION: FolioExecutionContext should not be used in asynchronous code executions (as it is stored in thread local), unless the appropriate data is manually set by using FolioExecutionContextSetter.

Example of asynchronous execution:

private final FolioModuleMetadata folioModuleMetadata;

@Async
void ayncMethod(Map<String, Collection<String>> headers) {
  try (var x = new FolioExecutionContextSetter(folioModuleMetadata, httpHeaders)) {
    _your_code_here_
  }
}

FOLIO scope implementation supports nested FolioExecutionContexts it means that the following code works correctly for

// Autowired
private final FolioModuleMetadata folioModuleMetadata;

// Autowired
protected final FolioExecutionContext context;

void someMethod(Map<String, Collection<String>> headers) {
  Map<String, Collection<String>> headers1 = getHeaderForTenant("Tenant1");
  try (var x = new FolioExecutionContextSetter(folioModuleMetadata, headers1)) {
    String tenant1 = context.getTenantId();
    businessMethod(tenant1);

    Map<String, Collection<String>> headers2 = getHeaderForTenant("Tenant2");
    try (var x = new FolioExecutionContextSetter(folioModuleMetadata, headers2)) {
      String tenant2 = context.getTenantId();
      businessMethod(tenant2);
    }
    
    String tenant1_1 = context.getTenantId();
    assert tenant1.equals(tenant1_1);
  }
}

...

void businessMethod(String tenantId) {
  _do_some_useful_stuff_
  String tenantId = context.getTenantId();

  assert tenant.equals(tenantId);
}

folio-spring-kafka provides a shared Spring Kafka RecordFilterStrategy bean named tenantAwareMessageFilter. The filter is disabled by default. When enabled, it allows a Kafka message to be processed only when the tenant is entitled to the module.

When enabled, the filter:

1. Reads the tenant from Kafka record headers - x-okapi-tenant, falling back to folio.tenantId if that's absent. The filter does not deserialize the message body.
2. Checks if the tenant is entitled to the module, using the in-process entitlement cache described below.
3. If the tenant is entitled, pass the message to the module listener.
4. Applies tenant-disabled-strategy when the tenant is not entitled to the current module.
5. Applies all-tenants-disabled-strategy when no tenants are entitled to the current module.

Per-message filtering never makes a network call. The entitled-tenants set is cached in-process and kept current three ways:

1. A synchronous fetch from the sidecar (GET /entitlements/modules/{moduleId}) on first use, whose result is cached.
2. Direct updates from ENTITLE/UPGRADE/REVOKE events on the entitlement Kafka topic. Each module instance uses its own unique consumer group id, so every instance observes every event.
3. A periodic full re-fetch that corrects any drift from a missed or duplicate event.

To use the filter in a module, add folio-spring-kafka as a dependency and reference it from the listener:

@KafkaListener(
  topics = "...",
  groupId = "...",
  filter = "tenantAwareMessageFilter")

The filter can be enabled through application configuration or deployment environment variables:

folio:
  kafka:
    tenant-filter:
      enabled: true

Or

FOLIO_KAFKA_TENANT_FILTER_ENABLED=true

The environment variable names above are also used by folio-kafka-wrapper, the equivalent library for non-Spring (Vert.x/RMB) modules, for the settings they have in common (all except ignore-empty-batch, which has no RMB equivalent) - so both libraries can be configured with the exact same variables.

The following strategy values are supported:

When Kafka tenant filtering is enabled, the FolioModuleMetadata bean must provide the module name and version. The default metadata bean from folio-spring-base uses spring.application.name and spring.application.version to build the current module id as <spring.application.name>-<spring.application.version>.

Modules can expose these values from Maven build metadata as follows:

spring:
  application:
    name: @project.artifactId@
    version: @project.version@

With Maven resource filtering enabled, @project.artifactId@ and @project.version@ are replaced at build time. For example, this produces a module id such as mod-search-6.0.8.

The tenantAwareMessageFilter bean is registered with @ConditionalOnMissingBean, so a module can provide its own bean with the same name when custom filtering behavior is required.

See Database Connection Pool Settings for the DB_* environment variables and configuration order.

To have ability to search entities in databases by CQL-queries:

• create repository interface for needed entity
• extend it from JpaCqlRepository<T, ID>, where T is entity class and ID is entity's id class.
• the implementation of the repository will be created by Spring

public interface PersonRepository extends JpaCqlRepository<Person, Integer> {

}

Two methods are available for CQL-queries:

public interface JpaCqlRepository<T, ID> extends JpaRepository<T, ID> {

  Page<T> findByCql(String cql, OffsetRequest offset);

  long count(String cql);
}

By default a CQL search a String field ignores case (= is case insensitive) and ignores accents; this is for consistency with RMB based modules. Use the annotations @RespectCase and/or @RespectAccents in the entity class to change the default.

Library uses log4j2 for logging. There are two default log4j2 configurations:

• log4j2.properties console/line based logger and it is the default
• log4j2-json.properties JSON structured logging

To choose the JSON structured logging by using setting: -Dlog4j.configurationFile=log4j2-json.properties A module that wants to generate log4J2 logs in a different format can create a log4j2.properties file in the /resources directory.

For comprehensive information about HTTP request and response logging, including:

• Request Logging (incoming requests to your application)
• Exchange Logging (outgoing HTTP client requests)
• Logging levels (NONE, BASIC, HEADERS, FULL)
• Configuration examples
• Performance and security considerations

See the Request Logging Guide.

Quick Configuration:

folio:
  logging:
    request:
      enabled: true
      level: BASIC    # NONE, BASIC, HEADERS, FULL
    exchange:
      enabled: true
      level: BASIC    # NONE, BASIC, HEADERS, FULL

logging:
  level:
    org.folio.spring.filter.IncomingRequestLoggingFilter: DEBUG
    org.folio.spring.client.ExchangeLoggingInterceptor: DEBUG

Note: In case you have async requests in your module (DeferredResult, CompletableFuture, etc.) then you should disable default logging for requests.

HTTP Service Clients (@HttpExchange) share one pooled Apache HttpClient. Pool size and timeouts are configurable through folio.exchange.http-client.* properties or FOLIO_EXCHANGE_HTTP_CLIENT_* environment variables. See the HTTP Client Configuration Guide.

Quick Configuration:

folio:
  exchange:
    http-client:
      max-connections-per-route: 50   # FOLIO_EXCHANGE_HTTP_CLIENT_MAX_CONNECTIONS_PER_ROUTE
      max-connections-total: 100      # FOLIO_EXCHANGE_HTTP_CLIENT_MAX_CONNECTIONS_TOTAL
      connect-timeout: 10s            # FOLIO_EXCHANGE_HTTP_CLIENT_CONNECT_TIMEOUT

There are many cases where you may want to add custom logic to the /_/tenant endpoint, such as for loading sample data or performing more complex database migration.

In order to do this, you can extend the TenantService within your module and override any of the methods listed below.

The following methods can be overridden by your module in order to add custom logic around events relating to tenant creation, updates, and deletion. All of these return void.

Many of these accept a TenantAttributes parameter which can provide information about the previous module (module_from), the module being upgraded to (module_to), as well as any other parameters provided.

⚠️ Please note that methods with "update" in the name will be run on updates as well as when new tenants are created. Be especially careful with methods run before Liquibase -- the database schema could potentially be in an unexpected state, particularly when a new tenant is created.

There are two methods that may be of use in your custom logic:

• boolean tenantExists() which will check if the database schema for this tenant exists (this says nothing about if it is up to date)
• String getSchemaName() will construct and return the name of the schema corresponding to the module and tenant

These fields will also be provided:

• JdbcTemplate jdbcTemplate, for running Postgres queries directly
• FolioExecutionContext context, for getting information about the module
• FolioSpringLiquibase folioSpringLiquibase, for interacting with Liquibase directly (this extends SpringLiquibase and may be null if Liquibase is not enabled!)

Additional Liquibase helper bean available from the library:

• LiquibaseMigrationLockService can be injected by modules that need to delay work while database migration is in progress.
• boolean isMigrationRunning() returns true when processing should be retried because Liquibase is still running or the Liquibase lock table is not ready yet, and returns false when the database is ready for normal processing.
• LiquibaseMigrationException is thrown when migration state cannot be determined because of an unexpected infrastructure or Liquibase access problem, including missing tenant context.

The events will be called in the following order:

1. beforeTenantUpdate
2. If Liquibase is enabled:
   1. beforeLiquibaseUpdate
   2. Internal logic to apply Liquibase changes
   3. afterLiquibaseUpdate
3. afterTenantUpdate
4. loadReferenceData, if applicable
5. loadSampleData, if applicable

1. beforeTenantDeletion
2. Internal logic to drop the schema
3. afterTenantDeletion

Overriding these methods to add your own custom logic is quite straightforward. Here is an example of how to override these in your very own @Service:

package org.folio.yourmodule.service;

import org.folio.spring.service.TenantService;
import org.folio.tenant.domain.dto.TenantAttributes;
import org.folio.yourmodule.SuperCoolDataRepository;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.context.annotation.Primary;
import org.springframework.stereotype.Service;

@Service
@Primary // required to ensure CustomTenantService will be loaded instead of TenantService
public class CustomTenantService extends TenantService {

  protected final SuperCoolDataRepository repository;

  /**
   * Load reference data
   */
  @Override
  protected void loadReferenceData() {
    repository.loadReferenceData();
  }

  /**
   * Add our custom initial data
   */
  @Override
  protected void beforeTenantUpdate(TenantAttributes attributes) {
    // some custom logic for potentially migrating data
  }
}

Translations may be performed in backend modules using the folio-spring-i18n library. For more information, see the folio-spring-i18n README.

See project FOLSPRINGB at the FOLIO issue tracker.