Arcana Cloud Rust

Enterprise-grade Rust microservices framework implementing Clean Architecture with dual-protocol support (REST/gRPC), WASM plugin system, and comprehensive resilience patterns. Achieves 2.1x performance improvement with gRPC over HTTP/JSON.

Architecture

flowchart TB
    subgraph Client["Client Layer"]
        WEB[Web Browser]
        MOBILE[Mobile App]
        SERVICE[External Service]
    end

    subgraph Gateway["API Gateway"]
        REST[REST API<br/>Axum :8080]
        GRPC[gRPC API<br/>Tonic :9090]
    end

    subgraph Plugins["Plugin System"]
        WASM[WASM Runtime<br/>Wasmtime]
        AUDIT[Audit Plugin]
        CUSTOM[Custom Plugins]
    end

    subgraph Controller["Controller Layer"]
        AUTH_CTRL[Auth Controller]
        USER_CTRL[User Controller]
        HEALTH[Health Controller]
    end

    subgraph Service["Service Layer"]
        AUTH_SVC[Auth Service]
        USER_SVC[User Service]
        TOKEN[Token Provider]
    end

    subgraph Repository["Repository Layer"]
        USER_REPO[User Repository]
        OAUTH_REPO[OAuth Repository]
    end

    subgraph Data["Data Layer"]
        MYSQL[(MySQL/PostgreSQL)]
        REDIS[(Redis Cache)]
    end

    subgraph Resilience["Resilience"]
        CB[Circuit Breaker]
        RETRY[Retry Policy]
        RATE[Rate Limiter]
    end

    WEB --> REST
    MOBILE --> GRPC
    SERVICE --> REST

    REST --> AUTH_CTRL
    REST --> USER_CTRL
    REST --> HEALTH
    GRPC --> AUTH_CTRL
    GRPC --> USER_CTRL

    AUTH_CTRL --> AUTH_SVC
    USER_CTRL --> USER_SVC
    AUTH_SVC --> TOKEN
    AUTH_SVC --> USER_REPO
    USER_SVC --> USER_REPO

    USER_REPO --> MYSQL
    OAUTH_REPO --> MYSQL
    TOKEN --> REDIS

    WASM --> AUDIT
    WASM --> CUSTOM
    AUTH_SVC -.-> WASM
    USER_SVC -.-> WASM

    USER_SVC -.-> CB
    USER_SVC -.-> RETRY
    REST -.-> RATE

    style Client fill:#e0f2fe
    style Gateway fill:#fef3c7
    style Plugins fill:#fce7f3
    style Controller fill:#dcfce7
    style Service fill:#f3e8ff
    style Repository fill:#fed7aa
    style Data fill:#f3f4f6
    style Resilience fill:#e0e7ff

Key Features

mindmap
  root((Arcana Cloud<br/>Rust))
    Clean Architecture
      Domain Layer
      Service Layer
      Repository Layer
      Controller Layer
    Dual Protocol
      REST/Axum
      gRPC/Tonic
      2.1x Speedup
    WASM Plugins
      Wasmtime Runtime
      Sandboxed Execution
      Hot Reload
      Signature Verification
    Job Queue
      Redis Backend
      Priority Queues
      Cron Scheduler
      Dead Letter Queue
    Security
      JWT Auth
      Argon2 Hashing
      RBAC
      mTLS Support
    Resilience
      Circuit Breaker
      Retry Policies
      Rate Limiting
      Timeouts
    Deployment
      Monolithic
      Layered HTTP
      Layered gRPC
      Kubernetes
    Observability
      Tracing
      Prometheus Metrics
      Structured Logging

Deployment Modes

The framework supports 5 deployment configurations optimized for different scenarios:

Mode	Description	Use Case	Latency
Monolithic	All layers in single process	Development, small deployments	0.5ms
Layered + HTTP	REST-based tier communication	Simple distributed	2.8ms
Layered + gRPC	High-performance tier communication	Production distributed	1.2ms
Kubernetes + HTTP	K8s with REST between services	Cloud-native	3.2ms
Kubernetes + gRPC	K8s with gRPC + mTLS	Production cloud	1.5ms

Deployment Mode Comparison

┌─────────────────────────────────────────────────────────────────────────────┐
│                        Monolithic Mode (Default)                             │
├─────────────────────────────────────────────────────────────────────────────┤
│                                                                              │
│   Client ──HTTP/REST──▶ [Controller → Service → Repository] ──▶ Database    │
│            (8080)              Single Process                                │
│                                                                              │
│   Pros: Lowest latency, simplest deployment, easy debugging                  │
│   Cons: No horizontal scaling per layer                                      │
└─────────────────────────────────────────────────────────────────────────────┘

┌─────────────────────────────────────────────────────────────────────────────┐
│                        Layered + gRPC Mode                                   │
├─────────────────────────────────────────────────────────────────────────────┤
│                                                                              │
│   Client ──REST──▶ Controller ──gRPC──▶ Service ──gRPC──▶ Repository        │
│          (8080)      Pod 1       (9090)   Pod 2    (9090)   Pod 3           │
│                                                                              │
│   Pros: Independent scaling, 2.1x faster than HTTP, type-safe contracts     │
│   Cons: Requires gRPC-aware load balancing                                   │
└─────────────────────────────────────────────────────────────────────────────┘

┌─────────────────────────────────────────────────────────────────────────────┐
│                     Kubernetes + gRPC Mode (Production)                      │
├─────────────────────────────────────────────────────────────────────────────┤
│                                                                              │
│   Ingress ──▶ Controller (3x) ──gRPC/mTLS──▶ Service (3x) ──▶ Repository    │
│                    HPA               ↓              HPA                      │
│                              Network Policies                                │
│                                                                              │
│   Pros: Full cloud-native, auto-scaling, security isolation, HA             │
│   Cons: Infrastructure complexity, requires K8s expertise                    │
└─────────────────────────────────────────────────────────────────────────────┘

Plugin System (WASM)

The framework features a WebAssembly-based plugin system using Wasmtime for sandboxed, secure extensibility.

Plugin Architecture

flowchart LR
    subgraph Host["Host Application"]
        RUNTIME[Plugin Runtime<br/>Wasmtime]
        MANAGER[Plugin Manager]
        API[Plugin API]
    end

    subgraph Plugins["WASM Plugins"]
        P1[Audit Plugin]
        P2[Custom Plugin]
        P3[Analytics Plugin]
    end

    subgraph Extensions["Extension Points"]
        E1[Event Handlers]
        E2[HTTP Interceptors]
        E3[Scheduled Tasks]
        E4[Custom Services]
    end

    MANAGER --> RUNTIME
    RUNTIME --> P1
    RUNTIME --> P2
    RUNTIME --> P3

    P1 --> E1
    P2 --> E2
    P3 --> E3
    P2 --> E4

    API -.-> RUNTIME

    style Host fill:#dcfce7
    style Plugins fill:#fce7f3
    style Extensions fill:#e0f2fe

Plugin Lifecycle

sequenceDiagram
    participant Manager as Plugin Manager
    participant Runtime as Wasmtime Runtime
    participant Plugin as WASM Plugin

    Manager->>Runtime: Load WASM module
    Runtime->>Plugin: Instantiate
    Plugin-->>Runtime: PluginDescriptor

    Manager->>Runtime: Enable plugin
    Runtime->>Plugin: on_enable()
    Plugin-->>Runtime: Ok(())

    Note over Manager,Plugin: Plugin Active

    Manager->>Runtime: Handle event
    Runtime->>Plugin: on_event(data)
    Plugin-->>Runtime: Response

    Manager->>Runtime: Disable plugin
    Runtime->>Plugin: on_disable()
    Plugin-->>Runtime: Ok(())

Extension Points

Extension	Description	Use Case
Event Handlers	React to domain events	Audit logging, notifications
HTTP Interceptors	Pre/post request processing	Auth, logging, transformation
Scheduled Tasks	Periodic execution	Cleanup, reports, sync
Custom Services	Register new endpoints	Feature extensions

Creating a Plugin

use arcana_plugin_api::{Plugin, PluginDescriptor, ExtensionPoint};

pub struct AuditPlugin {
    descriptor: PluginDescriptor,
}

impl Plugin for AuditPlugin {
    fn descriptor(&self) -> &PluginDescriptor {
        &self.descriptor
    }

    fn on_enable(&mut self) -> Result<(), String> {
        println!("Audit plugin enabled");
        Ok(())
    }

    fn on_disable(&mut self) -> Result<(), String> {
        println!("Audit plugin disabled");
        Ok(())
    }

    fn on_event(&mut self, event: &str, data: &[u8]) -> Result<Vec<u8>, String> {
        // Handle domain events
        println!("Event: {} with {} bytes", event, data.len());
        Ok(vec![])
    }

    fn extension_points(&self) -> Vec<ExtensionPoint> {
        vec![ExtensionPoint::EventHandler("user.*".to_string())]
    }
}

Compile to WASM:

cargo build --target wasm32-wasi --release -p arcana-audit-plugin
cp target/wasm32-wasi/release/arcana_audit_plugin.wasm plugins/

Distributed Job Queue

The framework includes a Redis-backed distributed job queue system for background task processing with enterprise-grade features.

Job Queue Architecture

flowchart TB
    subgraph Producers["Job Producers"]
        API[REST API]
        GRPC_P[gRPC Service]
        SCHEDULER[Cron Scheduler]
    end

    subgraph Redis["Redis Queue Backend"]
        PRIORITY[Priority Queues<br/>Critical/High/Normal/Low]
        DELAYED[Delayed Jobs]
        DLQ[Dead Letter Queue]
    end

    subgraph Workers["Worker Pool"]
        W1[Worker 1]
        W2[Worker 2]
        W3[Worker N]
    end

    subgraph Monitoring["Observability"]
        METRICS[Prometheus Metrics]
        STATUS[Status API]
        ALERTS[Alerting]
    end

    API --> PRIORITY
    GRPC_P --> PRIORITY
    SCHEDULER --> PRIORITY
    SCHEDULER --> DELAYED

    DELAYED -.->|scheduled time| PRIORITY

    PRIORITY --> W1
    PRIORITY --> W2
    PRIORITY --> W3

    W1 -->|success| COMPLETED[(Completed)]
    W1 -->|failure| RETRY{Retry?}
    RETRY -->|yes| PRIORITY
    RETRY -->|max retries| DLQ

    W1 -.-> METRICS
    W2 -.-> METRICS
    W3 -.-> METRICS
    SCHEDULER -.-> METRICS

    METRICS --> STATUS
    METRICS --> ALERTS

    style Producers fill:#dcfce7
    style Redis fill:#fef3c7
    style Workers fill:#e0f2fe
    style Monitoring fill:#f3e8ff

Job Queue Features

Feature	Description
Priority Queues	4 levels: Critical > High > Normal > Low
Retry Policies	Exponential, linear, fixed backoff with jitter
Dead Letter Queue	Automatic DLQ for exhausted retries
Delayed Jobs	Schedule jobs for future execution
Cron Scheduling	Recurring jobs with leader election
Deduplication	Unique keys prevent duplicate jobs
Job Timeout	Configurable per-job timeouts
Prometheus Metrics	Comprehensive monitoring

Defining a Job

use arcana_jobs::{Job, JobContext, JobError, RetryPolicy};
use serde::{Deserialize, Serialize};

#[derive(Debug, Serialize, Deserialize)]
pub struct SendEmailJob {
    pub to: String,
    pub subject: String,
    pub body: String,
}

#[async_trait::async_trait]
impl Job for SendEmailJob {
    const NAME: &'static str = "send_email";
    const QUEUE: &'static str = "emails";
    const MAX_RETRIES: u32 = 3;
    const TIMEOUT_SECS: u64 = 60;

    async fn execute(&self, ctx: JobContext) -> Result<(), JobError> {
        println!("Sending email to: {}", self.to);
        // Email sending logic here
        Ok(())
    }

    fn retry_policy(&self) -> RetryPolicy {
        RetryPolicy::exponential(3)
            .with_initial_delay(Duration::from_secs(5))
            .with_jitter(0.1)
    }
}

Enqueuing Jobs

use arcana_jobs::{Priority, QueuedJob};

// Simple enqueue
queue.enqueue(SendEmailJob {
    to: "[email protected]".into(),
    subject: "Welcome!".into(),
    body: "Hello...".into(),
}).await?;

// With options
queue.enqueue_with(
    QueuedJob::new(SendEmailJob { ... })
        .priority(Priority::High)
        .delay(Duration::from_secs(300))  // 5 min delay
        .tag("onboarding")
        .correlation_id("user-123")
).await?;

// Scheduled for specific time
queue.enqueue_at(job, Utc::now() + Duration::hours(24)).await?;

Worker Pool

use arcana_jobs::{WorkerPool, WorkerPoolConfig};

// Create worker pool
let config = WorkerPoolConfig {
    concurrency: 8,
    queues: vec!["critical", "high", "default", "low"],
    job_timeout: Duration::from_secs(300),
    poll_interval: Duration::from_millis(100),
    ..Default::default()
};

let pool = WorkerPool::new(queue, config);

// Register job handlers
pool.register_job::<SendEmailJob>();
pool.register_job::<ProcessPaymentJob>();

// Start processing
pool.start().await?;

Cron Scheduler

use arcana_jobs::{Scheduler, cron_expressions};

let scheduler = Scheduler::new(redis_pool, queue, config);

// Register scheduled jobs
scheduler.schedule(
    "daily-cleanup",
    cron_expressions::DAILY_MIDNIGHT,
    || CleanupJob::new()
)?;

scheduler.schedule(
    "hourly-report",
    "0 0 * * * *",  // Every hour
    || GenerateReportJob::new()
)?;

// Start scheduler (with leader election)
scheduler.start().await?;

Job Queue REST API

Endpoint	Method	Description
`/api/jobs/queues`	GET	List queues with stats
`/api/jobs/queues/:name/stats`	GET	Queue statistics
`/api/jobs/jobs`	GET	Search jobs
`/api/jobs/jobs/:id`	GET	Get job details
`/api/jobs/jobs/:id`	DELETE	Cancel job
`/api/jobs/jobs/:id/retry`	POST	Retry failed job
`/api/jobs/dlq`	GET	List DLQ jobs
`/api/jobs/dlq/:id/retry`	POST	Retry DLQ job
`/api/jobs/dashboard`	GET	Dashboard stats
`/api/jobs/scheduled`	GET	List scheduled jobs

Job Queue Metrics

Metric	Type	Description
`arcana_jobs_enqueued_total`	Counter	Jobs enqueued
`arcana_jobs_completed_total`	Counter	Jobs completed
`arcana_jobs_failed_total`	Counter	Jobs failed
`arcana_jobs_dead_lettered_total`	Counter	Jobs sent to DLQ
`arcana_jobs_pending`	Gauge	Current pending jobs
`arcana_jobs_active`	Gauge	Current running jobs
`arcana_job_duration_seconds`	Histogram	Execution duration
`arcana_workers_active`	Gauge	Active worker count
`arcana_scheduler_is_leader`	Gauge	Scheduler leadership

Kubernetes Deployment

# Deploy job queue components (included in the base kustomization already)
kubectl apply -k deployment/kubernetes/base/jobs/

# Workers autoscale on queue depth via the HPA; to change the bounds:
kubectl edit hpa arcana-job-worker-hpa -n arcana

# Check worker status
kubectl get pods -l component=job-worker

# View metrics (health probes and /metrics share port 8080 on job roles)
kubectl port-forward svc/arcana-job-worker 8080:8080
curl localhost:8080/metrics | grep arcana_jobs

The worker and the scheduler are the same binary as the API pods, selected by ARCANA_DEPLOYMENT__LAYER=worker / scheduler. In monolithic deployments set ARCANA_JOBS__ENABLED=true instead and the one process hosts both in-process.

Server-Side Rendering

sequenceDiagram
    participant Client
    participant Controller
    participant SSR as SSR Engine
    participant Cache
    participant Runtime as JS Runtime Pool

    Client->>Controller: GET /page
    Controller->>SSR: render(template, data)

    SSR->>Cache: Check cache
    alt Cache Hit
        Cache-->>SSR: Cached HTML
    else Cache Miss
        SSR->>Runtime: Acquire runtime
        Runtime-->>SSR: JS Context
        SSR->>Runtime: Execute render
        Runtime-->>SSR: HTML output
        SSR->>Cache: Store result
    end

    SSR-->>Controller: HTML
    Controller-->>Client: 200 OK + HTML

Quick Start

Prerequisites

Rust 1.75+ with cargo
MySQL 8.0+ or PostgreSQL 15+
Docker & Docker Compose (optional)
kubectl + Kubernetes cluster (for K8s deployment)

1. Clone and Configure

git clone https://github.com/jrjohn/arcana-cloud-rust.git
cd arcana-cloud-rust

# Copy and configure environment
cp config/default.toml config/local.toml
# Edit config/local.toml with your database credentials

2. Start Dependencies

# Using Docker Compose
docker-compose -f deployment/monolithic/docker-compose.yml up -d mysql redis

# Or use existing MySQL/PostgreSQL
export ARCANA_DATABASE__URL="mysql://user:pass@localhost:3306/arcana"

3. Run Migrations and Start

# Build and run
cargo run --package arcana-server

# Or with hot reload (requires cargo-watch)
cargo install cargo-watch
cargo watch -x 'run --package arcana-server'

4. Verify

Service	URL	Description
REST API	http://localhost:8080	Main HTTP endpoints
Health Check	http://localhost:8080/health	Service health
gRPC	localhost:9090	gRPC services
Metrics	http://localhost:8080/metrics	Prometheus metrics

# Test health
curl http://localhost:8080/health
# {"status":"healthy","version":"0.1.0"}

# Register user
curl -X POST http://localhost:8080/api/v1/auth/register \
  -H "Content-Type: application/json" \
  -d '{"email":"[email protected]","password":"secure123","name":"Test User"}'

API Endpoints

Authentication

Method	Endpoint	Description
POST	`/api/v1/auth/register`	Register new user
POST	`/api/v1/auth/login`	Login, returns JWT tokens
POST	`/api/v1/auth/refresh`	Refresh access token
POST	`/api/v1/auth/logout`	Invalidate refresh token
GET	`/api/v1/auth/me`	Get current user profile

User Management

Method	Endpoint	Description
GET	`/api/v1/users`	List users (admin only)
GET	`/api/v1/users/:id`	Get user by ID
PUT	`/api/v1/users/:id`	Update user
DELETE	`/api/v1/users/:id`	Delete user

System

Method	Endpoint	Description
GET	`/health`	Health check
GET	`/metrics`	Prometheus metrics
GET	`/api/v1/plugins`	List loaded plugins

gRPC Services

arcana.auth.v1.AuthService - Authentication operations
arcana.user.v1.UserService - User CRUD operations
arcana.health.v1.HealthService - Health checking

Performance Benchmarks

Test Environment: MacBook Pro M3, 50 concurrent connections, MySQL 8.0, Redis 7 Date: December 2024

Rust Monolithic - Real Benchmark Results

HTTP Endpoints (via `hey` / `wrk`)

Endpoint	Requests/sec	Avg Latency	p50	p99	Notes
`GET /health`	47,966	1.04ms	0.94ms	3.40ms	No DB
`GET /api/v1/auth/me`	15,793	3.1ms	2.9ms	6.8ms	JWT + MySQL read
`POST /api/v1/auth/login`	277	176ms	169ms	400ms	MySQL + Argon2¹

¹ Login is intentionally slow due to Argon2 password hashing (security feature)

gRPC Endpoints (via `ghz`)

Method	Requests/sec	Avg Latency	p50	p99	Notes
`Health.Check`	38,307	1.07ms	0.98ms	3.01ms	No DB
`AuthService.ValidateToken`	35,759	1.18ms	1.12ms	2.36ms	JWT only
`AuthService.Login`	284	70ms	67ms	152ms	MySQL + Argon2¹

Protocol Comparison (Monolithic Mode)

Metric	gRPC	HTTP	Improvement
Health Check RPS	38,307	47,966	HTTP 25% faster²
Auth Validate RPS	35,759	15,793	gRPC 2.3x faster
Login RPS	284	277	~Same (Argon2 bound)
Avg Latency (auth)	1.18ms	3.1ms	gRPC 2.6x faster
p99 Latency (auth)	2.36ms	6.8ms	gRPC 2.9x faster

² HTTP health is faster due to simpler connection handling for single requests

Latency Distribution

Authenticated GET User (MySQL Read)
═══════════════════════════════════════════════════════════════════

HTTP   │██████████████████████████████████████████████████│ p50: 2.9ms
(hey)  │████████████████████████████████████████████████████████████████████│ p99: 6.8ms

gRPC   │████████████████████████████████████│ p50: 1.12ms
(ghz)  │██████████████████████████████████████████████│ p99: 2.36ms

Deployment Mode Comparison

Mode	Protocol	Throughput	Avg Latency	p99 Latency
Monolithic	HTTP	47,966 rps	1.04ms	3.40ms
Monolithic	gRPC	38,307 rps	1.07ms	3.01ms
Layered + HTTP	HTTP	~8,000 rps	2.8ms	8.5ms
Layered + gRPC	gRPC	~18,000 rps	1.2ms	4.2ms
K8s + HTTP	HTTP	~7,500 rps	3.2ms	9.8ms
K8s + gRPC	gRPC	~15,000 rps	1.5ms	4.5ms

Cross-Platform Comparison: Rust vs Spring Boot

Benchmark comparison with Arcana Cloud Spring Boot:

Monolithic Mode (Real Tests)

Operation	Rust HTTP	Rust gRPC	Spring Boot HTTP¹	Spring Boot gRPC¹
Health Check	47,966 rps	38,307 rps	~10,000 rps	~15,000 rps
Get User (auth)	15,793 rps	35,759 rps	~3,000 rps	~8,000 rps
Login	277 rps	284 rps	~200 rps	~220 rps

¹ Spring Boot estimates based on documented benchmarks (JVM with G1GC, 512MB heap)

Latency Comparison

Operation	Rust (gRPC)	Spring Boot (gRPC)	Rust Advantage
Health Check	1.07ms	~2.5ms	2.3x faster
Token Validate	1.18ms	~3.0ms	2.5x faster
Get User	2.9ms	~7.5ms	2.6x faster
Login	70ms	~150ms	2.1x faster

Throughput Under Load (50 concurrent connections)

Requests per Second (higher is better)
═══════════════════════════════════════════════════════════════════

Rust HTTP     │████████████████████████████████████████████████████████████████████████████████│ 47,966
Rust gRPC     │██████████████████████████████████████████████████████████████████████████████│ 38,307
Spring gRPC   │████████████████████████████████████████│ ~15,000
Spring HTTP   │██████████████████████████│ ~10,000

Resource Efficiency

Metric	Rust	Spring Boot	Advantage
Memory (idle)	45 MB	150 MB	Rust 3.3x lower
Memory (load)	62 MB	300 MB	Rust 4.8x lower
Startup Time	50ms	2-5s	Rust 40-100x faster
Cold Start	50ms	5-10s	Rust 100-200x faster
Binary Size	25 MB	115 MB	Rust 4.6x smaller

Winner Summary

Aspect	Rust	Spring Boot
Throughput (monolithic)	✅ 3-5x higher
Latency (p50)	✅ 2-3x lower
Latency (p99)	✅ 2-3x lower
Memory Usage	✅ 3-5x lower
Cold Start	✅ 100x faster
Ecosystem Maturity		✅ More libraries
Learning Curve		✅ Easier for Java devs
Development Speed		✅ Faster iteration

When to Choose

Use Case	Recommended	Reason
High-throughput APIs	Rust	3-5x more RPS per instance
Low-latency (< 5ms)	Rust	Consistent sub-2ms p99
Serverless/Lambda	Rust	50ms cold start vs 5s
Memory-constrained	Rust	45MB vs 300MB
Existing Java team	Spring Boot	Faster development
Enterprise integration	Spring Boot	Better ecosystem
Rapid prototyping	Spring Boot	More tooling

Key Recommendations

Scenario	Recommendation	Reason
Internal microservices	gRPC	2.5x faster than HTTP
External/browser APIs	HTTP	Browser compatibility
High concurrency	gRPC	Better connection multiplexing
Development/debugging	HTTP	Easier to inspect
Mobile backends	gRPC	Lower latency, smaller payloads

Configuration

Layered Configuration

Configuration loads from multiple sources in order (later overrides earlier):

config/default.toml - Default values
config/{environment}.toml - Environment-specific (development, production)
config/{deployment_mode}.toml - Deployment mode overrides
config/local.toml - Local overrides (gitignored)
Environment variables with ARCANA_ prefix

Key Environment Variables

# Application
ARCANA_ENVIRONMENT=production
ARCANA_DEPLOYMENT_MODE=layeredgrpc

# Deployment role: all | controller | service | repository | worker | scheduler
ARCANA_DEPLOYMENT__LAYER=all

# Database
#
# Spelling rule: ONE underscore after the ARCANA prefix, TWO between config
# levels. `__` separates levels, not words -- the field is `server.rest_port`,
# so it is ARCANA_SERVER__REST_PORT, not ARCANA_SERVER__REST__PORT. An
# unmatched variable is discarded silently and the process runs on defaults.
ARCANA_DATABASE__URL=mysql://user:pass@host:3306/arcana
ARCANA_DATABASE__MAX_CONNECTIONS=20

# Security
ARCANA_SECURITY__JWT_SECRET=your-256-bit-secret
ARCANA_SECURITY__JWT_ACCESS_EXPIRATION_SECS=3600

# Server
ARCANA_SERVER__REST_PORT=8080
ARCANA_SERVER__GRPC_PORT=9090

# Plugins
ARCANA_PLUGINS__ENABLED=true
ARCANA_PLUGINS__DIRECTORY=./plugins

# Observability
ARCANA_OBSERVABILITY__LOG_LEVEL=info
ARCANA_OBSERVABILITY__LOG_FORMAT=json

Project Structure

arcana-cloud-rust/
├── crates/
│   ├── arcana-core/           # Core types, errors, traits
│   ├── arcana-domain/         # Domain entities, value objects, events
│   ├── arcana-repository/     # Repository implementations (SQLx)
│   ├── arcana-service/        # Business logic services
│   ├── arcana-rest/           # Axum REST controllers
│   ├── arcana-grpc/           # Tonic gRPC services
│   ├── arcana-security/       # JWT, password hashing, RBAC
│   ├── arcana-config/         # Configuration management
│   ├── arcana-plugin-api/     # WASM plugin API
│   ├── arcana-plugin-runtime/ # Wasmtime runtime
│   ├── arcana-ssr-engine/     # Server-side rendering
│   ├── arcana-resilience/     # Circuit breaker, retry, rate limit
│   └── arcana-server/         # Main application entry point
├── plugins/
│   └── arcana-audit-plugin/   # Sample WASM plugin
├── proto/                     # gRPC protobuf definitions
├── config/                    # Configuration files
├── migrations/                # Database migrations
├── deployment/
│   ├── monolithic/            # Docker Compose for monolithic
│   ├── layered/               # Layered deployment Dockerfile
│   └── kubernetes/            # K8s manifests & kustomize
└── docs/                      # Additional documentation

Coding Standard

Clippy is part of the build, not a report. The rules live in Cargo.toml under [workspace.lints], and every crate opts in with [lints] workspace = true.

Lint group	Level	Enforced by
`clippy::all` (default lints)	warn	CI: `cargo clippy --workspace --lib --bins -- -D warnings`
`clippy::pedantic`	warn	same command — any warning fails the build
`unsafe_code`	warn	same command

The Rust version is pinned in rust-toolchain.toml and every FROM rust:<version> uses the same tag, so local runs and CI see exactly the same lints. Renovate bumps them together in one "Rust toolchain" PR; any new lints are fixed in that PR.

Run it locally before pushing:

cargo clippy --workspace --lib --bins -- -D warnings

Exceptions

Allow a lint only on the smallest item that needs it (one function, statement or struct), and always give a reason:

#[allow(clippy::cast_precision_loss, reason = "Redis sorted-set scores are f64; ms timestamps stay exact up to 2^53")]

Never allow a lint for a whole module or crate, and never lower a level in Cargo.toml to get a build green.
The only module-level allows are for code we do not write: tonic-build output (arcana-grpc/src/proto/mod.rs) and shaku module! expansions (arcana-server/src/di.rs).

What the pedantic lints ask for

Every public function that returns Result documents its failures in a # Errors section, and every function that can panic has a # Panics section. Write them from the code, not as filler.
Numeric as casts that can truncate, wrap or lose sign (cast_possible_truncation, cast_possible_wrap, cast_sign_loss) use TryFrom or an explicit saturating conversion instead. Silent wrap-around is a bug.
Builders and pure functions whose result must be used are #[must_use].

SonarQube also imports the Clippy JSON report (sonar.rust.clippyReport.reportPaths), so the same findings show up there.

Testing

Test Summary

Category	Count	Coverage
Unit Tests	120+	Core logic, validation
Integration Tests	30+	Service interactions
Benchmark Tests	6	Serialization performance
Total	150+	-

Running Tests

# Run all tests
cargo test

# Run with output
cargo test -- --nocapture

# Run specific crate tests
cargo test --package arcana-service

# Run benchmarks
cargo bench --package arcana-server

# Generate coverage report (requires cargo-tarpaulin)
cargo install cargo-tarpaulin
cargo tarpaulin --out Html
open tarpaulin-report.html

Test Coverage by Feature

Feature	Tests	Status
Authentication	16	✅
User Management	19	✅
Authorization (RBAC)	4	✅
Circuit Breaker	2	✅
Rate Limiting	2	✅
Configuration	2	✅
Domain Events	18	✅
DTOs & Validation	32	✅
Entity Operations	8	✅

Documentation

Benchmark Results - gRPC vs HTTP comparison
Kubernetes Deployment - K8s deployment guide
Plugin Development - Creating WASM plugins
API Reference - Complete API documentation

Architecture Evaluation

Scorecard

Category	Score	Notes
Clean Architecture Adherence	9.5/10	Clear layer separation, dependency inversion
Type Safety & Memory Safety	9.5/10	Rust eliminates entire bug classes
Performance	9.5/10	Zero-cost abstractions, 2.1x gRPC speedup
Dual Protocol Support	9.0/10	REST + gRPC with seamless switching
Security Implementation	9.0/10	JWT, Argon2, RBAC, mTLS support
Resilience Patterns	9.0/10	Circuit breaker, retry, rate limiting
Caching Infrastructure	9.0/10	Redis with transparent cache-aside
Job Queue System	9.0/10	Priority queues, scheduling, workers
Dependency Injection	9.0/10	Compile-time verified with Shaku
Configuration Management	9.0/10	Layered TOML with env override
Deployment Flexibility	9.0/10	5 modes, K8s native
Observability	8.5/10	OpenTelemetry, Prometheus, tracing
Plugin System	8.5/10	WASM sandboxing, innovative approach
Testing Coverage	8.5/10	329+ tests, criterion benchmarks
API Documentation	8.5/10	Swagger UI auto-generated
Code Quality	9.0/10	Type-safe, no unsafe code
Overall	8.95/10	⭐⭐⭐⭐⭐ Enterprise-Grade

Strengths (Ranked by Impact)

🥇 Tier 1: Critical Advantages

Rank	Strength	Impact	Description
1	Type & Memory Safety	🔴 Critical	Rust eliminates null pointers, data races, buffer overflows at compile time
2	Performance	🔴 Critical	Zero-cost abstractions, no GC pauses, 2.1x faster than JSON/HTTP
3	Clean Architecture	🔴 Critical	Strict layer separation enables independent testing and deployment
4	Comprehensive Security	🔴 Critical	JWT + Argon2 + RBAC + mTLS built-in from day one

🥈 Tier 2: Major Advantages

Rank	Strength	Impact	Description
5	Dual Protocol	🟠 High	REST for browsers, gRPC for services - best of both worlds
6	Production Resilience	🟠 High	Circuit breaker, retry with backoff, rate limiting prevent cascading failures
7	Redis Caching	🟠 High	Transparent cache-aside pattern with TTL, invalidation on writes
8	Job Queue System	🟠 High	Priority-based processing, scheduling, DLQ, worker registry
9	Compile-Time DI	🟠 High	Shaku verifies all dependencies at compile time, zero runtime overhead

🥉 Tier 3: Valuable Advantages

Rank	Strength	Impact	Description
10	Cloud-Native	🟡 Medium	5 deployment modes, K8s manifests with HPA, PDB, network policies
11	Observability	🟡 Medium	OpenTelemetry traces, Prometheus metrics, structured JSON logging
12	WASM Plugins	🟡 Medium	Sandboxed extensibility without recompilation
13	OpenAPI/Swagger	🟡 Medium	Auto-generated API docs with utoipa
14	Request Validation	🟡 Medium	ValidatedJson extractor returns 422 with field-level errors

Weaknesses (Ranked by Impact)

⚠️ Tier 1: Significant Considerations

Rank	Weakness	Impact	Mitigation
1	Learning Curve	🔴 High	Rust ownership/borrowing takes weeks to internalize
2	Compile Times	🔴 High	Full rebuild: ~3-5 minutes, incremental: 5-15s
3	Async Debugging	🟠 Medium	Stack traces in async Rust can be cryptic

⚠️ Tier 2: Moderate Considerations

Rank	Weakness	Impact	Mitigation
4	Ecosystem Maturity	🟠 Medium	Some async crates still stabilizing
5	gRPC Browser Support	🟠 Medium	Browsers can't use gRPC directly
6	No Hot Reload	🟠 Medium	Code changes require recompilation
7	Infrastructure Needs	🟠 Medium	Full features require Redis + MySQL

⚠️ Tier 3: Minor Considerations

Rank	Weakness	Impact	Mitigation
8	13 Crates Complexity	🟡 Low	May be overkill for small projects
9	WASM Component Model	🟡 Low	Spec still evolving
10	Memory for Pools	🟡 Low	Connection pools consume memory

Trade-off Analysis

                    FLEXIBILITY
                         │
                         │    ┌─────────────────────┐
                         │    │ Arcana Cloud Rust   │
                         │    │ ★ Clean Arch        │
                         │    │ ★ Dual Protocol     │
                         │    │ ★ 5 Deploy Modes    │
         ┌───────────────┼────┴─────────────────────┤
         │               │                          │
         │   Spring Boot │                          │
         │               │                          │
         │               │                          │
 ◄───────┼───────────────┼──────────────────────────┼───────────► PERFORMANCE
  Slower │               │                          │            Faster
         │               │                          │
         │      Go       │                          │
         │   Microsvcs   │                          │
         │               │                          │
         └───────────────┼──────────────────────────┘
                         │
                         │
                    SIMPLICITY

Dimension	Arcana Cloud Rust	Go Microservices	Spring Boot
Performance	⭐⭐⭐⭐⭐ (Best)	⭐⭐⭐⭐	⭐⭐⭐
Type Safety	⭐⭐⭐⭐⭐ (Best)	⭐⭐⭐	⭐⭐⭐⭐
Learning Curve	⭐⭐ (Steepest)	⭐⭐⭐⭐	⭐⭐⭐
Ecosystem	⭐⭐⭐	⭐⭐⭐⭐	⭐⭐⭐⭐⭐
Memory Efficiency	⭐⭐⭐⭐⭐ (Best)	⭐⭐⭐⭐	⭐⭐⭐
Compile Speed	⭐⭐ (Slowest)	⭐⭐⭐⭐⭐	⭐⭐⭐⭐

When to Use This Architecture

✅ Ideal For

High-performance microservices requiring sub-2ms latency
Security-critical applications (fintech, healthcare, government)
Long-running services where memory efficiency matters
Teams with Rust experience or willingness to invest in learning
Polyglot environments needing REST + gRPC interoperability
Cloud-native deployments with Kubernetes

⚠️ Consider Alternatives For

Rapid prototyping where development speed is paramount
Teams without Rust experience and tight deadlines
Simple CRUD applications where this is overkill
Browser-only backends where gRPC benefits are minimal

Strengths Summary

Zero-Cost Abstractions: Rust's compile-time guarantees with no runtime overhead
Type-Safe gRPC: Protocol Buffers with compile-time verification
WASM Plugin Sandboxing: Secure, isolated plugin execution
Memory Safety: No garbage collection, predictable performance
Async-First: Tokio-based async runtime for high concurrency
Comprehensive Resilience: Built-in patterns for production reliability
Multi-Database Support: MySQL and PostgreSQL via SQLx
Production-Ready Security: JWT, Argon2, RBAC, mTLS
Cloud-Native: Kubernetes manifests with HPA, PDB, Network Policies
Transparent Caching: Redis cache-aside with automatic invalidation
Background Jobs: Priority queues, workers, scheduling, DLQ
API Documentation: OpenAPI/Swagger auto-generated from code

Architecture Patterns

Clean Architecture: Strict layer separation with dependency inversion
Repository Pattern: Abstract data access with trait-based interfaces
CQRS-Ready: Separated command/query DTOs
Domain Events: Event-driven communication between bounded contexts
Circuit Breaker: Fail-fast with automatic recovery
Rate Limiting: Token bucket algorithm with governor
Plugin Architecture: WASM-based extensibility
Dual Protocol: REST for external, gRPC for internal

Technology Stack

Component	Technology	Version
Language	Rust	1.75+
Async Runtime	Tokio	1.52
REST Framework	Axum	0.8
gRPC Framework	Tonic	0.14
Database	SQLx	0.9
WASM Runtime	Wasmtime	49
JWT	jsonwebtoken	11.0
Password Hashing	Argon2	0.6
Rate Limiting	Governor	0.10
Serialization	Serde	1.0
Logging	Tracing	0.1
Metrics	Prometheus	0.18
Middleware	Tower	0.5

License

MIT License - see LICENSE file for details.

Documentation | Issues | Contributing

Built with Rust for performance, safety, and reliability.

Name		Name	Last commit message	Last commit date
.github/workflows		.github/workflows
config		config
crates		crates
deployment		deployment
migrations		migrations
plugins/arcana-audit-plugin		plugins/arcana-audit-plugin
proto		proto
scripts		scripts
.dockerignore		.dockerignore
.gitignore		.gitignore
.pr_agent.toml		.pr_agent.toml
.release-please-manifest.json		.release-please-manifest.json
CHANGELOG.md		CHANGELOG.md
Cargo.lock		Cargo.lock
Cargo.toml		Cargo.toml
Dockerfile.ci		Dockerfile.ci
Dockerfile.coverage		Dockerfile.coverage
Jenkinsfile		Jenkinsfile
README.md		README.md
docker-compose.ci.yml		docker-compose.ci.yml
docker-compose.coverage.yml		docker-compose.coverage.yml
docker-compose.test.yml		docker-compose.test.yml
release-please-config.json		release-please-config.json
renovate.json		renovate.json
rust-toolchain.toml		rust-toolchain.toml
sonar-project.properties		sonar-project.properties

Latest commit

History

Folders and files

Repository files navigation

Arcana Cloud Rust

Architecture

Key Features

Deployment Modes

Deployment Mode Comparison

Plugin System (WASM)

Plugin Architecture

Plugin Lifecycle

Extension Points

Creating a Plugin

Distributed Job Queue

Job Queue Architecture

Job Queue Features

Defining a Job

Enqueuing Jobs

Worker Pool

Cron Scheduler

Job Queue REST API

Job Queue Metrics

Kubernetes Deployment

Server-Side Rendering

Quick Start

Prerequisites

1. Clone and Configure

2. Start Dependencies

3. Run Migrations and Start

4. Verify

API Endpoints

Authentication

User Management

System

gRPC Services

Performance Benchmarks

Rust Monolithic - Real Benchmark Results

HTTP Endpoints (via hey / wrk)

gRPC Endpoints (via ghz)

Protocol Comparison (Monolithic Mode)

Latency Distribution

Deployment Mode Comparison

Cross-Platform Comparison: Rust vs Spring Boot

Monolithic Mode (Real Tests)

Latency Comparison

Throughput Under Load (50 concurrent connections)

Resource Efficiency

Winner Summary

When to Choose

Key Recommendations

Configuration

Layered Configuration

Key Environment Variables

Project Structure

Coding Standard

Testing

Test Summary

Running Tests

Test Coverage by Feature

Documentation

Architecture Evaluation

Scorecard

Strengths (Ranked by Impact)

🥇 Tier 1: Critical Advantages

🥈 Tier 2: Major Advantages

🥉 Tier 3: Valuable Advantages

Weaknesses (Ranked by Impact)

⚠️ Tier 1: Significant Considerations

⚠️ Tier 2: Moderate Considerations

⚠️ Tier 3: Minor Considerations

Trade-off Analysis

When to Use This Architecture

✅ Ideal For

⚠️ Consider Alternatives For

Strengths Summary

Architecture Patterns

Technology Stack

License

About

HTTP Endpoints (via `hey` / `wrk`)

gRPC Endpoints (via `ghz`)