Documentation

tutorials/superset-setup.md

Apache Superset Analytics for Acsis Dynaplex Platform

Overview

Apache Superset is integrated into the Dynaplex platform to provide powerful data analytics and visualization capabilities. Superset runs as a containerized service alongside your other components and has direct access to your application database.

Quick Start

1. Enable Analytics in Your AppHost

Add Superset to your foundation configuration:

var foundation = builder.AddFoundation()
    .WithDatabase()
    .WithAnalytics()  // 🎉 Adds Superset + Redis
    // ... register your components
    .Build();

foundation.Run();

2. Start Aspire

# From repository root
dotnet run --project projects/bbu-rfid/src/Acsis.Dynaplex.Projects.BbuRfid/Acsis.Dynaplex.Projects.BbuRfid.csproj

3. Access Superset

Note: The local Superset port is dynamically assigned by Aspire to enable multiple simultaneous instances.

To find the current Superset URL:

  1. Open the Aspire Dashboard (URL shown when AppHost starts)
  2. Find the "superset" resource
  3. Click on its endpoint to open Superset
  • Local Development: Check Aspire Dashboard for current port
  • Azure Deployment: Check Azure Container App URL in Aspire dashboard

4. Login

Default Credentials:

  • Username: admin
  • Password: admin

⚠️ IMPORTANT: Change the default password immediately in production!

Architecture

Local Development

┌─────────────────────────────────────────────┐
│              Aspire AppHost                 │
├─────────────────────────────────────────────┤
│                                             │
│  ┌──────────┐  ┌──────────┐  ┌──────────┐  │
│  │PostgreSQL│  │  Redis   │  │ Superset │  │
│  │          │  │          │  │          │  │
│  │ acsis db │◄─┤  cache   │◄─┤  :8088   │  │
│  │ ss schema│  │          │  │          │  │
│  └──────────┘  └──────────┘  └──────────┘  │
│       ▲                          │          │
│       │                          ▼          │
│  ┌────┴─────────────────────────────────┐  │
│  │ Component Services (CoreData, etc.)  │  │
│  └──────────────────────────────────────┘  │
└─────────────────────────────────────────────┘

Azure Deployment

┌─────────────────────────────────────────────┐
│         Azure Container Apps                │
├─────────────────────────────────────────────┤
│                                             │
│  ┌──────────────┐  ┌──────────┐            │
│  │  PostgreSQL  │  │  Redis   │            │
│  │   Flexible   │  │  Cache   │            │
│  │   Server     │◄─┤  (Basic) │            │
│  │              │  │          │            │
│  └──────────────┘  └──────────┘            │
│         ▲               ▲                   │
│         │               │                   │
│  ┌──────┴───────────────┴────────────────┐ │
│  │  Superset Container App               │ │
│  │  (ca-superset-{group})                │ │
│  │  Public ingress enabled               │ │
│  └───────────────────────────────────────┘ │
└─────────────────────────────────────────────┘

Database Schema

Superset stores its metadata (dashboards, users, permissions, etc.) in the ss schema of your main acsis database:

  • Database: acsis (shared with all Dynaplex components)
  • Schema: ss (isolated Superset metadata)
  • Application Data: public schema (accessible for analytics)

This approach:

  • ✅ Simplifies deployment (no separate database needed)
  • ✅ Isolates Superset tables from application data
  • ✅ Allows Superset to query all component data directly

Connecting to Application Data

Step 1: Add Database Connection

  1. Navigate to Settings → Database Connections
  2. Click + Database
  3. Select PostgreSQL
  4. Configure connection:

Local Development:

Host: postgres
Port: 5432
Database: acsis
Username: postgres
Password: postgres
Display Name: Acsis Application Data

Azure Deployment:

Host: {your-postgres-server}.postgres.database.azure.com
Port: 5432
Database: acsis
Username: postgres
Password: {your-password}
SSL: Required
Display Name: Acsis Application Data

Step 2: Test Connection

Click Test Connection to verify Superset can access the database.

Step 3: Explore Data

  1. Navigate to SQL Lab → SQL Editor
  2. Select your "Acsis Application Data" database
  3. Browse schemas and tables
  4. Run test queries

Creating Your First Dashboard

1. Create a Dataset

  1. Go to Data → Datasets
  2. Click + Dataset
  3. Select:
    • Database: Acsis Application Data
    • Schema: public (or your component's schema)
    • Table: Choose a table (e.g., users, assets, etc.)
  4. Click Add

2. Create a Chart

  1. Go to Charts → + Chart
  2. Select your dataset
  3. Choose a chart type (e.g., Table, Bar Chart, Line Chart)
  4. Configure:
    • Metrics: What to measure (COUNT, SUM, AVG, etc.)
    • Dimensions: How to group/filter data
    • Filters: Narrow down data
  5. Click Run Query to preview
  6. Click Save and name your chart

3. Build a Dashboard

  1. Go to Dashboards → + Dashboard
  2. Name your dashboard
  3. Drag and drop charts onto the canvas
  4. Resize and arrange as needed
  5. Add filters for interactivity
  6. Click Save

Configuration Files

Superset configuration is stored in docs/internal/implementation-plans/superset/:

superset/
├── config.py              # Main Superset configuration
├── init-schema.sql        # PostgreSQL schema creation
├── init-podman.sh         # Manual initialization script
├── docker-entrypoint.sh   # Container auto-init entrypoint
└── docker-compose.yml     # Standalone docker compose (reference)

Key Configuration Settings

config.py highlights:

# Database connection (auto-configured via environment)
SQLALCHEMY_DATABASE_URI = postgresql://postgres@postgres:5432/acsis

# Schema isolation
SQLALCHEMY_ENGINE_OPTIONS = {
    'connect_args': {
        'options': '-csearch_path=ss,public'
    }
}

# Redis caching
CACHE_CONFIG = {
    'CACHE_TYPE': 'RedisCache',
    'CACHE_REDIS_URL': 'redis://superset-redis:6379/0'
}

# Performance tuning
ROW_LIMIT = 50000
SUPERSET_WEBSERVER_TIMEOUT = 300

Manual Initialization

If automatic initialization fails, you can manually initialize Superset:

# Get the Superset container ID
docker ps | grep superset

# Run initialization script inside container
docker exec <container-id> /app/pythonpath/init-podman.sh

# Or step by step:
docker exec <container-id> superset db upgrade
docker exec <container-id> superset fab create-admin --username admin --password admin --email admin@acsis.local
docker exec <container-id> superset init

Troubleshooting

Superset won't start

Check logs:

# Via Aspire Dashboard
# Navigate to Superset resource → View Logs

# Or via Docker
docker logs <superset-container-id>

Common issues:

  1. PostgreSQL not ready: Superset waits up to 60s, check DB is running
  2. Redis not available: Check redis container is running
  3. Schema doesn't exist: Manually run init-schema.sql on your database

Can't connect to database

Verify connection:

# Test PostgreSQL connection from Superset container
docker exec <superset-container-id> psql -h postgres -U postgres -d acsis -c '\dt ss.*'

Check environment variables:

docker exec <superset-container-id> env | grep DATABASE

Charts are slow

  1. Enable caching: Already configured via Redis
  2. Check indexes: Ensure your database tables have proper indexes
  3. Limit row count: Adjust ROW_LIMIT in config.py
  4. Use aggregations: Pre-aggregate data in views

Can't login

Reset admin password:

docker exec -it <superset-container-id> superset fab reset-password --username admin

Schema changes not visible

Sync metadata:

  1. Go to Data → Databases
  2. Click your database
  3. Click Sync columns from source (in Advanced settings)

Advanced Configuration

Custom Admin User

Set environment variables before starting:

.WithEnvironment("SUPERSET_ADMIN_USERNAME", "your-username")
.WithEnvironment("SUPERSET_ADMIN_PASSWORD", "your-secure-password")
.WithEnvironment("SUPERSET_ADMIN_EMAIL", "you@company.com")

Enable CORS (for embedding)

Edit config.py:

ENABLE_CORS = True
CORS_OPTIONS = {
    'origins': ['https://your-app.com'],
    'supports_credentials': True
}

Increase Worker Count (performance)

For Azure Container Apps:

.WithEnvironment("SUPERSET_WORKERS", "8")  // More workers for high load
.WithEnvironment("SUPERSET_THREADS", "40") // More threads per worker

SSL/TLS for PostgreSQL (Azure)

Azure PostgreSQL requires SSL. The connection will automatically use SSL when connecting to Azure:

# config.py automatically handles this
SQLALCHEMY_ENGINE_OPTIONS = {
    'connect_args': {
        'sslmode': 'require'  # Added automatically for Azure
    }
}

Security Best Practices

  1. Change default password: First thing after deployment
  2. Enable authentication: Configure OAuth, LDAP, or SAML in production
  3. Restrict access: Use Superset's role-based access control (RBAC)
  4. Use secrets: Store passwords in Azure Key Vault or environment variables
  5. Enable HTTPS: Use Azure Container Apps with custom domains
  6. Regular backups: Backup the ss schema for disaster recovery

Performance Tuning

Database Query Optimization

  1. Add indexes: On frequently queried columns
  2. Create views: Pre-aggregate common queries
  3. Partition tables: For large time-series data
  4. Use materialized views: For complex aggregations

Superset Optimization

  1. Redis caching: Already configured, adjust timeouts in config.py
  2. Async queries: Enable for long-running queries
  3. SQL Lab: Limit result row count
  4. Dashboard filters: Use parameter caching

Monitoring

Via Aspire Dashboard

  1. Resource status: Check Superset health in resources list
  2. Logs: View real-time logs for errors
  3. Metrics: CPU, memory usage

Superset Internal Metrics

  1. Navigate to Settings → List Users to see activity
  2. Check SQL Lab → Query History for slow queries
  3. Use Chart → Run History to find performance issues

Integration with Other Tools

Export Data

Superset can export:

  • Charts: PNG, CSV, JSON
  • Dashboards: PDF (requires additional setup)
  • Datasets: CSV, Excel

Programmatic Access

Use Superset's REST API:

# Get auth token
curl -X POST http://localhost:8088/api/v1/security/login \
  -H "Content-Type: application/json" \
  -d '{"username":"admin","password":"admin","provider":"db"}'

# List dashboards
curl -X GET http://localhost:8088/api/v1/dashboard/ \
  -H "Authorization: Bearer <token>"

Component-Specific Analytics Guides

Once you have Superset set up, check out these component-specific guides for pre-built views and dashboards:

  • BBU Superset Analytics - BBU RFID pipeline analytics with Oracle ingestion tracking, OBLPN shipment views, and component breakdown dashboards

Resources

Support

For issues specific to the Dynaplex integration:

  1. Check this documentation
  2. Review container logs in Aspire Dashboard
  3. Verify database connectivity
  4. Check configuration files in docs/internal/implementation-plans/superset/

For general Superset questions: