Configuration

Mon, 01 Jan 0001 00:00:00 +0000

This document details various configuration parameters of the DataMate system.

Environment Variables

Common Configuration

Variable	Default	Description
`DB_PASSWORD`	`password`	Database password
`DATAMATE_JWT_ENABLE`	`false`	Enable JWT authentication
`REGISTRY`	`ghcr.io/modelengine-group/`	Image registry
`VERSION`	`latest`	Image version tag

Database Configuration

Variable	Default	Description
`DB_HOST`	`datamate-database`	Database host
`DB_PORT`	`5432`	Database port
`DB_NAME`	`datamate`	Database name
`DB_USER`	`postgres`	Database username
`DB_PASSWORD`	`password`	Database password

Redis Configuration

Variable	Default	Description
`REDIS_HOST`	`datamate-redis`	Redis host
`REDIS_PORT`	`6379`	Redis port
`REDIS_PASSWORD`	-	Redis password (optional)
`REDIS_DB`	`0`	Redis database number

Milvus Configuration

Variable	Default	Description
`MILVUS_HOST`	`milvus`	Milvus host
`MILVUS_PORT`	`19530`	Milvus port
`MILVUS_INDEX_TYPE`	`IVF_FLAT`	Vector index type
`MILVUS_EMBEDDING_DIM`	`768`	Vector dimension

MinIO Configuration

Variable	Default	Description
`MINIO_ENDPOINT`	`minio:9000`	MinIO endpoint
`MINIO_ACCESS_KEY`	`minioadmin`	Access key
`MINIO_SECRET_KEY`	`minioadmin`	Secret key
`MINIO_BUCKET`	`datamate`	Bucket name

LLM Configuration

Variable	Default	Description
`OPENAI_API_KEY`	-	OpenAI API key
`OPENAI_BASE_URL`	`https://api.openai.com/v1`	API base URL
`OPENAI_MODEL`	`gpt-4`	Model to use

JWT Configuration

Variable	Default	Description
`JWT_SECRET`	`default-insecure-key`	JWT secret (CHANGE IN PRODUCTION)
`JWT_EXPIRATION`	`86400`	Token expiration (seconds)

Logging Configuration

Variable	Default	Description
`LOG_LEVEL`	`INFO`	Log level
`LOG_PATH`	`/var/log/datamate`	Log path

application.yml Configuration

Main Config

datamate:
 jwt:
 enable: ${DATAMATE_JWT_ENABLE:false}
 secret: ${JWT_SECRET:default-insecure-key}
 expiration: ${JWT_EXPIRATION:86400}

 storage:
 type: minio
 endpoint: ${MINIO_ENDPOINT:minio:9000}
 access-key: ${MINIO_ACCESS_KEY:minioadmin}
 secret-key: ${MINIO_SECRET_KEY:minioadmin}

Spring Boot Config

spring:
 datasource:
 url: jdbc:postgresql://${DB_HOST:datamate-database}:${DB_PORT:5432}/${DB_NAME:datamate}
 username: ${DB_USER:postgres}
 password: ${DB_PASSWORD:password}

 jpa:
 hibernate:
 ddl-auto: validate
 show-sql: false

server:
 port: 8092

Docker Compose Configuration

Environment Variables

services:
 datamate-backend:
 environment:
 - DB_PASSWORD=${DB_PASSWORD:-password}
 - LOG_LEVEL=${LOG_LEVEL:-INFO}

Resource Limits

services:
 datamate-backend:
 deploy:
 resources:
 limits:
 cpus: '2'
 memory: 4G

Kubernetes Configuration

ConfigMap

apiVersion: v1
kind: ConfigMap
metadata:
 name: datamate-config
data:
 LOG_LEVEL: "INFO"

Secret

apiVersion: v1
kind: Secret
metadata:
 name: datamate-secret
type: Opaque
data:
 DB_PASSWORD: cGFzc3dvcmQ=  # base64 encoded

Performance Tuning

Database Connection Pool

spring:
 datasource:
 hikari:
 maximum-pool-size: 20
 minimum-idle: 5
 connection-timeout: 30000

JVM Parameters

JAVA_OPTS="-Xms2g -Xmx4g -XX:+UseG1GC"

Installation Guide - Deployment configuration
Troubleshooting - Common issues

Troubleshooting

Mon, 01 Jan 0001 00:00:00 +0000

This document provides troubleshooting steps and solutions for common DataMate issues.

Service Startup Issues

Service Won’t Start

Symptoms

Service fails to start or exits immediately after running make install.

Troubleshooting Steps

Check Port Conflicts

# Check port usage
lsof -i :8080 # API Gateway
lsof -i :30000 # Frontend

If port is occupied:

# Kill process
kill -9 <PID>

View Container Logs

# View all containers
docker ps -a

# View specific container logs
docker logs datamate-backend

Check Docker Resources

# View Docker system info
docker system df

# Clean unused resources
docker system prune -a

Common Causes and Solutions

Cause	Solution
Port occupied	Kill process or modify port mapping
Insufficient memory	Increase Docker memory limit
Image not pulled	Run `docker pull`
Network issues	Check firewall and network config

Container Exits Immediately

Troubleshooting

# View exit code
docker ps -a

# View detailed logs
docker logs <container-name> --tail 100

Database Connection Issues

Cannot Connect to Database

Troubleshooting Steps

Check Database Container

docker ps | grep datamate-database
docker logs datamate-database

Test Database Connection

# Enter database container
docker exec -it datamate-database psql -U postgres -d datamate

Check Database Config

# Check environment variables
docker exec datamate-backend env | grep DB_

Frontend Issues

Frontend Not Accessible

Symptoms

Browser cannot access http://localhost:30000

Appendix on DataMate

Configuration

Environment Variables

Common Configuration

Database Configuration

Redis Configuration

Milvus Configuration

MinIO Configuration

LLM Configuration

JWT Configuration

Logging Configuration

application.yml Configuration

Main Config

Spring Boot Config

Docker Compose Configuration

Environment Variables

Resource Limits

Kubernetes Configuration

ConfigMap

Secret

Performance Tuning

Database Connection Pool

JVM Parameters

Related Documentation

Troubleshooting

Service Startup Issues

Service Won’t Start

Symptoms

Troubleshooting Steps

Common Causes and Solutions

Container Exits Immediately

Troubleshooting

Database Connection Issues

Cannot Connect to Database

Troubleshooting Steps

Frontend Issues

Frontend Not Accessible

Symptoms