mwitnessing/_deploy

Deployment Guide for MWitnessing Application

This guide covers four deployment methods:

1. Standard Docker Deployment - Builds a complete Docker image with your code
2. Runtime Git Clone Deployment - Clones the repository at container startup (current method)
3. Webhook-Based Deployment - Automatically deploys when code is pushed to the repository
4. Registry-Based Deployment - Builds and pushes images to a private Docker registry

Method 1: Standard Docker Deployment (Recommended)

This method builds a complete Docker image containing your application code, providing:

• Faster container startup
• No dependency on Git repository during runtime
• Better security (no Git credentials in environment variables)
• Standard Docker deployment workflow

Prerequisites

• Docker and Docker Compose installed on the host machine
• Access to the application source code

Deployment Steps

1. Build and start the containers:

   cd /path/to/project
   docker-compose -f _deploy/standard-docker-compose.yml up -d
   

2. Access the application:

   • Web application: http://localhost:3000
   • Database admin: http://localhost:8080

3. Update the application:

   # Pull the latest code
   git pull
   
   # Rebuild and restart
   docker-compose -f _deploy/standard-docker-compose.yml up -d --build
   

Configuration

Edit the environment variables in _deploy/standard-docker-compose.yml before deployment:

• DATABASE_URL: Database connection string
• NEXTAUTH_URL: Public URL of your application
• NEXTAUTH_SECRET: Secret for NextAuth.js encryption

Persistent Data

The following Docker volumes are used to persist data:

• app_permits: Stores permit files
• app_uploads: Stores uploaded files
• app_logs: Stores application logs
• db_data: Stores database data

Method 2: Runtime Git Clone Deployment (Legacy)

This method clones the repository at container startup, which is the current deployment method.

Prerequisites

• Docker and Docker Compose installed on the host machine
• Git repository access credentials

Deployment Steps

1. Configure environment variables: Edit the Docker Compose file (e.g., _deploy/deoloy.azure.production.yml) to set:

   • GIT_USERNAME: Username for Git repository access
   • GIT_PASSWORD: Password for Git repository access
   • GIT_BRANCH: Branch to deploy (default: main)
   • UPDATE_CODE_FROM_GIT: Set to "true" to enable Git clone

2. Start the containers:

   docker-compose -f _deploy/deoloy.azure.production.yml up -d
   

3. Update the application:

   • Updates happen automatically when containers restart
   • Force an update by restarting the container:

     docker-compose -f _deploy/deoloy.azure.production.yml restart nextjs-app
     

How It Works

The entrypoint.sh script:

1. Clones the repository at container startup
2. Backs up important directories like permits and uploads
3. Syncs new code from the repository
4. Restores or merges the backed-up directories
5. Rebuilds the application

Method 3: Webhook-Based Deployment (Recommended for Production)

This method uses Gitea webhooks to automatically deploy changes when code is pushed to the repository, providing:

• Automated continuous deployment
• No manual intervention needed for updates
• Immediate deployment of changes
• Preservation of user-uploaded content

Prerequisites

• Docker and Docker Compose installed on the host machine
• Gitea repository with webhook capabilities
• Server with port 9000 (or your configured port) accessible to Gitea

Setup Steps

1. Build and start the webhook service:

   cd /path/to/project
   mkdir -p _deploy/webhook
   cp _deploy/webhook-receiver.js _deploy/webhook/
   cp _deploy/webhook-deploy.sh _deploy/webhook/
   docker-compose -f _deploy/webhook-docker-compose.yml up -d
   

2. Configure Gitea webhook:

   • Go to your repository settings in Gitea
   • Navigate to "Webhooks" > "Add Webhook" > "Gitea"
   • Set Target URL: http://your-server-ip:9000/webhook
   • Set Content Type: application/json
   • Set Secret: Same as WEBHOOK_SECRET in webhook-docker-compose.yml
   • Select "Just the push event"
   • Select "Active"
   • Click "Add Webhook"

3. Test the webhook:

   • Make a change to your repository and push it
   • The webhook service will receive the event and trigger a deployment
   • Check logs: docker logs mwitnessing-webhook

Configuration

Edit the environment variables in _deploy/webhook-docker-compose.yml before deployment:

• WEBHOOK_SECRET: Secret for authenticating webhook requests (must match Gitea webhook secret)
• ALLOWED_BRANCHES: Comma-separated list of branches that trigger deployments
• ALLOWED_REPOSITORIES: Comma-separated list of repositories to monitor
• GIT_USERNAME and GIT_PASSWORD: Credentials for Git repository access

How It Works

1. Webhook Receiver:

   • Listens for push events from Gitea
   • Validates the webhook signature using the shared secret
   • Triggers the deployment script when a valid push event is received

2. Deployment Script:

   • Clones the repository at the specified branch
   • Checks if dependencies have changed (package.json, Dockerfile, etc.)
   • Performs a full rebuild if necessary, or just updates code and restarts
   • Preserves user-uploaded content during deployment

3. Intelligent Updates:

   • Intelligently determines whether a full rebuild is needed
   • Only rebuilds Docker images when necessary (package.json changes, etc.)
   • Preserves data in volumes across deployments

Monitoring and Maintenance

• View logs:

  docker logs mwitnessing-webhook
  

• Manual trigger:

  docker exec mwitnessing-webhook /app/webhook-deploy.sh main
  

• Update webhook code:

  # Edit webhook files, then:
  docker-compose -f _deploy/webhook-docker-compose.yml up -d --build
  

Method 4: Registry-Based Deployment

This method involves building Docker images locally or in a CI/CD pipeline, pushing them to your private Docker registry (docker.d-popov.com), and then deploying using those pre-built images.

Prerequisites

• Docker and Docker Compose installed on the build machine
• Access to your private Docker registry at docker.d-popov.com
• Authentication configured for the Docker registry

Build and Push Process

1. Log in to the Docker registry:

   docker login docker.d-popov.com -u <username> -p <password>
   

2. Build the Docker image:

   # Navigate to the project root
   cd /path/to/project
   
   # Build the image with the jwpw tag
   docker build -t docker.d-popov.com/jwpw:latest -f _deploy/prod.Dockerfile .
   

3. Push the image to the registry:

   docker push docker.d-popov.com/jwpw:latest
   

4. Tag with version (optional but recommended):

   # Tag with version number
   docker tag docker.d-popov.com/jwpw:latest docker.d-popov.com/jwpw:v1.0.0
   
   # Push the versioned tag
   docker push docker.d-popov.com/jwpw:v1.0.0
   

Deployment Process

1. On the server, create or update your deployment YAML file: Create or edit a file like _deploy/deploy.production.yml:

   version: "3"
   services:
     nextjs-app:
       hostname: jwpw-app-production
       image: docker.d-popov.com/jwpw:latest
       volumes:
         - /mnt/docker_volumes/pw-prod/app/public/content/uploads/:/app/public/content/uploads
         - /mnt/docker_volumes/pw-prod/app/logs:/app/logs
         - /mnt/docker_volumes/pw-prod/app/public/content/permits/:/app/public/content/permits
       environment:
         - APP_ENV=production
         - NODE_ENV=production
         - TZ=Europe/Sofia
         - DATABASE=mysql://username:password@db:3306/database_name
       restart: always
       networks:
         - infrastructure_default
         - default
     # Add database service if needed
   

2. Deploy using Docker Compose:

   docker-compose -f _deploy/deploy.production.yml up -d
   

3. Update the deployment (when a new image is available):

   # Pull the latest image
   docker pull docker.d-popov.com/jwpw:latest
   
   # Restart the service
   docker-compose -f _deploy/deploy.production.yml up -d
   

Continuous Integration/Continuous Deployment (CI/CD)

This method is ideal for integration with CI/CD pipelines. For example, you could set up a GitHub Actions or GitLab CI pipeline that:

1. Builds the Docker image
2. Runs tests
3. Pushes the image to your private registry
4. Triggers a deployment on your server

Advantages of Registry-Based Deployment

• Consistent environments: The exact same image is used in all environments
• Faster deployments: No building on the production server
• Rollback capability: Easy to roll back to a previous version by specifying an older tag
• Better separation of concerns: Build and deployment processes are separated
• Audit trail: Registry maintains a history of image versions

Current Use

This method is currently used for your staging environment, as seen in _deploy/deoloy.azure.staging.yml, which references docker.d-popov.com/jwpw:latest as the image source.

Using the Automated Build Script

For convenience, an automated build script is included that handles the building and pushing process:

1. Make the script executable:

   chmod +x _deploy/build-and-push.sh
   

2. Run the script (from the project root):

   # Build and push with 'latest' tag
   ./_deploy/build-and-push.sh
   
   # Build and push with specific version tag
   ./_deploy/build-and-push.sh v1.2.3
   

3. What the script does:

   • Builds the Docker image using _deploy/prod.Dockerfile
   • Tags the image with the specified version and 'latest'
   • Pushes both tags to your private Docker registry
   • Performs basic error checking (Docker installed, logged into registry)

This script simplifies the process of creating and publishing new versions of your application, making it easier to maintain consistent deployment practices.

Migrating Between Methods

From Git Clone to Standard Docker

1. Back up your data:

   docker cp <container_name>:/app/public/content/permits ./permits_backup
   docker cp <container_name>:/app/public/content/uploads ./uploads_backup
   

2. Stop the old containers:

   docker-compose -f _deploy/deoloy.azure.production.yml down
   

3. Deploy using the standard method:

   docker-compose -f _deploy/standard-docker-compose.yml up -d
   

4. Restore your data if needed:

   docker cp ./permits_backup/. <new_container_name>:/app/public/content/permits
   docker cp ./uploads_backup/. <new_container_name>:/app/public/content/uploads
   

Troubleshooting

Docker Logs

View container logs:

docker logs <container_name>

Application Logs

Application logs are stored in:

• /app/logs inside the container
• The app_logs Docker volume

Common Issues

• Database connection error: Check the DATABASE_URL environment variable
• Missing files: Check volume mappings and permissions
• Container exits immediately: Check the logs for startup errors