diff --git a/_deploy/README.md b/_deploy/README.md new file mode 100644 index 0000000..4d75e82 --- /dev/null +++ b/_deploy/README.md @@ -0,0 +1,347 @@ +# Deployment Guide for MWitnessing Application + +This guide covers four deployment methods: +1. **Standard Docker Deployment** - Builds a complete Docker image with your code +2. **Runtime Git Clone Deployment** - Clones the repository at container startup (current method) +3. **Webhook-Based Deployment** - Automatically deploys when code is pushed to the repository +4. **Registry-Based Deployment** - Builds and pushes images to a private Docker registry + +## Method 1: Standard Docker Deployment (Recommended) + +This method builds a complete Docker image containing your application code, providing: +- Faster container startup +- No dependency on Git repository during runtime +- Better security (no Git credentials in environment variables) +- Standard Docker deployment workflow + +### Prerequisites +- Docker and Docker Compose installed on the host machine +- Access to the application source code + +### Deployment Steps + +1. **Build and start the containers**: + ```bash + cd /path/to/project + docker-compose -f _deploy/standard-docker-compose.yml up -d + ``` + +2. **Access the application**: + - Web application: http://localhost:3000 + - Database admin: http://localhost:8080 + +3. **Update the application**: + ```bash + # Pull the latest code + git pull + + # Rebuild and restart + docker-compose -f _deploy/standard-docker-compose.yml up -d --build + ``` + +### Configuration + +Edit the environment variables in `_deploy/standard-docker-compose.yml` before deployment: +- `DATABASE_URL`: Database connection string +- `NEXTAUTH_URL`: Public URL of your application +- `NEXTAUTH_SECRET`: Secret for NextAuth.js encryption + +### Persistent Data + +The following Docker volumes are used to persist data: +- `app_permits`: Stores permit files +- `app_uploads`: Stores uploaded files +- `app_logs`: Stores application logs +- `db_data`: Stores database data + +## Method 2: Runtime Git Clone Deployment (Legacy) + +This method clones the repository at container startup, which is the current deployment method. + +### Prerequisites +- Docker and Docker Compose installed on the host machine +- Git repository access credentials + +### Deployment Steps + +1. **Configure environment variables**: + Edit the Docker Compose file (e.g., `_deploy/deoloy.azure.production.yml`) to set: + - `GIT_USERNAME`: Username for Git repository access + - `GIT_PASSWORD`: Password for Git repository access + - `GIT_BRANCH`: Branch to deploy (default: main) + - `UPDATE_CODE_FROM_GIT`: Set to "true" to enable Git clone + +2. **Start the containers**: + ```bash + docker-compose -f _deploy/deoloy.azure.production.yml up -d + ``` + +3. **Update the application**: + - Updates happen automatically when containers restart + - Force an update by restarting the container: + ```bash + docker-compose -f _deploy/deoloy.azure.production.yml restart nextjs-app + ``` + +### How It Works + +The `entrypoint.sh` script: +1. Clones the repository at container startup +2. Backs up important directories like `permits` and `uploads` +3. Syncs new code from the repository +4. Restores or merges the backed-up directories +5. Rebuilds the application + +## Method 3: Webhook-Based Deployment (Recommended for Production) + +This method uses Gitea webhooks to automatically deploy changes when code is pushed to the repository, providing: +- Automated continuous deployment +- No manual intervention needed for updates +- Immediate deployment of changes +- Preservation of user-uploaded content + +### Prerequisites +- Docker and Docker Compose installed on the host machine +- Gitea repository with webhook capabilities +- Server with port 9000 (or your configured port) accessible to Gitea + +### Setup Steps + +1. **Build and start the webhook service**: + ```bash + cd /path/to/project + mkdir -p _deploy/webhook + cp _deploy/webhook-receiver.js _deploy/webhook/ + cp _deploy/webhook-deploy.sh _deploy/webhook/ + docker-compose -f _deploy/webhook-docker-compose.yml up -d + ``` + +2. **Configure Gitea webhook**: + - Go to your repository settings in Gitea + - Navigate to "Webhooks" > "Add Webhook" > "Gitea" + - Set Target URL: `http://your-server-ip:9000/webhook` + - Set Content Type: `application/json` + - Set Secret: Same as `WEBHOOK_SECRET` in webhook-docker-compose.yml + - Select "Just the push event" + - Select "Active" + - Click "Add Webhook" + +3. **Test the webhook**: + - Make a change to your repository and push it + - The webhook service will receive the event and trigger a deployment + - Check logs: `docker logs mwitnessing-webhook` + +### Configuration + +Edit the environment variables in `_deploy/webhook-docker-compose.yml` before deployment: +- `WEBHOOK_SECRET`: Secret for authenticating webhook requests (must match Gitea webhook secret) +- `ALLOWED_BRANCHES`: Comma-separated list of branches that trigger deployments +- `ALLOWED_REPOSITORIES`: Comma-separated list of repositories to monitor +- `GIT_USERNAME` and `GIT_PASSWORD`: Credentials for Git repository access + +### How It Works + +1. **Webhook Receiver**: + - Listens for push events from Gitea + - Validates the webhook signature using the shared secret + - Triggers the deployment script when a valid push event is received + +2. **Deployment Script**: + - Clones the repository at the specified branch + - Checks if dependencies have changed (package.json, Dockerfile, etc.) + - Performs a full rebuild if necessary, or just updates code and restarts + - Preserves user-uploaded content during deployment + +3. **Intelligent Updates**: + - Intelligently determines whether a full rebuild is needed + - Only rebuilds Docker images when necessary (package.json changes, etc.) + - Preserves data in volumes across deployments + +### Monitoring and Maintenance + +- **View logs**: + ```bash + docker logs mwitnessing-webhook + ``` + +- **Manual trigger**: + ```bash + docker exec mwitnessing-webhook /app/webhook-deploy.sh main + ``` + +- **Update webhook code**: + ```bash + # Edit webhook files, then: + docker-compose -f _deploy/webhook-docker-compose.yml up -d --build + ``` + +## Method 4: Registry-Based Deployment + +This method involves building Docker images locally or in a CI/CD pipeline, pushing them to your private Docker registry (`docker.d-popov.com`), and then deploying using those pre-built images. + +### Prerequisites +- Docker and Docker Compose installed on the build machine +- Access to your private Docker registry at `docker.d-popov.com` +- Authentication configured for the Docker registry + +### Build and Push Process + +1. **Log in to the Docker registry**: + ```bash + docker login docker.d-popov.com -u -p + ``` + +2. **Build the Docker image**: + ```bash + # Navigate to the project root + cd /path/to/project + + # Build the image with the jwpw tag + docker build -t docker.d-popov.com/jwpw:latest -f _deploy/prod.Dockerfile . + ``` + +3. **Push the image to the registry**: + ```bash + docker push docker.d-popov.com/jwpw:latest + ``` + +4. **Tag with version (optional but recommended)**: + ```bash + # Tag with version number + docker tag docker.d-popov.com/jwpw:latest docker.d-popov.com/jwpw:v1.0.0 + + # Push the versioned tag + docker push docker.d-popov.com/jwpw:v1.0.0 + ``` + +### Deployment Process + +1. **On the server, create or update your deployment YAML file**: + Create or edit a file like `_deploy/deploy.production.yml`: + + ```yaml + version: "3" + services: + nextjs-app: + hostname: jwpw-app-production + image: docker.d-popov.com/jwpw:latest + volumes: + - /mnt/docker_volumes/pw-prod/app/public/content/uploads/:/app/public/content/uploads + - /mnt/docker_volumes/pw-prod/app/logs:/app/logs + - /mnt/docker_volumes/pw-prod/app/public/content/permits/:/app/public/content/permits + environment: + - APP_ENV=production + - NODE_ENV=production + - TZ=Europe/Sofia + - DATABASE=mysql://username:password@db:3306/database_name + restart: always + networks: + - infrastructure_default + - default + # Add database service if needed + ``` + +2. **Deploy using Docker Compose**: + ```bash + docker-compose -f _deploy/deploy.production.yml up -d + ``` + +3. **Update the deployment** (when a new image is available): + ```bash + # Pull the latest image + docker pull docker.d-popov.com/jwpw:latest + + # Restart the service + docker-compose -f _deploy/deploy.production.yml up -d + ``` + +### Continuous Integration/Continuous Deployment (CI/CD) + +This method is ideal for integration with CI/CD pipelines. For example, you could set up a GitHub Actions or GitLab CI pipeline that: + +1. Builds the Docker image +2. Runs tests +3. Pushes the image to your private registry +4. Triggers a deployment on your server + +### Advantages of Registry-Based Deployment + +- **Consistent environments**: The exact same image is used in all environments +- **Faster deployments**: No building on the production server +- **Rollback capability**: Easy to roll back to a previous version by specifying an older tag +- **Better separation of concerns**: Build and deployment processes are separated +- **Audit trail**: Registry maintains a history of image versions + +### Current Use + +This method is currently used for your staging environment, as seen in `_deploy/deoloy.azure.staging.yml`, which references `docker.d-popov.com/jwpw:latest` as the image source. + +### Using the Automated Build Script + +For convenience, an automated build script is included that handles the building and pushing process: + +1. **Make the script executable**: + ```bash + chmod +x _deploy/build-and-push.sh + ``` + +2. **Run the script** (from the project root): + ```bash + # Build and push with 'latest' tag + ./_deploy/build-and-push.sh + + # Build and push with specific version tag + ./_deploy/build-and-push.sh v1.2.3 + ``` + +3. **What the script does**: + - Builds the Docker image using `_deploy/prod.Dockerfile` + - Tags the image with the specified version and 'latest' + - Pushes both tags to your private Docker registry + - Performs basic error checking (Docker installed, logged into registry) + +This script simplifies the process of creating and publishing new versions of your application, making it easier to maintain consistent deployment practices. + +## Migrating Between Methods + +### From Git Clone to Standard Docker + +1. Back up your data: + ```bash + docker cp :/app/public/content/permits ./permits_backup + docker cp :/app/public/content/uploads ./uploads_backup + ``` + +2. Stop the old containers: + ```bash + docker-compose -f _deploy/deoloy.azure.production.yml down + ``` + +3. Deploy using the standard method: + ```bash + docker-compose -f _deploy/standard-docker-compose.yml up -d + ``` + +4. Restore your data if needed: + ```bash + docker cp ./permits_backup/. :/app/public/content/permits + docker cp ./uploads_backup/. :/app/public/content/uploads + ``` + +## Troubleshooting + +### Docker Logs +View container logs: +```bash +docker logs +``` + +### Application Logs +Application logs are stored in: +- `/app/logs` inside the container +- The `app_logs` Docker volume + +### Common Issues +- **Database connection error**: Check the `DATABASE_URL` environment variable +- **Missing files**: Check volume mappings and permissions +- **Container exits immediately**: Check the logs for startup errors \ No newline at end of file diff --git a/_deploy/build-and-push.sh b/_deploy/build-and-push.sh new file mode 100644 index 0000000..0dbdd76 --- /dev/null +++ b/_deploy/build-and-push.sh @@ -0,0 +1,58 @@ +#!/bin/bash +# Script to build and push Docker images to the private registry at docker.d-popov.com +# Usage: ./build-and-push.sh [version] + +set -e + +# Configuration +REGISTRY="docker.d-popov.com" +IMAGE_NAME="jwpw" +DOCKERFILE="_deploy/prod.Dockerfile" +VERSION="${1:-latest}" # Use first argument as version or default to 'latest' + +# Display build information +echo "=====================================================" +echo "Building ${REGISTRY}/${IMAGE_NAME}:${VERSION}" +echo "Using Dockerfile: ${DOCKERFILE}" +echo "=====================================================" + +# Check if Docker is available +if ! command -v docker &> /dev/null; then + echo "Error: Docker is not installed or not in the PATH" + exit 1 +fi + +# Check if user is logged in to the registry +if ! docker info | grep -q "Registry: ${REGISTRY}"; then + echo "You need to log in to the registry first:" + echo "docker login ${REGISTRY}" + exit 1 +fi + +# Build the Docker image +echo "Building Docker image..." +docker build -t "${REGISTRY}/${IMAGE_NAME}:${VERSION}" -f "${DOCKERFILE}" . + +# Tag as latest if version is not 'latest' +if [ "${VERSION}" != "latest" ]; then + echo "Tagging also as latest..." + docker tag "${REGISTRY}/${IMAGE_NAME}:${VERSION}" "${REGISTRY}/${IMAGE_NAME}:latest" +fi + +# Push the image to the registry +echo "Pushing Docker image ${REGISTRY}/${IMAGE_NAME}:${VERSION} to registry..." +docker push "${REGISTRY}/${IMAGE_NAME}:${VERSION}" + +# Push latest tag if version is not 'latest' +if [ "${VERSION}" != "latest" ]; then + echo "Pushing Docker image ${REGISTRY}/${IMAGE_NAME}:latest to registry..." + docker push "${REGISTRY}/${IMAGE_NAME}:latest" +fi + +echo "=====================================================" +echo "Build and push completed successfully!" +echo "Image: ${REGISTRY}/${IMAGE_NAME}:${VERSION}" +if [ "${VERSION}" != "latest" ]; then + echo "Image: ${REGISTRY}/${IMAGE_NAME}:latest" +fi +echo "=====================================================" \ No newline at end of file diff --git a/_deploy/migrate-deployment.sh b/_deploy/migrate-deployment.sh new file mode 100644 index 0000000..4f3f4c2 --- /dev/null +++ b/_deploy/migrate-deployment.sh @@ -0,0 +1,114 @@ +#!/bin/bash +# Script to migrate from Git clone deployment to standard Docker deployment + +set -e + +# Print usage +print_usage() { + echo "Usage: $0 [OPTIONS]" + echo "Migrate from Git clone deployment to standard Docker deployment" + echo "" + echo "Options:" + echo " -s, --source CONTAINER Source container name (default: jwpw-app-staging)" + echo " -c, --compose FILE Docker Compose file for new deployment (default: _deploy/standard-docker-compose.yml)" + echo " -b, --backup DIR Backup directory (default: ./deployment_backup)" + echo " -h, --help Show this help message" + exit 1 +} + +# Default values +SOURCE_CONTAINER="jwpw-app-staging" +COMPOSE_FILE="_deploy/standard-docker-compose.yml" +BACKUP_DIR="./deployment_backup" + +# Parse command line arguments +while [[ $# -gt 0 ]]; do + case $1 in + -s|--source) + SOURCE_CONTAINER="$2" + shift 2 + ;; + -c|--compose) + COMPOSE_FILE="$2" + shift 2 + ;; + -b|--backup) + BACKUP_DIR="$2" + shift 2 + ;; + -h|--help) + print_usage + ;; + *) + echo "Unknown option: $1" + print_usage + ;; + esac +done + +echo "==========================================================" +echo "Migration from Git clone deployment to standard deployment" +echo "==========================================================" +echo "Source container: $SOURCE_CONTAINER" +echo "Compose file: $COMPOSE_FILE" +echo "Backup directory: $BACKUP_DIR" +echo "" + +# Confirm before proceeding +read -p "Do you want to proceed? (y/n) " -n 1 -r +echo "" +if [[ ! $REPLY =~ ^[Yy]$ ]]; then + echo "Migration aborted." + exit 1 +fi + +# Check if source container exists +if ! docker ps -a | grep -q "$SOURCE_CONTAINER"; then + echo "Error: Source container '$SOURCE_CONTAINER' not found." + exit 1 +fi + +# Create backup directory +mkdir -p "$BACKUP_DIR/permits" +mkdir -p "$BACKUP_DIR/uploads" +mkdir -p "$BACKUP_DIR/logs" + +echo "Step 1: Backing up data from source container..." +docker cp "$SOURCE_CONTAINER:/app/public/content/permits/." "$BACKUP_DIR/permits/" +docker cp "$SOURCE_CONTAINER:/app/public/content/uploads/." "$BACKUP_DIR/uploads/" +docker cp "$SOURCE_CONTAINER:/app/logs/." "$BACKUP_DIR/logs/" +echo "Backup completed successfully." + +echo "Step 2: Checking if the Compose file exists..." +if [ ! -f "$COMPOSE_FILE" ]; then + echo "Error: Compose file '$COMPOSE_FILE' not found." + exit 1 +fi + +echo "Step 3: Deploying new Docker containers..." +docker-compose -f "$COMPOSE_FILE" up -d +echo "New containers deployed." + +# Get the new container name +NEW_CONTAINER=$(docker-compose -f "$COMPOSE_FILE" ps -q nextjs-app) +if [ -z "$NEW_CONTAINER" ]; then + echo "Error: Could not find new container. Please check the deployment." + exit 1 +fi + +echo "Step 4: Restoring data to new container..." +docker cp "$BACKUP_DIR/permits/." "$NEW_CONTAINER:/app/public/content/permits/" +docker cp "$BACKUP_DIR/uploads/." "$NEW_CONTAINER:/app/public/content/uploads/" +echo "Data restored successfully." + +echo "Step 5: Setting correct permissions..." +docker exec -it "$NEW_CONTAINER" chown -R nextjs:nodejs /app/public/content + +echo "==========================================================" +echo "Migration completed successfully!" +echo "New container: $NEW_CONTAINER" +echo "Backup saved to: $BACKUP_DIR" +echo "" +echo "To verify the deployment, visit your application URL." +echo "To roll back, stop the new containers and restart the old ones." +echo "==========================================================" \ No newline at end of file diff --git a/_deploy/standard-docker-compose.yml b/_deploy/standard-docker-compose.yml new file mode 100644 index 0000000..de15efd --- /dev/null +++ b/_deploy/standard-docker-compose.yml @@ -0,0 +1,65 @@ +version: "3.8" + +services: + nextjs-app: + container_name: mwitnessing-app + build: + context: .. + dockerfile: _deploy/standard.Dockerfile + restart: always + ports: + - "3000:3000" + environment: + - DATABASE_URL=mysql://mwitnessing_user:password@db:3306/mwitnessing + - NEXTAUTH_URL=http://localhost:3000 + - NEXTAUTH_SECRET=your-nextauth-secret + volumes: + - app_permits:/app/public/content/permits + - app_uploads:/app/public/content/uploads + - app_logs:/app/logs + depends_on: + - db + networks: + - app-network + + db: + container_name: mwitnessing-db + image: mysql:8.0 + restart: always + environment: + - MYSQL_ROOT_PASSWORD=rootpassword + - MYSQL_DATABASE=mwitnessing + - MYSQL_USER=mwitnessing_user + - MYSQL_PASSWORD=password + volumes: + - db_data:/var/lib/mysql + ports: + - "3306:3306" + networks: + - app-network + command: --default-authentication-plugin=mysql_native_password + + adminer: + container_name: mwitnessing-adminer + image: adminer:latest + restart: always + ports: + - "8080:8080" + depends_on: + - db + networks: + - app-network + +volumes: + app_permits: + driver: local + app_uploads: + driver: local + app_logs: + driver: local + db_data: + driver: local + +networks: + app-network: + driver: bridge \ No newline at end of file diff --git a/_deploy/standard.Dockerfile b/_deploy/standard.Dockerfile new file mode 100644 index 0000000..1fdc9ad --- /dev/null +++ b/_deploy/standard.Dockerfile @@ -0,0 +1,57 @@ +# Standard Dockerfile for normal deployment +# This builds a complete image with the application code included + +# Build stage +FROM node:18-alpine AS builder + +WORKDIR /app + +# Copy package files and install dependencies +COPY package*.json ./ +RUN npm ci + +# Copy application code +COPY . . + +# Generate Prisma client +RUN npx prisma generate + +# Build the Next.js application +RUN npm run build + +# Production stage +FROM node:18-alpine AS runner + +WORKDIR /app + +# Set production environment +ENV NODE_ENV=production + +# Install necessary runtime dependencies +RUN apk --no-cache add bash + +# Create non-root user for security +RUN addgroup --system --gid 1001 nodejs \ + && adduser --system --uid 1001 nextjs + +# Copy necessary files from build stage +COPY --from=builder /app/next.config.js ./ +COPY --from=builder /app/public ./public +COPY --from=builder /app/package*.json ./ +COPY --from=builder /app/.next/standalone ./ +COPY --from=builder /app/.next/static ./.next/static +COPY --from=builder /app/node_modules/.prisma ./node_modules/.prisma +COPY --from=builder /app/node_modules/@prisma ./node_modules/@prisma + +# Create required directories with proper permissions +RUN mkdir -p public/content/permits public/content/uploads logs \ + && chown -R nextjs:nodejs public/content logs + +# Switch to non-root user +USER nextjs + +# Expose the application port +EXPOSE 3000 + +# Start the application +CMD ["node", "server.js"] \ No newline at end of file diff --git a/_deploy/webhook-deploy.sh b/_deploy/webhook-deploy.sh new file mode 100644 index 0000000..08c4ae8 --- /dev/null +++ b/_deploy/webhook-deploy.sh @@ -0,0 +1,152 @@ +#!/bin/bash +# Webhook deployment script +# This script is triggered by the webhook receiver when a push event is received + +set -e + +# Configuration +APP_DIR="/app" # Application directory +REPO_URL="https://git.d-popov.com/popov/mwitnessing.git" # Git repository URL +GIT_USERNAME="${GIT_USERNAME:-deploy}" # Git username +GIT_PASSWORD="${GIT_PASSWORD:-}" # Git password +DOCKER_COMPOSE_FILE="${DOCKER_COMPOSE_FILE:-_deploy/standard-docker-compose.yml}" # Docker Compose file +LOG_FILE="${LOG_FILE:-/app/logs/webhook-deploy.log}" # Log file + +# Create log directory if it doesn't exist +mkdir -p "$(dirname "$LOG_FILE")" + +# Function to log messages +log() { + echo "[$(date '+%Y-%m-%d %H:%M:%S')] $1" | tee -a "$LOG_FILE" +} + +# Get the branch name from the first argument or use main as default +BRANCH="${1:-main}" +log "Starting deployment for branch: $BRANCH" + +# Create a temporary directory for the Git clone +TEMP_DIR=$(mktemp -d) +log "Using temporary directory: $TEMP_DIR" + +# Clean up function to remove the temporary directory on exit +cleanup() { + log "Cleaning up temporary directory" + rm -rf "$TEMP_DIR" +} +trap cleanup EXIT + +# Clone the repository +log "Cloning repository: $REPO_URL (branch: $BRANCH)" +if [ -n "$GIT_PASSWORD" ]; then + # Use authentication with username and password + AUTH_URL="${REPO_URL/https:\/\//https:\/\/$GIT_USERNAME:$GIT_PASSWORD@}" + git clone -b "$BRANCH" --depth 1 "$AUTH_URL" "$TEMP_DIR" +else + # Use authentication with SSH keys + git clone -b "$BRANCH" --depth 1 "$REPO_URL" "$TEMP_DIR" +fi + +# Navigate to the cloned repository +cd "$TEMP_DIR" + +# Backup important directories +log "Backing up important directories" +mkdir -p "/tmp/webhook_backup/permits" "/tmp/webhook_backup/uploads" + +if [ -d "$APP_DIR/public/content/permits" ]; then + cp -r "$APP_DIR/public/content/permits/." "/tmp/webhook_backup/permits/" + log "Permits directory backed up" +fi + +if [ -d "$APP_DIR/public/content/uploads" ]; then + cp -r "$APP_DIR/public/content/uploads/." "/tmp/webhook_backup/uploads/" + log "Uploads directory backed up" +fi + +# Check if we should rebuild the Docker image +REBUILD=0 +if [ -f "$TEMP_DIR/package.json" ] && [ -f "$APP_DIR/package.json" ]; then + if ! cmp -s "$TEMP_DIR/package.json" "$APP_DIR/package.json"; then + log "package.json has changed, triggering full rebuild" + REBUILD=1 + elif [ -f "$TEMP_DIR/package-lock.json" ] && [ -f "$APP_DIR/package-lock.json" ]; then + if ! cmp -s "$TEMP_DIR/package-lock.json" "$APP_DIR/package-lock.json"; then + log "package-lock.json has changed, triggering full rebuild" + REBUILD=1 + fi + fi +fi + +# Check if the Dockerfile or docker-compose file has changed +if [ -f "$TEMP_DIR/_deploy/standard.Dockerfile" ] && [ -f "$APP_DIR/_deploy/standard.Dockerfile" ]; then + if ! cmp -s "$TEMP_DIR/_deploy/standard.Dockerfile" "$APP_DIR/_deploy/standard.Dockerfile"; then + log "Dockerfile has changed, triggering full rebuild" + REBUILD=1 + fi +fi + +if [ -f "$TEMP_DIR/$DOCKER_COMPOSE_FILE" ] && [ -f "$APP_DIR/$DOCKER_COMPOSE_FILE" ]; then + if ! cmp -s "$TEMP_DIR/$DOCKER_COMPOSE_FILE" "$APP_DIR/$DOCKER_COMPOSE_FILE"; then + log "Docker Compose file has changed, triggering full rebuild" + REBUILD=1 + fi +fi + +# Deploy the application +if [ "$REBUILD" -eq 1 ]; then + log "Performing full rebuild and restart" + + # Copy new code to application directory + rsync -av --exclude 'node_modules' --exclude '.git' --exclude 'public/content/permits' --exclude 'public/content/uploads' "$TEMP_DIR/" "$APP_DIR/" + + # Rebuild and restart using Docker Compose + cd "$APP_DIR" + docker-compose -f "$DOCKER_COMPOSE_FILE" up -d --build +else + log "Performing code update without rebuild" + + # Copy new code to application directory, preserving special directories + rsync -av --exclude 'node_modules' --exclude '.git' --exclude 'public/content/permits' --exclude 'public/content/uploads' "$TEMP_DIR/" "$APP_DIR/" + + # Find container ID for the Next.js app + CONTAINER_ID=$(docker ps -qf "name=nextjs-app") + + if [ -n "$CONTAINER_ID" ]; then + log "Restarting Next.js container: $CONTAINER_ID" + docker restart "$CONTAINER_ID" + else + log "Next.js container not found, starting with Docker Compose" + cd "$APP_DIR" + docker-compose -f "$DOCKER_COMPOSE_FILE" up -d + fi +fi + +# Restore backed up directories and merge with new content from Git +log "Restoring and merging backed up directories" + +# Create permits directory if it doesn't exist +mkdir -p "$APP_DIR/public/content/permits" + +# Merge Git permits (if any) with backed up permits +if [ -d "$TEMP_DIR/public/content/permits" ] && [ "$(ls -A "$TEMP_DIR/public/content/permits")" ]; then + log "Copying permits from Git repository" + cp -r "$TEMP_DIR/public/content/permits/." "$APP_DIR/public/content/permits/" +fi + +# Restore backed up permits (these will take precedence over Git permits) +if [ -d "/tmp/webhook_backup/permits" ] && [ "$(ls -A "/tmp/webhook_backup/permits")" ]; then + log "Restoring backed up permits" + cp -r "/tmp/webhook_backup/permits/." "$APP_DIR/public/content/permits/" +fi + +# Restore uploads (these should only come from the backup, not Git) +if [ -d "/tmp/webhook_backup/uploads" ] && [ "$(ls -A "/tmp/webhook_backup/uploads")" ]; then + log "Restoring backed up uploads" + mkdir -p "$APP_DIR/public/content/uploads" + cp -r "/tmp/webhook_backup/uploads/." "$APP_DIR/public/content/uploads/" +fi + +# Clean up backup +rm -rf "/tmp/webhook_backup" + +log "Deployment completed successfully" \ No newline at end of file diff --git a/_deploy/webhook-docker-compose.yml b/_deploy/webhook-docker-compose.yml new file mode 100644 index 0000000..9a34cf2 --- /dev/null +++ b/_deploy/webhook-docker-compose.yml @@ -0,0 +1,39 @@ +version: "3.8" + +services: + webhook-receiver: + container_name: mwitnessing-webhook + build: + context: ./webhook + dockerfile: Dockerfile + restart: always + ports: + - "9000:9000" + environment: + - WEBHOOK_PORT=9000 + - WEBHOOK_SECRET=change-this-secret-in-production + - DEPLOY_SCRIPT=/app/webhook-deploy.sh + - LOG_FILE=/app/logs/webhook.log + - ALLOWED_BRANCHES=main,develop + - ALLOWED_REPOSITORIES=mwhitnessing + # Git credentials for deployment + - GIT_USERNAME=deploy + - GIT_PASSWORD=your-git-password + # Docker configuration + - DOCKER_COMPOSE_FILE=_deploy/standard-docker-compose.yml + volumes: + - ./webhook-receiver.js:/app/webhook-receiver.js + - ./webhook-deploy.sh:/app/webhook-deploy.sh + - webhook_logs:/app/logs + # Mount Docker socket to allow container restart + - /var/run/docker.sock:/var/run/docker.sock + networks: + - app-network + +volumes: + webhook_logs: + driver: local + +networks: + app-network: + external: true \ No newline at end of file diff --git a/_deploy/webhook-receiver.js b/_deploy/webhook-receiver.js new file mode 100644 index 0000000..2b841f1 --- /dev/null +++ b/_deploy/webhook-receiver.js @@ -0,0 +1,137 @@ +// Webhook receiver for Gitea push events +// This service listens for webhook events from Gitea and triggers a deployment +// when changes are pushed to the monitored branches + +const express = require('express'); +const http = require('http'); +const crypto = require('crypto'); +const { exec } = require('child_process'); +const path = require('path'); +const fs = require('fs'); + +// Configuration (can be moved to environment variables) +const config = { + port: process.env.WEBHOOK_PORT || 9000, + secret: process.env.WEBHOOK_SECRET || 'change-this-secret-in-production', + deployScript: process.env.DEPLOY_SCRIPT || path.join(__dirname, 'webhook-deploy.sh'), + logFile: process.env.LOG_FILE || path.join(__dirname, '../logs/webhook.log'), + allowedBranches: (process.env.ALLOWED_BRANCHES || 'main,master').split(','), + allowedRepositories: (process.env.ALLOWED_REPOSITORIES || 'mwhitnessing').split(',') +}; + +// Create logs directory if it doesn't exist +const logsDir = path.dirname(config.logFile); +if (!fs.existsSync(logsDir)) { + fs.mkdirSync(logsDir, { recursive: true }); +} + +// Create Express app +const app = express(); +app.use(express.json()); + +// Helper function to log messages +function log(message) { + const timestamp = new Date().toISOString(); + const logMessage = `[${timestamp}] ${message}\n`; + console.log(logMessage.trim()); + fs.appendFileSync(config.logFile, logMessage); +} + +// Verify Gitea webhook signature +function verifySignature(req) { + const signature = req.headers['x-gitea-signature']; + if (!signature) { + return false; + } + + const hmac = crypto.createHmac('sha256', config.secret); + const computedSignature = hmac.update(JSON.stringify(req.body)).digest('hex'); + + return crypto.timingSafeEqual( + Buffer.from(signature), + Buffer.from(computedSignature) + ); +} + +// Handle Gitea push webhook +app.post('/webhook', (req, res) => { + // Log receipt of webhook + log('Received webhook request'); + + try { + // Check if webhook signature is valid + if (!verifySignature(req)) { + log('Invalid webhook signature'); + return res.status(403).send('Invalid signature'); + } + + // Extract relevant information from the webhook payload + const { ref, repository } = req.body; + + // Check if this is a branch we care about + const branchName = ref.replace('refs/heads/', ''); + if (!config.allowedBranches.includes(branchName)) { + log(`Ignoring push to branch: ${branchName}`); + return res.status(200).send('Ignored branch'); + } + + // Check if this is a repository we care about + const repoName = repository.name; + if (!config.allowedRepositories.includes(repoName)) { + log(`Ignoring push to repository: ${repoName}`); + return res.status(200).send('Ignored repository'); + } + + // Log the event + log(`Received push event for ${repoName}/${branchName}`); + + // Respond to webhook immediately + res.status(200).send('Processing deployment'); + + // Execute the deployment script with the branch name + const command = `${config.deployScript} ${branchName}`; + log(`Executing: ${command}`); + + exec(command, (error, stdout, stderr) => { + if (error) { + log(`Deployment error: ${error.message}`); + return; + } + if (stderr) { + log(`Deployment stderr: ${stderr}`); + } + log(`Deployment stdout: ${stdout}`); + log('Deployment completed successfully'); + }); + } catch (error) { + log(`Error processing webhook: ${error.message}`); + if (!res.headersSent) { + res.status(500).send('Error processing webhook'); + } + } +}); + +// Start the server +const server = http.createServer(app); +server.listen(config.port, () => { + log(`Webhook receiver listening on port ${config.port}`); + log(`Monitoring branches: ${config.allowedBranches.join(', ')}`); + log(`Monitoring repositories: ${config.allowedRepositories.join(', ')}`); +}); + +// Handle server shutdown +process.on('SIGTERM', () => { + log('Shutting down webhook receiver...'); + server.close(() => { + log('Webhook receiver stopped'); + process.exit(0); + }); +}); + +process.on('SIGINT', () => { + log('Shutting down webhook receiver...'); + server.close(() => { + log('Webhook receiver stopped'); + process.exit(0); + }); +}); \ No newline at end of file diff --git a/_deploy/webhook/Dockerfile b/_deploy/webhook/Dockerfile new file mode 100644 index 0000000..73ac2f4 --- /dev/null +++ b/_deploy/webhook/Dockerfile @@ -0,0 +1,28 @@ +FROM node:18-alpine + +# Install necessary tools +RUN apk --no-cache add git docker-cli bash rsync + +WORKDIR /app + +# Copy package.json +COPY package.json . + +# Install dependencies +RUN npm install express http crypto + +# Create logs directory +RUN mkdir -p /app/logs + +# Make deployment script executable +COPY webhook-deploy.sh . +RUN chmod +x webhook-deploy.sh + +# Copy webhook receiver +COPY webhook-receiver.js . + +# Expose webhook port +EXPOSE 9000 + +# Start the webhook receiver +CMD ["node", "webhook-receiver.js"] \ No newline at end of file diff --git a/_deploy/webhook/package.json b/_deploy/webhook/package.json new file mode 100644 index 0000000..3a46bc7 --- /dev/null +++ b/_deploy/webhook/package.json @@ -0,0 +1,13 @@ +{ + "name": "mwitnessing-webhook", + "version": "1.0.0", + "description": "Webhook receiver for automatic deployments", + "main": "webhook-receiver.js", + "scripts": { + "start": "node webhook-receiver.js" + }, + "dependencies": { + "express": "^4.18.2", + "crypto": "^1.0.1" + } +} \ No newline at end of file