Compare commits

...

10 Commits

Author SHA1 Message Date
8bcdbdd157 sql to fix novasmqna 2025-07-23 00:04:53 +03:00
a03ed81956 edit texts 2025-04-19 04:13:13 +03:00
ac9da25d64 try deploy fix 2025-04-10 03:39:55 +03:00
bf43efe75d scripts 2025-04-10 03:18:40 +03:00
0d3914f3c6 deployment methods - new and docs 2025-04-10 03:02:30 +03:00
fe96003eee merge permits folder instead of only keeping server's 2025-04-10 02:04:53 +03:00
8f066c17e7 try to fix permits upload restore/merge upon deployment 2025-04-10 02:01:53 +03:00
6f5166254b better styling and tone 2025-04-10 01:50:08 +03:00
13c1f46360 transport info 2025-04-10 01:49:03 +03:00
5bcb943761 better popup 2025-04-09 15:10:04 +03:00
16 changed files with 1235 additions and 28 deletions

3
.gitignore vendored
View File

@ -35,6 +35,9 @@ content/output/*
public/content/output/*
public/content/output/shifts 2024.1.json
!public/content/uploads/*
# Exclude the permits folder content from Git tracking but keep the folder itself
public/content/permits/*
!public/content/permits/.gitkeep
.aider*
/shift_generate_log_*.txt
.env

347
_deploy/README.md Normal file
View File

@ -0,0 +1,347 @@
# Deployment Guide for MWitnessing Application
This guide covers four deployment methods:
1. **Standard Docker Deployment** - Builds a complete Docker image with your code
2. **Runtime Git Clone Deployment** - Clones the repository at container startup (current method)
3. **Webhook-Based Deployment** - Automatically deploys when code is pushed to the repository
4. **Registry-Based Deployment** - Builds and pushes images to a private Docker registry
## Method 1: Standard Docker Deployment (Recommended)
This method builds a complete Docker image containing your application code, providing:
- Faster container startup
- No dependency on Git repository during runtime
- Better security (no Git credentials in environment variables)
- Standard Docker deployment workflow
### Prerequisites
- Docker and Docker Compose installed on the host machine
- Access to the application source code
### Deployment Steps
1. **Build and start the containers**:
```bash
cd /path/to/project
docker-compose -f _deploy/standard-docker-compose.yml up -d
```
2. **Access the application**:
- Web application: http://localhost:3000
- Database admin: http://localhost:8080
3. **Update the application**:
```bash
# Pull the latest code
git pull
# Rebuild and restart
docker-compose -f _deploy/standard-docker-compose.yml up -d --build
```
### Configuration
Edit the environment variables in `_deploy/standard-docker-compose.yml` before deployment:
- `DATABASE_URL`: Database connection string
- `NEXTAUTH_URL`: Public URL of your application
- `NEXTAUTH_SECRET`: Secret for NextAuth.js encryption
### Persistent Data
The following Docker volumes are used to persist data:
- `app_permits`: Stores permit files
- `app_uploads`: Stores uploaded files
- `app_logs`: Stores application logs
- `db_data`: Stores database data
## Method 2: Runtime Git Clone Deployment (Legacy)
This method clones the repository at container startup, which is the current deployment method.
### Prerequisites
- Docker and Docker Compose installed on the host machine
- Git repository access credentials
### Deployment Steps
1. **Configure environment variables**:
Edit the Docker Compose file (e.g., `_deploy/deoloy.azure.production.yml`) to set:
- `GIT_USERNAME`: Username for Git repository access
- `GIT_PASSWORD`: Password for Git repository access
- `GIT_BRANCH`: Branch to deploy (default: main)
- `UPDATE_CODE_FROM_GIT`: Set to "true" to enable Git clone
2. **Start the containers**:
```bash
docker-compose -f _deploy/deoloy.azure.production.yml up -d
```
3. **Update the application**:
- Updates happen automatically when containers restart
- Force an update by restarting the container:
```bash
docker-compose -f _deploy/deoloy.azure.production.yml restart nextjs-app
```
### How It Works
The `entrypoint.sh` script:
1. Clones the repository at container startup
2. Backs up important directories like `permits` and `uploads`
3. Syncs new code from the repository
4. Restores or merges the backed-up directories
5. Rebuilds the application
## Method 3: Webhook-Based Deployment (Recommended for Production)
This method uses Gitea webhooks to automatically deploy changes when code is pushed to the repository, providing:
- Automated continuous deployment
- No manual intervention needed for updates
- Immediate deployment of changes
- Preservation of user-uploaded content
### Prerequisites
- Docker and Docker Compose installed on the host machine
- Gitea repository with webhook capabilities
- Server with port 9000 (or your configured port) accessible to Gitea
### Setup Steps
1. **Build and start the webhook service**:
```bash
cd /path/to/project
mkdir -p _deploy/webhook
cp _deploy/webhook-receiver.js _deploy/webhook/
cp _deploy/webhook-deploy.sh _deploy/webhook/
docker-compose -f _deploy/webhook-docker-compose.yml up -d
```
2. **Configure Gitea webhook**:
- Go to your repository settings in Gitea
- Navigate to "Webhooks" > "Add Webhook" > "Gitea"
- Set Target URL: `http://your-server-ip:9000/webhook`
- Set Content Type: `application/json`
- Set Secret: Same as `WEBHOOK_SECRET` in webhook-docker-compose.yml
- Select "Just the push event"
- Select "Active"
- Click "Add Webhook"
3. **Test the webhook**:
- Make a change to your repository and push it
- The webhook service will receive the event and trigger a deployment
- Check logs: `docker logs mwitnessing-webhook`
### Configuration
Edit the environment variables in `_deploy/webhook-docker-compose.yml` before deployment:
- `WEBHOOK_SECRET`: Secret for authenticating webhook requests (must match Gitea webhook secret)
- `ALLOWED_BRANCHES`: Comma-separated list of branches that trigger deployments
- `ALLOWED_REPOSITORIES`: Comma-separated list of repositories to monitor
- `GIT_USERNAME` and `GIT_PASSWORD`: Credentials for Git repository access
### How It Works
1. **Webhook Receiver**:
- Listens for push events from Gitea
- Validates the webhook signature using the shared secret
- Triggers the deployment script when a valid push event is received
2. **Deployment Script**:
- Clones the repository at the specified branch
- Checks if dependencies have changed (package.json, Dockerfile, etc.)
- Performs a full rebuild if necessary, or just updates code and restarts
- Preserves user-uploaded content during deployment
3. **Intelligent Updates**:
- Intelligently determines whether a full rebuild is needed
- Only rebuilds Docker images when necessary (package.json changes, etc.)
- Preserves data in volumes across deployments
### Monitoring and Maintenance
- **View logs**:
```bash
docker logs mwitnessing-webhook
```
- **Manual trigger**:
```bash
docker exec mwitnessing-webhook /app/webhook-deploy.sh main
```
- **Update webhook code**:
```bash
# Edit webhook files, then:
docker-compose -f _deploy/webhook-docker-compose.yml up -d --build
```
## Method 4: Registry-Based Deployment
This method involves building Docker images locally or in a CI/CD pipeline, pushing them to your private Docker registry (`docker.d-popov.com`), and then deploying using those pre-built images.
### Prerequisites
- Docker and Docker Compose installed on the build machine
- Access to your private Docker registry at `docker.d-popov.com`
- Authentication configured for the Docker registry
### Build and Push Process
1. **Log in to the Docker registry**:
```bash
docker login docker.d-popov.com -u <username> -p <password>
```
2. **Build the Docker image**:
```bash
# Navigate to the project root
cd /path/to/project
# Build the image with the jwpw tag
docker build -t docker.d-popov.com/jwpw:latest -f _deploy/prod.Dockerfile .
```
3. **Push the image to the registry**:
```bash
docker push docker.d-popov.com/jwpw:latest
```
4. **Tag with version (optional but recommended)**:
```bash
# Tag with version number
docker tag docker.d-popov.com/jwpw:latest docker.d-popov.com/jwpw:v1.0.0
# Push the versioned tag
docker push docker.d-popov.com/jwpw:v1.0.0
```
### Deployment Process
1. **On the server, create or update your deployment YAML file**:
Create or edit a file like `_deploy/deploy.production.yml`:
```yaml
version: "3"
services:
nextjs-app:
hostname: jwpw-app-production
image: docker.d-popov.com/jwpw:latest
volumes:
- /mnt/docker_volumes/pw-prod/app/public/content/uploads/:/app/public/content/uploads
- /mnt/docker_volumes/pw-prod/app/logs:/app/logs
- /mnt/docker_volumes/pw-prod/app/public/content/permits/:/app/public/content/permits
environment:
- APP_ENV=production
- NODE_ENV=production
- TZ=Europe/Sofia
- DATABASE=mysql://username:password@db:3306/database_name
restart: always
networks:
- infrastructure_default
- default
# Add database service if needed
```
2. **Deploy using Docker Compose**:
```bash
docker-compose -f _deploy/deploy.production.yml up -d
```
3. **Update the deployment** (when a new image is available):
```bash
# Pull the latest image
docker pull docker.d-popov.com/jwpw:latest
# Restart the service
docker-compose -f _deploy/deploy.production.yml up -d
```
### Continuous Integration/Continuous Deployment (CI/CD)
This method is ideal for integration with CI/CD pipelines. For example, you could set up a GitHub Actions or GitLab CI pipeline that:
1. Builds the Docker image
2. Runs tests
3. Pushes the image to your private registry
4. Triggers a deployment on your server
### Advantages of Registry-Based Deployment
- **Consistent environments**: The exact same image is used in all environments
- **Faster deployments**: No building on the production server
- **Rollback capability**: Easy to roll back to a previous version by specifying an older tag
- **Better separation of concerns**: Build and deployment processes are separated
- **Audit trail**: Registry maintains a history of image versions
### Current Use
This method is currently used for your staging environment, as seen in `_deploy/deoloy.azure.staging.yml`, which references `docker.d-popov.com/jwpw:latest` as the image source.
### Using the Automated Build Script
For convenience, an automated build script is included that handles the building and pushing process:
1. **Make the script executable**:
```bash
chmod +x _deploy/build-and-push.sh
```
2. **Run the script** (from the project root):
```bash
# Build and push with 'latest' tag
./_deploy/build-and-push.sh
# Build and push with specific version tag
./_deploy/build-and-push.sh v1.2.3
```
3. **What the script does**:
- Builds the Docker image using `_deploy/prod.Dockerfile`
- Tags the image with the specified version and 'latest'
- Pushes both tags to your private Docker registry
- Performs basic error checking (Docker installed, logged into registry)
This script simplifies the process of creating and publishing new versions of your application, making it easier to maintain consistent deployment practices.
## Migrating Between Methods
### From Git Clone to Standard Docker
1. Back up your data:
```bash
docker cp <container_name>:/app/public/content/permits ./permits_backup
docker cp <container_name>:/app/public/content/uploads ./uploads_backup
```
2. Stop the old containers:
```bash
docker-compose -f _deploy/deoloy.azure.production.yml down
```
3. Deploy using the standard method:
```bash
docker-compose -f _deploy/standard-docker-compose.yml up -d
```
4. Restore your data if needed:
```bash
docker cp ./permits_backup/. <new_container_name>:/app/public/content/permits
docker cp ./uploads_backup/. <new_container_name>:/app/public/content/uploads
```
## Troubleshooting
### Docker Logs
View container logs:
```bash
docker logs <container_name>
```
### Application Logs
Application logs are stored in:
- `/app/logs` inside the container
- The `app_logs` Docker volume
### Common Issues
- **Database connection error**: Check the `DATABASE_URL` environment variable
- **Missing files**: Check volume mappings and permissions
- **Container exits immediately**: Check the logs for startup errors

58
_deploy/build-and-push.sh Normal file
View File

@ -0,0 +1,58 @@
#!/bin/bash
# Script to build and push Docker images to the private registry at docker.d-popov.com
# Usage: ./build-and-push.sh [version]
set -e
# Configuration
REGISTRY="docker.d-popov.com"
IMAGE_NAME="jwpw"
DOCKERFILE="_deploy/prod.Dockerfile"
VERSION="${1:-latest}" # Use first argument as version or default to 'latest'
# Display build information
echo "====================================================="
echo "Building ${REGISTRY}/${IMAGE_NAME}:${VERSION}"
echo "Using Dockerfile: ${DOCKERFILE}"
echo "====================================================="
# Check if Docker is available
if ! command -v docker &> /dev/null; then
echo "Error: Docker is not installed or not in the PATH"
exit 1
fi
# Check if user is logged in to the registry
if ! docker info | grep -q "Registry: ${REGISTRY}"; then
echo "You need to log in to the registry first:"
echo "docker login ${REGISTRY}"
exit 1
fi
# Build the Docker image
echo "Building Docker image..."
docker build -t "${REGISTRY}/${IMAGE_NAME}:${VERSION}" -f "${DOCKERFILE}" .
# Tag as latest if version is not 'latest'
if [ "${VERSION}" != "latest" ]; then
echo "Tagging also as latest..."
docker tag "${REGISTRY}/${IMAGE_NAME}:${VERSION}" "${REGISTRY}/${IMAGE_NAME}:latest"
fi
# Push the image to the registry
echo "Pushing Docker image ${REGISTRY}/${IMAGE_NAME}:${VERSION} to registry..."
docker push "${REGISTRY}/${IMAGE_NAME}:${VERSION}"
# Push latest tag if version is not 'latest'
if [ "${VERSION}" != "latest" ]; then
echo "Pushing Docker image ${REGISTRY}/${IMAGE_NAME}:latest to registry..."
docker push "${REGISTRY}/${IMAGE_NAME}:latest"
fi
echo "====================================================="
echo "Build and push completed successfully!"
echo "Image: ${REGISTRY}/${IMAGE_NAME}:${VERSION}"
if [ "${VERSION}" != "latest" ]; then
echo "Image: ${REGISTRY}/${IMAGE_NAME}:latest"
fi
echo "====================================================="

View File

@ -27,15 +27,20 @@ if [ "$UPDATE_CODE_FROM_GIT" = "true" ]; then
########################################################################################
# Backup permits folder if it exists
mkdir -p /tmp/content
if [ -d "/app/public/content/permits" ]; then
mv /app/public/content/permits /tmp/content/permits
echo "Permits folder backed up successfully." | tee -a /app/logs/deploy.txt
echo "Backing up server permits folder..." | tee -a /app/logs/deploy.txt
cp -r /app/public/content/permits /tmp/content/
echo "Server permits folder backed up successfully." | tee -a /app/logs/deploy.txt
ls -la /tmp/content/permits >> /app/logs/deploy.txt 2>&1
else
echo "Permits folder not found, skipping backup." | tee -a /app/logs/deploy.txt
echo "Server permits folder not found, will create it after deployment." | tee -a /app/logs/deploy.txt
mkdir -p /tmp/content/permits
fi
# Run rsync with verbose output and itemize-changes
echo "Running rsync..." | tee -a /app/logs/deploy.txt
# Run rsync with verbose output and itemize-changes, excluding the permits folder
echo "Running rsync for main codebase..." | tee -a /app/logs/deploy.txt
rsync -av --itemize-changes \
--exclude='package.json' \
--exclude='package-lock.json' \
@ -53,23 +58,89 @@ if [ "$UPDATE_CODE_FROM_GIT" = "true" ]; then
tail -n 20 /app/logs/deploy.txt # Display the last 20 lines of the log
fi
# Restore permits folder
echo "Restoring permits folder..." | tee -a /app/logs/deploy.txt
if [ -d "/tmp/content/permits" ]; then
# Ensure the destination directory exists
mkdir -p /app/public/content
mv /tmp/content/permits /app/public/content/permits
echo "Permits folder restored successfully." | tee -a /app/logs/deploy.txt
# Check if the Git project contains a permits folder
if [ -d "/tmp/clone/public/content/permits" ] && [ "$(ls -A /tmp/clone/public/content/permits)" ]; then
echo "Found permits files in Git project, preparing to merge..." | tee -a /app/logs/deploy.txt
# Create a temporary directory for Git project permits files
mkdir -p /tmp/git_permits
cp -r /tmp/clone/public/content/permits/* /tmp/git_permits/ 2>/dev/null || true
ls -la /tmp/git_permits >> /app/logs/deploy.txt 2>&1
else
echo "No permits folder to restore." | tee -a /app/logs/deploy.txt
echo "No permits files found in Git project." | tee -a /app/logs/deploy.txt
fi
# Check contents after restoration
echo "Contents of /app/public/content after restoration:" >> /app/logs/deploy.txt
ls -la /app/public/content >> /app/logs/deploy.txt 2>&1
# Restore/merge permits folder
echo "Restoring and merging permits folders..." | tee -a /app/logs/deploy.txt
# Ensure the destination directory exists
mkdir -p /app/public/content/permits
# First restore the server permits
if [ -d "/tmp/content/permits" ]; then
cp -r /tmp/content/permits/* /app/public/content/permits/ 2>/dev/null || true
echo "Server permits restored." | tee -a /app/logs/deploy.txt
fi
# Then merge in the Git project permits (will overwrite if same filename)
if [ -d "/tmp/git_permits" ] && [ "$(ls -A /tmp/git_permits)" ]; then
cp -r /tmp/git_permits/* /app/public/content/permits/ 2>/dev/null || true
echo "Git project permits merged." | tee -a /app/logs/deploy.txt
fi
# Check contents after restoration and merge
echo "Contents of merged permits folder:" >> /app/logs/deploy.txt
ls -la /app/public/content/permits >> /app/logs/deploy.txt 2>&1
########################################################################################
echo "\r\n\r\n Fixing problematic npm packages..." | tee -a /app/logs/deploy.txt
cd /app
# Remove problematic node_modules if they exist
if [ -d "node_modules/abbrev" ]; then
echo "Removing problematic abbrev package..." | tee -a /app/logs/deploy.txt
rm -rf node_modules/abbrev
fi
if [ -d "node_modules/bcrypt" ]; then
echo "Removing problematic bcrypt package..." | tee -a /app/logs/deploy.txt
rm -rf node_modules/bcrypt
fi
# Ensure clean node_modules state
if [ -d "node_modules" ]; then
echo "Cleaning up node_modules..." | tee -a /app/logs/deploy.txt
find node_modules -type d -name "node_modules" -not -path "node_modules" -exec rm -rf {} + 2>/dev/null || true
find node_modules -name "*.node" -type f -delete 2>/dev/null || true
fi
# Install problematic packages individually
echo "Installing abbrev package separately..." | tee -a /app/logs/deploy.txt
npm install abbrev@latest --no-save --no-audit --no-fund || echo "Failed to install abbrev, continuing anyway"
echo "Installing bcrypt package separately with build from source..." | tee -a /app/logs/deploy.txt
npm install bcrypt@latest --build-from-source --no-save --no-audit --no-fund || echo "Failed to install bcrypt, continuing anyway"
# Fix Prisma-specific issues
echo "Fixing Prisma-specific dependencies..." | tee -a /app/logs/deploy.txt
# Remove problematic Prisma nested dependencies
if [ -d "node_modules/@prisma" ]; then
echo "Cleaning up Prisma dependencies..." | tee -a /app/logs/deploy.txt
rm -rf node_modules/@prisma/internals/node_modules/@prisma/engines
rm -rf node_modules/@prisma/internals/node_modules/@prisma/engines-version
rm -rf node_modules/@prisma/client/node_modules/@prisma/engines-version
rm -rf node_modules/.prisma
fi
# Install specific Prisma versions known to work
echo "Installing specific Prisma versions..." | tee -a /app/logs/deploy.txt
npm uninstall prisma @prisma/client --no-save || true
npm install prisma@5.12.0 @prisma/client@5.12.0 --save-exact --no-audit --no-fund || echo "Failed to install Prisma, continuing anyway"
# Ensure Prisma engines are downloaded
echo "Generating Prisma client..." | tee -a /app/logs/deploy.txt
npx prisma generate --schema=./prisma/schema.prisma || echo "Failed to generate Prisma client, continuing anyway"
echo "\r\n\r\n Checking for changes in package files..."
# Determine if package.json or package-lock.json has changed
PACKAGE_CHANGE=0
@ -79,18 +150,36 @@ if [ "$UPDATE_CODE_FROM_GIT" = "true" ]; then
# If package.json or package-lock.json has changed, copy them and reinstall node_modules
if [ "$PACKAGE_CHANGE" -eq 1 ]; then
echo "Package files have changed. Updating packages..."
echo "Package files have changed. Updating packages..." | tee -a /app/logs/deploy.txt
rsync -av /tmp/clone/package.json /app/package.json || echo "Rsync failed: Issue copying package.json"
rsync -av /tmp/clone/package-lock.json /app/package-lock.json || echo "Rsync failed: Issue copying package-lock.json"
echo "Cleaning node_modules directory..." | tee -a /app/logs/deploy.txt
rm -rf /app/node_modules
yes | npx prisma generate
# Clean npm cache
echo "Cleaning npm cache..." | tee -a /app/logs/deploy.txt
npm cache clean --force
echo "Installing dependencies with safer options..." | tee -a /app/logs/deploy.txt
# Try different installation strategies in case of failure
npm install --production --no-optional --legacy-peer-deps || \
npm install --production --no-optional --force || \
npm install --production --no-optional --no-package-lock || \
echo "WARNING: All npm install attempts failed, continuing with deployment" | tee -a /app/logs/deploy.txt
# Generate Prisma client
echo "Generating Prisma client..." | tee -a /app/logs/deploy.txt
npx prisma generate || echo "WARNING: Prisma generate failed" | tee -a /app/logs/deploy.txt
else
echo "Package files have not changed. Skipping package installation."
echo "Package files have not changed. Skipping package installation." | tee -a /app/logs/deploy.txt
fi
cd /app
npm install --no-audit --no-fund --no-optional --omit=optional
npx next build
# Build the Next.js application
echo "Building Next.js application..." | tee -a /app/logs/deploy.txt
npx next build || echo "WARNING: Next.js build failed" | tee -a /app/logs/deploy.txt
# Clean up
# rm -rf /tmp/clone

72
_deploy/fix-npm-issues.sh Normal file
View File

@ -0,0 +1,72 @@
#!/bin/bash
# Fix npm and Prisma dependency issues
# Run this script when connected to the container to resolve installation problems
set -e
LOG_FILE="/app/logs/npm-fix.log"
echo "[$(date)] Starting npm fix script" | tee -a $LOG_FILE
# Navigate to app directory
cd /app
echo "[$(date)] Working in directory: $(pwd)" | tee -a $LOG_FILE
# Clean up problematic packages
echo "[$(date)] Removing problematic packages..." | tee -a $LOG_FILE
rm -rf node_modules/abbrev 2>/dev/null || true
rm -rf node_modules/bcrypt 2>/dev/null || true
rm -rf node_modules/.prisma 2>/dev/null || true
# Clean up Prisma nested modules
echo "[$(date)] Cleaning up Prisma dependencies..." | tee -a $LOG_FILE
rm -rf node_modules/@prisma/internals/node_modules/@prisma/engines 2>/dev/null || true
rm -rf node_modules/@prisma/internals/node_modules/@prisma/engines-version 2>/dev/null || true
rm -rf node_modules/@prisma/client/node_modules/@prisma/engines-version 2>/dev/null || true
# Clean npm cache
echo "[$(date)] Cleaning npm cache..." | tee -a $LOG_FILE
npm cache clean --force
# Fix nested node_modules issues
echo "[$(date)] Removing nested node_modules..." | tee -a $LOG_FILE
find node_modules -type d -name "node_modules" -not -path "node_modules" -exec rm -rf {} + 2>/dev/null || true
# Install critical packages individually
echo "[$(date)] Installing critical packages individually..." | tee -a $LOG_FILE
npm install abbrev@latest --no-save --no-audit --no-fund | tee -a $LOG_FILE
npm install bcrypt@latest --build-from-source --no-save --no-audit --no-fund | tee -a $LOG_FILE
# Install specific Prisma versions
echo "[$(date)] Installing specific Prisma versions..." | tee -a $LOG_FILE
npm uninstall prisma @prisma/client --no-save | tee -a $LOG_FILE
npm install prisma@5.12.0 @prisma/client@5.12.0 --save-exact --no-audit --no-fund | tee -a $LOG_FILE
# Reinstall all dependencies with safer options
echo "[$(date)] Reinstalling all dependencies..." | tee -a $LOG_FILE
npm install --production --no-optional --legacy-peer-deps | tee -a $LOG_FILE
# Generate Prisma client
echo "[$(date)] Generating Prisma client..." | tee -a $LOG_FILE
npx prisma generate | tee -a $LOG_FILE
# Rebuild Next.js
echo "[$(date)] Building Next.js application..." | tee -a $LOG_FILE
npx next build | tee -a $LOG_FILE
echo "[$(date)] Fix script completed. Check for any errors above." | tee -a $LOG_FILE
echo "[$(date)] You may need to restart the application now: npm start" | tee -a $LOG_FILE
# Print instructions for running the script in the background
cat << "EOF"
To run this script in the background (detached from terminal):
nohup /app/_deploy/fix-npm-issues.sh > /app/logs/npm-fix.log 2>&1 &
To monitor progress:
tail -f /app/logs/npm-fix.log
To restart the application after fixing:
pkill -f "npm start" || true
nohup npm start > /app/logs/app.log 2>&1 &
EOF

View File

@ -0,0 +1,114 @@
#!/bin/bash
# Script to migrate from Git clone deployment to standard Docker deployment
set -e
# Print usage
print_usage() {
echo "Usage: $0 [OPTIONS]"
echo "Migrate from Git clone deployment to standard Docker deployment"
echo ""
echo "Options:"
echo " -s, --source CONTAINER Source container name (default: jwpw-app-staging)"
echo " -c, --compose FILE Docker Compose file for new deployment (default: _deploy/standard-docker-compose.yml)"
echo " -b, --backup DIR Backup directory (default: ./deployment_backup)"
echo " -h, --help Show this help message"
exit 1
}
# Default values
SOURCE_CONTAINER="jwpw-app-staging"
COMPOSE_FILE="_deploy/standard-docker-compose.yml"
BACKUP_DIR="./deployment_backup"
# Parse command line arguments
while [[ $# -gt 0 ]]; do
case $1 in
-s|--source)
SOURCE_CONTAINER="$2"
shift 2
;;
-c|--compose)
COMPOSE_FILE="$2"
shift 2
;;
-b|--backup)
BACKUP_DIR="$2"
shift 2
;;
-h|--help)
print_usage
;;
*)
echo "Unknown option: $1"
print_usage
;;
esac
done
echo "=========================================================="
echo "Migration from Git clone deployment to standard deployment"
echo "=========================================================="
echo "Source container: $SOURCE_CONTAINER"
echo "Compose file: $COMPOSE_FILE"
echo "Backup directory: $BACKUP_DIR"
echo ""
# Confirm before proceeding
read -p "Do you want to proceed? (y/n) " -n 1 -r
echo ""
if [[ ! $REPLY =~ ^[Yy]$ ]]; then
echo "Migration aborted."
exit 1
fi
# Check if source container exists
if ! docker ps -a | grep -q "$SOURCE_CONTAINER"; then
echo "Error: Source container '$SOURCE_CONTAINER' not found."
exit 1
fi
# Create backup directory
mkdir -p "$BACKUP_DIR/permits"
mkdir -p "$BACKUP_DIR/uploads"
mkdir -p "$BACKUP_DIR/logs"
echo "Step 1: Backing up data from source container..."
docker cp "$SOURCE_CONTAINER:/app/public/content/permits/." "$BACKUP_DIR/permits/"
docker cp "$SOURCE_CONTAINER:/app/public/content/uploads/." "$BACKUP_DIR/uploads/"
docker cp "$SOURCE_CONTAINER:/app/logs/." "$BACKUP_DIR/logs/"
echo "Backup completed successfully."
echo "Step 2: Checking if the Compose file exists..."
if [ ! -f "$COMPOSE_FILE" ]; then
echo "Error: Compose file '$COMPOSE_FILE' not found."
exit 1
fi
echo "Step 3: Deploying new Docker containers..."
docker-compose -f "$COMPOSE_FILE" up -d
echo "New containers deployed."
# Get the new container name
NEW_CONTAINER=$(docker-compose -f "$COMPOSE_FILE" ps -q nextjs-app)
if [ -z "$NEW_CONTAINER" ]; then
echo "Error: Could not find new container. Please check the deployment."
exit 1
fi
echo "Step 4: Restoring data to new container..."
docker cp "$BACKUP_DIR/permits/." "$NEW_CONTAINER:/app/public/content/permits/"
docker cp "$BACKUP_DIR/uploads/." "$NEW_CONTAINER:/app/public/content/uploads/"
echo "Data restored successfully."
echo "Step 5: Setting correct permissions..."
docker exec -it "$NEW_CONTAINER" chown -R nextjs:nodejs /app/public/content
echo "=========================================================="
echo "Migration completed successfully!"
echo "New container: $NEW_CONTAINER"
echo "Backup saved to: $BACKUP_DIR"
echo ""
echo "To verify the deployment, visit your application URL."
echo "To roll back, stop the new containers and restart the old ones."
echo "=========================================================="

View File

@ -0,0 +1,65 @@
version: "3.8"
services:
nextjs-app:
container_name: mwitnessing-app
build:
context: ..
dockerfile: _deploy/standard.Dockerfile
restart: always
ports:
- "3000:3000"
environment:
- DATABASE_URL=mysql://mwitnessing_user:password@db:3306/mwitnessing
- NEXTAUTH_URL=http://localhost:3000
- NEXTAUTH_SECRET=your-nextauth-secret
volumes:
- app_permits:/app/public/content/permits
- app_uploads:/app/public/content/uploads
- app_logs:/app/logs
depends_on:
- db
networks:
- app-network
db:
container_name: mwitnessing-db
image: mysql:8.0
restart: always
environment:
- MYSQL_ROOT_PASSWORD=rootpassword
- MYSQL_DATABASE=mwitnessing
- MYSQL_USER=mwitnessing_user
- MYSQL_PASSWORD=password
volumes:
- db_data:/var/lib/mysql
ports:
- "3306:3306"
networks:
- app-network
command: --default-authentication-plugin=mysql_native_password
adminer:
container_name: mwitnessing-adminer
image: adminer:latest
restart: always
ports:
- "8080:8080"
depends_on:
- db
networks:
- app-network
volumes:
app_permits:
driver: local
app_uploads:
driver: local
app_logs:
driver: local
db_data:
driver: local
networks:
app-network:
driver: bridge

View File

@ -0,0 +1,57 @@
# Standard Dockerfile for normal deployment
# This builds a complete image with the application code included
# Build stage
FROM node:18-alpine AS builder
WORKDIR /app
# Copy package files and install dependencies
COPY package*.json ./
RUN npm ci
# Copy application code
COPY . .
# Generate Prisma client
RUN npx prisma generate
# Build the Next.js application
RUN npm run build
# Production stage
FROM node:18-alpine AS runner
WORKDIR /app
# Set production environment
ENV NODE_ENV=production
# Install necessary runtime dependencies
RUN apk --no-cache add bash
# Create non-root user for security
RUN addgroup --system --gid 1001 nodejs \
&& adduser --system --uid 1001 nextjs
# Copy necessary files from build stage
COPY --from=builder /app/next.config.js ./
COPY --from=builder /app/public ./public
COPY --from=builder /app/package*.json ./
COPY --from=builder /app/.next/standalone ./
COPY --from=builder /app/.next/static ./.next/static
COPY --from=builder /app/node_modules/.prisma ./node_modules/.prisma
COPY --from=builder /app/node_modules/@prisma ./node_modules/@prisma
# Create required directories with proper permissions
RUN mkdir -p public/content/permits public/content/uploads logs \
&& chown -R nextjs:nodejs public/content logs
# Switch to non-root user
USER nextjs
# Expose the application port
EXPOSE 3000
# Start the application
CMD ["node", "server.js"]

152
_deploy/webhook-deploy.sh Normal file
View File

@ -0,0 +1,152 @@
#!/bin/bash
# Webhook deployment script
# This script is triggered by the webhook receiver when a push event is received
set -e
# Configuration
APP_DIR="/app" # Application directory
REPO_URL="https://git.d-popov.com/popov/mwitnessing.git" # Git repository URL
GIT_USERNAME="${GIT_USERNAME:-deploy}" # Git username
GIT_PASSWORD="${GIT_PASSWORD:-}" # Git password
DOCKER_COMPOSE_FILE="${DOCKER_COMPOSE_FILE:-_deploy/standard-docker-compose.yml}" # Docker Compose file
LOG_FILE="${LOG_FILE:-/app/logs/webhook-deploy.log}" # Log file
# Create log directory if it doesn't exist
mkdir -p "$(dirname "$LOG_FILE")"
# Function to log messages
log() {
echo "[$(date '+%Y-%m-%d %H:%M:%S')] $1" | tee -a "$LOG_FILE"
}
# Get the branch name from the first argument or use main as default
BRANCH="${1:-main}"
log "Starting deployment for branch: $BRANCH"
# Create a temporary directory for the Git clone
TEMP_DIR=$(mktemp -d)
log "Using temporary directory: $TEMP_DIR"
# Clean up function to remove the temporary directory on exit
cleanup() {
log "Cleaning up temporary directory"
rm -rf "$TEMP_DIR"
}
trap cleanup EXIT
# Clone the repository
log "Cloning repository: $REPO_URL (branch: $BRANCH)"
if [ -n "$GIT_PASSWORD" ]; then
# Use authentication with username and password
AUTH_URL="${REPO_URL/https:\/\//https:\/\/$GIT_USERNAME:$GIT_PASSWORD@}"
git clone -b "$BRANCH" --depth 1 "$AUTH_URL" "$TEMP_DIR"
else
# Use authentication with SSH keys
git clone -b "$BRANCH" --depth 1 "$REPO_URL" "$TEMP_DIR"
fi
# Navigate to the cloned repository
cd "$TEMP_DIR"
# Backup important directories
log "Backing up important directories"
mkdir -p "/tmp/webhook_backup/permits" "/tmp/webhook_backup/uploads"
if [ -d "$APP_DIR/public/content/permits" ]; then
cp -r "$APP_DIR/public/content/permits/." "/tmp/webhook_backup/permits/"
log "Permits directory backed up"
fi
if [ -d "$APP_DIR/public/content/uploads" ]; then
cp -r "$APP_DIR/public/content/uploads/." "/tmp/webhook_backup/uploads/"
log "Uploads directory backed up"
fi
# Check if we should rebuild the Docker image
REBUILD=0
if [ -f "$TEMP_DIR/package.json" ] && [ -f "$APP_DIR/package.json" ]; then
if ! cmp -s "$TEMP_DIR/package.json" "$APP_DIR/package.json"; then
log "package.json has changed, triggering full rebuild"
REBUILD=1
elif [ -f "$TEMP_DIR/package-lock.json" ] && [ -f "$APP_DIR/package-lock.json" ]; then
if ! cmp -s "$TEMP_DIR/package-lock.json" "$APP_DIR/package-lock.json"; then
log "package-lock.json has changed, triggering full rebuild"
REBUILD=1
fi
fi
fi
# Check if the Dockerfile or docker-compose file has changed
if [ -f "$TEMP_DIR/_deploy/standard.Dockerfile" ] && [ -f "$APP_DIR/_deploy/standard.Dockerfile" ]; then
if ! cmp -s "$TEMP_DIR/_deploy/standard.Dockerfile" "$APP_DIR/_deploy/standard.Dockerfile"; then
log "Dockerfile has changed, triggering full rebuild"
REBUILD=1
fi
fi
if [ -f "$TEMP_DIR/$DOCKER_COMPOSE_FILE" ] && [ -f "$APP_DIR/$DOCKER_COMPOSE_FILE" ]; then
if ! cmp -s "$TEMP_DIR/$DOCKER_COMPOSE_FILE" "$APP_DIR/$DOCKER_COMPOSE_FILE"; then
log "Docker Compose file has changed, triggering full rebuild"
REBUILD=1
fi
fi
# Deploy the application
if [ "$REBUILD" -eq 1 ]; then
log "Performing full rebuild and restart"
# Copy new code to application directory
rsync -av --exclude 'node_modules' --exclude '.git' --exclude 'public/content/permits' --exclude 'public/content/uploads' "$TEMP_DIR/" "$APP_DIR/"
# Rebuild and restart using Docker Compose
cd "$APP_DIR"
docker-compose -f "$DOCKER_COMPOSE_FILE" up -d --build
else
log "Performing code update without rebuild"
# Copy new code to application directory, preserving special directories
rsync -av --exclude 'node_modules' --exclude '.git' --exclude 'public/content/permits' --exclude 'public/content/uploads' "$TEMP_DIR/" "$APP_DIR/"
# Find container ID for the Next.js app
CONTAINER_ID=$(docker ps -qf "name=nextjs-app")
if [ -n "$CONTAINER_ID" ]; then
log "Restarting Next.js container: $CONTAINER_ID"
docker restart "$CONTAINER_ID"
else
log "Next.js container not found, starting with Docker Compose"
cd "$APP_DIR"
docker-compose -f "$DOCKER_COMPOSE_FILE" up -d
fi
fi
# Restore backed up directories and merge with new content from Git
log "Restoring and merging backed up directories"
# Create permits directory if it doesn't exist
mkdir -p "$APP_DIR/public/content/permits"
# Merge Git permits (if any) with backed up permits
if [ -d "$TEMP_DIR/public/content/permits" ] && [ "$(ls -A "$TEMP_DIR/public/content/permits")" ]; then
log "Copying permits from Git repository"
cp -r "$TEMP_DIR/public/content/permits/." "$APP_DIR/public/content/permits/"
fi
# Restore backed up permits (these will take precedence over Git permits)
if [ -d "/tmp/webhook_backup/permits" ] && [ "$(ls -A "/tmp/webhook_backup/permits")" ]; then
log "Restoring backed up permits"
cp -r "/tmp/webhook_backup/permits/." "$APP_DIR/public/content/permits/"
fi
# Restore uploads (these should only come from the backup, not Git)
if [ -d "/tmp/webhook_backup/uploads" ] && [ "$(ls -A "/tmp/webhook_backup/uploads")" ]; then
log "Restoring backed up uploads"
mkdir -p "$APP_DIR/public/content/uploads"
cp -r "/tmp/webhook_backup/uploads/." "$APP_DIR/public/content/uploads/"
fi
# Clean up backup
rm -rf "/tmp/webhook_backup"
log "Deployment completed successfully"

View File

@ -0,0 +1,39 @@
version: "3.8"
services:
webhook-receiver:
container_name: mwitnessing-webhook
build:
context: ./webhook
dockerfile: Dockerfile
restart: always
ports:
- "9000:9000"
environment:
- WEBHOOK_PORT=9000
- WEBHOOK_SECRET=change-this-secret-in-production
- DEPLOY_SCRIPT=/app/webhook-deploy.sh
- LOG_FILE=/app/logs/webhook.log
- ALLOWED_BRANCHES=main,develop
- ALLOWED_REPOSITORIES=mwhitnessing
# Git credentials for deployment
- GIT_USERNAME=deploy
- GIT_PASSWORD=your-git-password
# Docker configuration
- DOCKER_COMPOSE_FILE=_deploy/standard-docker-compose.yml
volumes:
- ./webhook-receiver.js:/app/webhook-receiver.js
- ./webhook-deploy.sh:/app/webhook-deploy.sh
- webhook_logs:/app/logs
# Mount Docker socket to allow container restart
- /var/run/docker.sock:/var/run/docker.sock
networks:
- app-network
volumes:
webhook_logs:
driver: local
networks:
app-network:
external: true

137
_deploy/webhook-receiver.js Normal file
View File

@ -0,0 +1,137 @@
// Webhook receiver for Gitea push events
// This service listens for webhook events from Gitea and triggers a deployment
// when changes are pushed to the monitored branches
const express = require('express');
const http = require('http');
const crypto = require('crypto');
const { exec } = require('child_process');
const path = require('path');
const fs = require('fs');
// Configuration (can be moved to environment variables)
const config = {
port: process.env.WEBHOOK_PORT || 9000,
secret: process.env.WEBHOOK_SECRET || 'change-this-secret-in-production',
deployScript: process.env.DEPLOY_SCRIPT || path.join(__dirname, 'webhook-deploy.sh'),
logFile: process.env.LOG_FILE || path.join(__dirname, '../logs/webhook.log'),
allowedBranches: (process.env.ALLOWED_BRANCHES || 'main,master').split(','),
allowedRepositories: (process.env.ALLOWED_REPOSITORIES || 'mwhitnessing').split(',')
};
// Create logs directory if it doesn't exist
const logsDir = path.dirname(config.logFile);
if (!fs.existsSync(logsDir)) {
fs.mkdirSync(logsDir, { recursive: true });
}
// Create Express app
const app = express();
app.use(express.json());
// Helper function to log messages
function log(message) {
const timestamp = new Date().toISOString();
const logMessage = `[${timestamp}] ${message}\n`;
console.log(logMessage.trim());
fs.appendFileSync(config.logFile, logMessage);
}
// Verify Gitea webhook signature
function verifySignature(req) {
const signature = req.headers['x-gitea-signature'];
if (!signature) {
return false;
}
const hmac = crypto.createHmac('sha256', config.secret);
const computedSignature = hmac.update(JSON.stringify(req.body)).digest('hex');
return crypto.timingSafeEqual(
Buffer.from(signature),
Buffer.from(computedSignature)
);
}
// Handle Gitea push webhook
app.post('/webhook', (req, res) => {
// Log receipt of webhook
log('Received webhook request');
try {
// Check if webhook signature is valid
if (!verifySignature(req)) {
log('Invalid webhook signature');
return res.status(403).send('Invalid signature');
}
// Extract relevant information from the webhook payload
const { ref, repository } = req.body;
// Check if this is a branch we care about
const branchName = ref.replace('refs/heads/', '');
if (!config.allowedBranches.includes(branchName)) {
log(`Ignoring push to branch: ${branchName}`);
return res.status(200).send('Ignored branch');
}
// Check if this is a repository we care about
const repoName = repository.name;
if (!config.allowedRepositories.includes(repoName)) {
log(`Ignoring push to repository: ${repoName}`);
return res.status(200).send('Ignored repository');
}
// Log the event
log(`Received push event for ${repoName}/${branchName}`);
// Respond to webhook immediately
res.status(200).send('Processing deployment');
// Execute the deployment script with the branch name
const command = `${config.deployScript} ${branchName}`;
log(`Executing: ${command}`);
exec(command, (error, stdout, stderr) => {
if (error) {
log(`Deployment error: ${error.message}`);
return;
}
if (stderr) {
log(`Deployment stderr: ${stderr}`);
}
log(`Deployment stdout: ${stdout}`);
log('Deployment completed successfully');
});
} catch (error) {
log(`Error processing webhook: ${error.message}`);
if (!res.headersSent) {
res.status(500).send('Error processing webhook');
}
}
});
// Start the server
const server = http.createServer(app);
server.listen(config.port, () => {
log(`Webhook receiver listening on port ${config.port}`);
log(`Monitoring branches: ${config.allowedBranches.join(', ')}`);
log(`Monitoring repositories: ${config.allowedRepositories.join(', ')}`);
});
// Handle server shutdown
process.on('SIGTERM', () => {
log('Shutting down webhook receiver...');
server.close(() => {
log('Webhook receiver stopped');
process.exit(0);
});
});
process.on('SIGINT', () => {
log('Shutting down webhook receiver...');
server.close(() => {
log('Webhook receiver stopped');
process.exit(0);
});
});

View File

@ -0,0 +1,28 @@
FROM node:18-alpine
# Install necessary tools
RUN apk --no-cache add git docker-cli bash rsync
WORKDIR /app
# Copy package.json
COPY package.json .
# Install dependencies
RUN npm install express http crypto
# Create logs directory
RUN mkdir -p /app/logs
# Make deployment script executable
COPY webhook-deploy.sh .
RUN chmod +x webhook-deploy.sh
# Copy webhook receiver
COPY webhook-receiver.js .
# Expose webhook port
EXPOSE 9000
# Start the webhook receiver
CMD ["node", "webhook-receiver.js"]

View File

@ -0,0 +1,13 @@
{
"name": "mwitnessing-webhook",
"version": "1.0.0",
"description": "Webhook receiver for automatic deployments",
"main": "webhook-receiver.js",
"scripts": {
"start": "node webhook-receiver.js"
},
"dependencies": {
"express": "^4.18.2",
"crypto": "^1.0.1"
}
}

View File

@ -0,0 +1,19 @@
UPDATE Shift
SET name = CONCAT(
CASE DAYOFWEEK(startTime)
WHEN 1 THEN 'Неделя'
WHEN 2 THEN 'Понеделник'
WHEN 3 THEN 'Вторник'
WHEN 4 THEN 'Сряда'
WHEN 5 THEN 'Четвъртък'
WHEN 6 THEN 'Петък'
WHEN 7 THEN 'Събота'
END,
' ',
DAY(startTime),
', ',
TIME_FORMAT(DATE_ADD(startTime, INTERVAL 3 HOUR), '%H:%i'),
' - ',
TIME_FORMAT(DATE_ADD(endTime, INTERVAL 3 HOUR), '%H:%i')
)
WHERE name = 'Нова смяна';

View File

@ -196,14 +196,28 @@ export default async function handler(req: NextApiRequest, res: NextApiResponse)
// if day is thursday, change the note to "Специален текст с линк към снимка" : public/images/cart_EN.jpg - opened in a popup modal
//check if the day is thursday
const dayOfWeek = common.getDayOfWeekNameEnEnumForDate(shift.date);
if (dayOfWeek == "Thursday") {
shift.notes = `<a href="#" onclick="document.getElementById('imageModal').style.display='block'; return false;" style="color: blue; text-decoration: underline; cursor: pointer;">Специален текст с линк към снимка</a>
<div id="imageModal" style="display: none; position: fixed; z-index: 1; left: 0; top: 0; width: 100%; height: 100%; background-color: rgba(0,0,0,0.7);">
<div style="position: relative; margin: auto; padding: 20px; width: 90%; height: 90%; max-width: 90vw; max-height: 90vh; background-color: white; border-radius: 5px; top: 50%; transform: translateY(-50%); display: flex; flex-direction: column;">
<span onclick="document.getElementById('imageModal').style.display='none'" style="position: absolute; right: 10px; top: 10px; color: #aaa; font-size: 28px; font-weight: bold; cursor: pointer;">&times;</span>
<div style="flex: 1; display: flex; justify-content: center; align-items: center; overflow: hidden;">
<img src="/images/cart_EN.jpg" style="max-width: 100%; max-height: 100%; object-fit: contain;" />
if (dayOfWeek == "Thursday" && shift.requiresTransport) {
// For Thursday, use the regular transport text (if any) but add a special note indicator
let originalNotes = shift.notes || "";
shift.notes = `<a href="#" onclick="document.getElementById('cart-image-modal').style.display='block'; return false;" style="color: #2563eb; text-decoration: underline; cursor: pointer; font-weight: bold; padding: 2px 6px; border: 1px solid #2563eb; border-radius: 4px; background-color: #eff6ff; display: inline-block;">Специални инструкции ↗</a>
${originalNotes}
<div id="cart-image-modal" style="display: none;">
<div class="fixed inset-0 flex items-center justify-center z-50">
<div style="background-color: white; padding: 16px; border-radius: 8px; box-shadow: 0 4px 6px rgba(0, 0, 0, 0.1); position: relative; max-width: fit-content;">
<span onclick="document.getElementById('cart-image-modal').style.display='none'" style="position: absolute; right: 10px; top: 5px; cursor: pointer; font-size: 24px; color: #6b7280;">&times;</span>
<div style="display: flex; gap: 20px; align-items: start;">
<img src="/images/cart_EN.jpg" style="max-height: 60vh; object-fit: contain;" />
<div style="max-width: 300px;">
<h3 style="font-weight: bold; margin-bottom: 10px;">Специални инструкции за четвъртък:</h3>
<ul style="list-style-type: disc; padding-left: 20px;">
<li style="margin-bottom: 8px; color: #e53e3e;"><strong>ВАЖНО:</strong> Моля, извършете всички дейности извън склада, в който се съхраняват количките.</li>
<li style="margin-bottom: 8px;"><strong>Четвъртък - първа смяна:</strong> Заредете брошури и плакати на английски език.</li>
<li style="margin-bottom: 8px;"><strong>Четвъртък - последна смяна:</strong> Подгответе количките за следващия ден с български материали.</li>
</ul>
</div>
</div>
</div>
<div class="fixed inset-0 bg-black opacity-50" onclick="document.getElementById('cart-image-modal').style.display='none'"></div>
</div>
</div>`;
}