deployment methods - new and docs
This commit is contained in:
Dobromir Popov
347
_deploy/README.md
Normal file
347
_deploy/README.md
Normal file
@ -0,0 +1,347 @@
|
||||
# Deployment Guide for MWitnessing Application
|
||||
|
||||
This guide covers four deployment methods:
|
||||
1. **Standard Docker Deployment** - Builds a complete Docker image with your code
|
||||
2. **Runtime Git Clone Deployment** - Clones the repository at container startup (current method)
|
||||
3. **Webhook-Based Deployment** - Automatically deploys when code is pushed to the repository
|
||||
4. **Registry-Based Deployment** - Builds and pushes images to a private Docker registry
|
||||
|
||||
## Method 1: Standard Docker Deployment (Recommended)
|
||||
|
||||
This method builds a complete Docker image containing your application code, providing:
|
||||
- Faster container startup
|
||||
- No dependency on Git repository during runtime
|
||||
- Better security (no Git credentials in environment variables)
|
||||
- Standard Docker deployment workflow
|
||||
|
||||
### Prerequisites
|
||||
- Docker and Docker Compose installed on the host machine
|
||||
- Access to the application source code
|
||||
|
||||
### Deployment Steps
|
||||
|
||||
1. **Build and start the containers**:
|
||||
```bash
|
||||
cd /path/to/project
|
||||
docker-compose -f _deploy/standard-docker-compose.yml up -d
|
||||
```
|
||||
|
||||
2. **Access the application**:
|
||||
- Web application: http://localhost:3000
|
||||
- Database admin: http://localhost:8080
|
||||
|
||||
3. **Update the application**:
|
||||
```bash
|
||||
# Pull the latest code
|
||||
git pull
|
||||
|
||||
# Rebuild and restart
|
||||
docker-compose -f _deploy/standard-docker-compose.yml up -d --build
|
||||
```
|
||||
|
||||
### Configuration
|
||||
|
||||
Edit the environment variables in `_deploy/standard-docker-compose.yml` before deployment:
|
||||
- `DATABASE_URL`: Database connection string
|
||||
- `NEXTAUTH_URL`: Public URL of your application
|
||||
- `NEXTAUTH_SECRET`: Secret for NextAuth.js encryption
|
||||
|
||||
### Persistent Data
|
||||
|
||||
The following Docker volumes are used to persist data:
|
||||
- `app_permits`: Stores permit files
|
||||
- `app_uploads`: Stores uploaded files
|
||||
- `app_logs`: Stores application logs
|
||||
- `db_data`: Stores database data
|
||||
|
||||
## Method 2: Runtime Git Clone Deployment (Legacy)
|
||||
|
||||
This method clones the repository at container startup, which is the current deployment method.
|
||||
|
||||
### Prerequisites
|
||||
- Docker and Docker Compose installed on the host machine
|
||||
- Git repository access credentials
|
||||
|
||||
### Deployment Steps
|
||||
|
||||
1. **Configure environment variables**:
|
||||
Edit the Docker Compose file (e.g., `_deploy/deoloy.azure.production.yml`) to set:
|
||||
- `GIT_USERNAME`: Username for Git repository access
|
||||
- `GIT_PASSWORD`: Password for Git repository access
|
||||
- `GIT_BRANCH`: Branch to deploy (default: main)
|
||||
- `UPDATE_CODE_FROM_GIT`: Set to "true" to enable Git clone
|
||||
|
||||
2. **Start the containers**:
|
||||
```bash
|
||||
docker-compose -f _deploy/deoloy.azure.production.yml up -d
|
||||
```
|
||||
|
||||
3. **Update the application**:
|
||||
- Updates happen automatically when containers restart
|
||||
- Force an update by restarting the container:
|
||||
```bash
|
||||
docker-compose -f _deploy/deoloy.azure.production.yml restart nextjs-app
|
||||
```
|
||||
|
||||
### How It Works
|
||||
|
||||
The `entrypoint.sh` script:
|
||||
1. Clones the repository at container startup
|
||||
2. Backs up important directories like `permits` and `uploads`
|
||||
3. Syncs new code from the repository
|
||||
4. Restores or merges the backed-up directories
|
||||
5. Rebuilds the application
|
||||
|
||||
## Method 3: Webhook-Based Deployment (Recommended for Production)
|
||||
|
||||
This method uses Gitea webhooks to automatically deploy changes when code is pushed to the repository, providing:
|
||||
- Automated continuous deployment
|
||||
- No manual intervention needed for updates
|
||||
- Immediate deployment of changes
|
||||
- Preservation of user-uploaded content
|
||||
|
||||
### Prerequisites
|
||||
- Docker and Docker Compose installed on the host machine
|
||||
- Gitea repository with webhook capabilities
|
||||
- Server with port 9000 (or your configured port) accessible to Gitea
|
||||
|
||||
### Setup Steps
|
||||
|
||||
1. **Build and start the webhook service**:
|
||||
```bash
|
||||
cd /path/to/project
|
||||
mkdir -p _deploy/webhook
|
||||
cp _deploy/webhook-receiver.js _deploy/webhook/
|
||||
cp _deploy/webhook-deploy.sh _deploy/webhook/
|
||||
docker-compose -f _deploy/webhook-docker-compose.yml up -d
|
||||
```
|
||||
|
||||
2. **Configure Gitea webhook**:
|
||||
- Go to your repository settings in Gitea
|
||||
- Navigate to "Webhooks" > "Add Webhook" > "Gitea"
|
||||
- Set Target URL: `http://your-server-ip:9000/webhook`
|
||||
- Set Content Type: `application/json`
|
||||
- Set Secret: Same as `WEBHOOK_SECRET` in webhook-docker-compose.yml
|
||||
- Select "Just the push event"
|
||||
- Select "Active"
|
||||
- Click "Add Webhook"
|
||||
|
||||
3. **Test the webhook**:
|
||||
- Make a change to your repository and push it
|
||||
- The webhook service will receive the event and trigger a deployment
|
||||
- Check logs: `docker logs mwitnessing-webhook`
|
||||
|
||||
### Configuration
|
||||
|
||||
Edit the environment variables in `_deploy/webhook-docker-compose.yml` before deployment:
|
||||
- `WEBHOOK_SECRET`: Secret for authenticating webhook requests (must match Gitea webhook secret)
|
||||
- `ALLOWED_BRANCHES`: Comma-separated list of branches that trigger deployments
|
||||
- `ALLOWED_REPOSITORIES`: Comma-separated list of repositories to monitor
|
||||
- `GIT_USERNAME` and `GIT_PASSWORD`: Credentials for Git repository access
|
||||
|
||||
### How It Works
|
||||
|
||||
1. **Webhook Receiver**:
|
||||
- Listens for push events from Gitea
|
||||
- Validates the webhook signature using the shared secret
|
||||
- Triggers the deployment script when a valid push event is received
|
||||
|
||||
2. **Deployment Script**:
|
||||
- Clones the repository at the specified branch
|
||||
- Checks if dependencies have changed (package.json, Dockerfile, etc.)
|
||||
- Performs a full rebuild if necessary, or just updates code and restarts
|
||||
- Preserves user-uploaded content during deployment
|
||||
|
||||
3. **Intelligent Updates**:
|
||||
- Intelligently determines whether a full rebuild is needed
|
||||
- Only rebuilds Docker images when necessary (package.json changes, etc.)
|
||||
- Preserves data in volumes across deployments
|
||||
|
||||
### Monitoring and Maintenance
|
||||
|
||||
- **View logs**:
|
||||
```bash
|
||||
docker logs mwitnessing-webhook
|
||||
```
|
||||
|
||||
- **Manual trigger**:
|
||||
```bash
|
||||
docker exec mwitnessing-webhook /app/webhook-deploy.sh main
|
||||
```
|
||||
|
||||
- **Update webhook code**:
|
||||
```bash
|
||||
# Edit webhook files, then:
|
||||
docker-compose -f _deploy/webhook-docker-compose.yml up -d --build
|
||||
```
|
||||
|
||||
## Method 4: Registry-Based Deployment
|
||||
|
||||
This method involves building Docker images locally or in a CI/CD pipeline, pushing them to your private Docker registry (`docker.d-popov.com`), and then deploying using those pre-built images.
|
||||
|
||||
### Prerequisites
|
||||
- Docker and Docker Compose installed on the build machine
|
||||
- Access to your private Docker registry at `docker.d-popov.com`
|
||||
- Authentication configured for the Docker registry
|
||||
|
||||
### Build and Push Process
|
||||
|
||||
1. **Log in to the Docker registry**:
|
||||
```bash
|
||||
docker login docker.d-popov.com -u <username> -p <password>
|
||||
```
|
||||
|
||||
2. **Build the Docker image**:
|
||||
```bash
|
||||
# Navigate to the project root
|
||||
cd /path/to/project
|
||||
|
||||
# Build the image with the jwpw tag
|
||||
docker build -t docker.d-popov.com/jwpw:latest -f _deploy/prod.Dockerfile .
|
||||
```
|
||||
|
||||
3. **Push the image to the registry**:
|
||||
```bash
|
||||
docker push docker.d-popov.com/jwpw:latest
|
||||
```
|
||||
|
||||
4. **Tag with version (optional but recommended)**:
|
||||
```bash
|
||||
# Tag with version number
|
||||
docker tag docker.d-popov.com/jwpw:latest docker.d-popov.com/jwpw:v1.0.0
|
||||
|
||||
# Push the versioned tag
|
||||
docker push docker.d-popov.com/jwpw:v1.0.0
|
||||
```
|
||||
|
||||
### Deployment Process
|
||||
|
||||
1. **On the server, create or update your deployment YAML file**:
|
||||
Create or edit a file like `_deploy/deploy.production.yml`:
|
||||
|
||||
```yaml
|
||||
version: "3"
|
||||
services:
|
||||
nextjs-app:
|
||||
hostname: jwpw-app-production
|
||||
image: docker.d-popov.com/jwpw:latest
|
||||
volumes:
|
||||
- /mnt/docker_volumes/pw-prod/app/public/content/uploads/:/app/public/content/uploads
|
||||
- /mnt/docker_volumes/pw-prod/app/logs:/app/logs
|
||||
- /mnt/docker_volumes/pw-prod/app/public/content/permits/:/app/public/content/permits
|
||||
environment:
|
||||
- APP_ENV=production
|
||||
- NODE_ENV=production
|
||||
- TZ=Europe/Sofia
|
||||
- DATABASE=mysql://username:password@db:3306/database_name
|
||||
restart: always
|
||||
networks:
|
||||
- infrastructure_default
|
||||
- default
|
||||
# Add database service if needed
|
||||
```
|
||||
|
||||
2. **Deploy using Docker Compose**:
|
||||
```bash
|
||||
docker-compose -f _deploy/deploy.production.yml up -d
|
||||
```
|
||||
|
||||
3. **Update the deployment** (when a new image is available):
|
||||
```bash
|
||||
# Pull the latest image
|
||||
docker pull docker.d-popov.com/jwpw:latest
|
||||
|
||||
# Restart the service
|
||||
docker-compose -f _deploy/deploy.production.yml up -d
|
||||
```
|
||||
|
||||
### Continuous Integration/Continuous Deployment (CI/CD)
|
||||
|
||||
This method is ideal for integration with CI/CD pipelines. For example, you could set up a GitHub Actions or GitLab CI pipeline that:
|
||||
|
||||
1. Builds the Docker image
|
||||
2. Runs tests
|
||||
3. Pushes the image to your private registry
|
||||
4. Triggers a deployment on your server
|
||||
|
||||
### Advantages of Registry-Based Deployment
|
||||
|
||||
- **Consistent environments**: The exact same image is used in all environments
|
||||
- **Faster deployments**: No building on the production server
|
||||
- **Rollback capability**: Easy to roll back to a previous version by specifying an older tag
|
||||
- **Better separation of concerns**: Build and deployment processes are separated
|
||||
- **Audit trail**: Registry maintains a history of image versions
|
||||
|
||||
### Current Use
|
||||
|
||||
This method is currently used for your staging environment, as seen in `_deploy/deoloy.azure.staging.yml`, which references `docker.d-popov.com/jwpw:latest` as the image source.
|
||||
|
||||
### Using the Automated Build Script
|
||||
|
||||
For convenience, an automated build script is included that handles the building and pushing process:
|
||||
|
||||
1. **Make the script executable**:
|
||||
```bash
|
||||
chmod +x _deploy/build-and-push.sh
|
||||
```
|
||||
|
||||
2. **Run the script** (from the project root):
|
||||
```bash
|
||||
# Build and push with 'latest' tag
|
||||
./_deploy/build-and-push.sh
|
||||
|
||||
# Build and push with specific version tag
|
||||
./_deploy/build-and-push.sh v1.2.3
|
||||
```
|
||||
|
||||
3. **What the script does**:
|
||||
- Builds the Docker image using `_deploy/prod.Dockerfile`
|
||||
- Tags the image with the specified version and 'latest'
|
||||
- Pushes both tags to your private Docker registry
|
||||
- Performs basic error checking (Docker installed, logged into registry)
|
||||
|
||||
This script simplifies the process of creating and publishing new versions of your application, making it easier to maintain consistent deployment practices.
|
||||
|
||||
## Migrating Between Methods
|
||||
|
||||
### From Git Clone to Standard Docker
|
||||
|
||||
1. Back up your data:
|
||||
```bash
|
||||
docker cp <container_name>:/app/public/content/permits ./permits_backup
|
||||
docker cp <container_name>:/app/public/content/uploads ./uploads_backup
|
||||
```
|
||||
|
||||
2. Stop the old containers:
|
||||
```bash
|
||||
docker-compose -f _deploy/deoloy.azure.production.yml down
|
||||
```
|
||||
|
||||
3. Deploy using the standard method:
|
||||
```bash
|
||||
docker-compose -f _deploy/standard-docker-compose.yml up -d
|
||||
```
|
||||
|
||||
4. Restore your data if needed:
|
||||
```bash
|
||||
docker cp ./permits_backup/. <new_container_name>:/app/public/content/permits
|
||||
docker cp ./uploads_backup/. <new_container_name>:/app/public/content/uploads
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Docker Logs
|
||||
View container logs:
|
||||
```bash
|
||||
docker logs <container_name>
|
||||
```
|
||||
|
||||
### Application Logs
|
||||
Application logs are stored in:
|
||||
- `/app/logs` inside the container
|
||||
- The `app_logs` Docker volume
|
||||
|
||||
### Common Issues
|
||||
- **Database connection error**: Check the `DATABASE_URL` environment variable
|
||||
- **Missing files**: Check volume mappings and permissions
|
||||
- **Container exits immediately**: Check the logs for startup errors
|
||||
Reference in New Issue
Block a user