mirror of
https://github.com/chrislusf/seaweedfs
synced 2025-09-10 05:12:47 +02:00
435 lines
9.7 KiB
Markdown
435 lines
9.7 KiB
Markdown
# FoundationDB Filer Store Installation Guide
|
|
|
|
This guide covers the installation and setup of the FoundationDB filer store for SeaweedFS.
|
|
|
|
## Prerequisites
|
|
|
|
### FoundationDB Server
|
|
|
|
1. **Install FoundationDB Server**
|
|
|
|
**Ubuntu/Debian:**
|
|
```bash
|
|
# Add FoundationDB repository
|
|
curl -L https://github.com/apple/foundationdb/releases/download/7.1.61/foundationdb-clients_7.1.61-1_amd64.deb -o foundationdb-clients.deb
|
|
curl -L https://github.com/apple/foundationdb/releases/download/7.1.61/foundationdb-server_7.1.61-1_amd64.deb -o foundationdb-server.deb
|
|
|
|
sudo dpkg -i foundationdb-clients.deb foundationdb-server.deb
|
|
```
|
|
|
|
**CentOS/RHEL:**
|
|
```bash
|
|
# Install RPM packages
|
|
wget https://github.com/apple/foundationdb/releases/download/7.1.61/foundationdb-clients-7.1.61-1.el7.x86_64.rpm
|
|
wget https://github.com/apple/foundationdb/releases/download/7.1.61/foundationdb-server-7.1.61-1.el7.x86_64.rpm
|
|
|
|
sudo rpm -Uvh foundationdb-clients-7.1.61-1.el7.x86_64.rpm foundationdb-server-7.1.61-1.el7.x86_64.rpm
|
|
```
|
|
|
|
**macOS:**
|
|
```bash
|
|
# Using Homebrew (if available)
|
|
brew install foundationdb
|
|
|
|
# Or download from GitHub releases
|
|
# https://github.com/apple/foundationdb/releases
|
|
```
|
|
|
|
2. **Initialize FoundationDB Cluster**
|
|
|
|
**Single Node (Development):**
|
|
```bash
|
|
# Start FoundationDB service
|
|
sudo systemctl start foundationdb
|
|
sudo systemctl enable foundationdb
|
|
|
|
# Initialize database
|
|
fdbcli --exec 'configure new single ssd'
|
|
```
|
|
|
|
**Multi-Node Cluster (Production):**
|
|
```bash
|
|
# On each node, edit /etc/foundationdb/fdb.cluster
|
|
# Example: testing:testing@node1:4500,node2:4500,node3:4500
|
|
|
|
# On one node, initialize cluster
|
|
fdbcli --exec 'configure new double ssd'
|
|
```
|
|
|
|
3. **Verify Installation**
|
|
```bash
|
|
fdbcli --exec 'status'
|
|
```
|
|
|
|
### FoundationDB Client Libraries
|
|
|
|
The SeaweedFS FoundationDB integration requires the FoundationDB client libraries.
|
|
|
|
**Ubuntu/Debian:**
|
|
```bash
|
|
sudo apt-get install libfdb-dev
|
|
```
|
|
|
|
**CentOS/RHEL:**
|
|
```bash
|
|
sudo yum install foundationdb-devel
|
|
```
|
|
|
|
**macOS:**
|
|
```bash
|
|
# Client libraries are included with the server installation
|
|
export LIBRARY_PATH=/usr/local/lib
|
|
export CPATH=/usr/local/include
|
|
```
|
|
|
|
## Building SeaweedFS with FoundationDB Support
|
|
|
|
### Download FoundationDB Go Bindings
|
|
|
|
```bash
|
|
go mod init seaweedfs-foundationdb
|
|
go get github.com/apple/foundationdb/bindings/go/src/fdb
|
|
```
|
|
|
|
### Build SeaweedFS
|
|
|
|
```bash
|
|
# Clone SeaweedFS repository
|
|
git clone https://github.com/seaweedfs/seaweedfs.git
|
|
cd seaweedfs
|
|
|
|
# Build with FoundationDB support
|
|
go build -tags foundationdb -o weed
|
|
```
|
|
|
|
### Verify Build
|
|
|
|
```bash
|
|
./weed version
|
|
# Should show version information
|
|
|
|
./weed help
|
|
# Should list available commands
|
|
```
|
|
|
|
## Configuration
|
|
|
|
### Basic Configuration
|
|
|
|
Create or edit `filer.toml`:
|
|
|
|
```toml
|
|
[foundationdb]
|
|
enabled = true
|
|
cluster_file = "/etc/foundationdb/fdb.cluster"
|
|
api_version = 720
|
|
timeout = "5s"
|
|
max_retry_delay = "1s"
|
|
directory_prefix = "seaweedfs"
|
|
```
|
|
|
|
### Environment Variables
|
|
|
|
Alternative configuration via environment variables:
|
|
|
|
```bash
|
|
export WEED_FOUNDATIONDB_ENABLED=true
|
|
export WEED_FOUNDATIONDB_CLUSTER_FILE=/etc/foundationdb/fdb.cluster
|
|
export WEED_FOUNDATIONDB_API_VERSION=720
|
|
export WEED_FOUNDATIONDB_TIMEOUT=5s
|
|
export WEED_FOUNDATIONDB_MAX_RETRY_DELAY=1s
|
|
export WEED_FOUNDATIONDB_DIRECTORY_PREFIX=seaweedfs
|
|
```
|
|
|
|
### Advanced Configuration
|
|
|
|
For production deployments:
|
|
|
|
```toml
|
|
[foundationdb]
|
|
enabled = true
|
|
cluster_file = "/etc/foundationdb/fdb.cluster"
|
|
api_version = 720
|
|
timeout = "30s"
|
|
max_retry_delay = "5s"
|
|
directory_prefix = "seaweedfs_prod"
|
|
|
|
# Path-specific configuration for backups
|
|
[foundationdb.backup]
|
|
enabled = true
|
|
cluster_file = "/etc/foundationdb/fdb.cluster"
|
|
directory_prefix = "seaweedfs_backup"
|
|
location = "/backup"
|
|
timeout = "60s"
|
|
```
|
|
|
|
## Deployment
|
|
|
|
### Single Node Deployment
|
|
|
|
```bash
|
|
# Start SeaweedFS with FoundationDB filer
|
|
./weed server -filer \
|
|
-master.port=9333 \
|
|
-volume.port=8080 \
|
|
-filer.port=8888 \
|
|
-s3.port=8333
|
|
```
|
|
|
|
### Distributed Deployment
|
|
|
|
**Master Servers:**
|
|
```bash
|
|
# Node 1
|
|
./weed master -port=9333 -peers=master1:9333,master2:9333,master3:9333
|
|
|
|
# Node 2
|
|
./weed master -port=9333 -peers=master1:9333,master2:9333,master3:9333 -ip=master2
|
|
|
|
# Node 3
|
|
./weed master -port=9333 -peers=master1:9333,master2:9333,master3:9333 -ip=master3
|
|
```
|
|
|
|
**Filer Servers with FoundationDB:**
|
|
```bash
|
|
# Filer nodes
|
|
./weed filer -master=master1:9333,master2:9333,master3:9333 -port=8888
|
|
```
|
|
|
|
**Volume Servers:**
|
|
```bash
|
|
./weed volume -master=master1:9333,master2:9333,master3:9333 -port=8080
|
|
```
|
|
|
|
### Docker Deployment
|
|
|
|
**docker-compose.yml:**
|
|
```yaml
|
|
version: '3.9'
|
|
services:
|
|
foundationdb:
|
|
image: foundationdb/foundationdb:7.1.61
|
|
ports:
|
|
- "4500:4500"
|
|
volumes:
|
|
- fdb_data:/var/fdb/data
|
|
- fdb_config:/var/fdb/config
|
|
|
|
seaweedfs:
|
|
image: chrislusf/seaweedfs:latest
|
|
command: "server -filer -ip=seaweedfs"
|
|
ports:
|
|
- "9333:9333"
|
|
- "8888:8888"
|
|
- "8333:8333"
|
|
environment:
|
|
WEED_FOUNDATIONDB_ENABLED: "true"
|
|
WEED_FOUNDATIONDB_CLUSTER_FILE: "/var/fdb/config/fdb.cluster"
|
|
volumes:
|
|
- fdb_config:/var/fdb/config
|
|
depends_on:
|
|
- foundationdb
|
|
|
|
volumes:
|
|
fdb_data:
|
|
fdb_config:
|
|
```
|
|
|
|
### Kubernetes Deployment
|
|
|
|
**FoundationDB Operator:**
|
|
```bash
|
|
# Install FoundationDB operator
|
|
kubectl apply -f https://raw.githubusercontent.com/FoundationDB/fdb-kubernetes-operator/main/config/samples/deployment.yaml
|
|
```
|
|
|
|
**SeaweedFS with FoundationDB:**
|
|
```yaml
|
|
apiVersion: apps/v1
|
|
kind: Deployment
|
|
metadata:
|
|
name: seaweedfs-filer
|
|
spec:
|
|
replicas: 3
|
|
selector:
|
|
matchLabels:
|
|
app: seaweedfs-filer
|
|
template:
|
|
metadata:
|
|
labels:
|
|
app: seaweedfs-filer
|
|
spec:
|
|
containers:
|
|
- name: seaweedfs
|
|
image: chrislusf/seaweedfs:latest
|
|
command: ["weed", "filer"]
|
|
env:
|
|
- name: WEED_FOUNDATIONDB_ENABLED
|
|
value: "true"
|
|
- name: WEED_FOUNDATIONDB_CLUSTER_FILE
|
|
value: "/var/fdb/config/cluster_file"
|
|
ports:
|
|
- containerPort: 8888
|
|
volumeMounts:
|
|
- name: fdb-config
|
|
mountPath: /var/fdb/config
|
|
volumes:
|
|
- name: fdb-config
|
|
configMap:
|
|
name: fdb-cluster-config
|
|
```
|
|
|
|
## Testing Installation
|
|
|
|
### Quick Test
|
|
|
|
```bash
|
|
# Start SeaweedFS with FoundationDB
|
|
./weed server -filer &
|
|
|
|
# Test file operations
|
|
echo "Hello FoundationDB" > test.txt
|
|
curl -F file=@test.txt "http://localhost:8888/test/"
|
|
curl "http://localhost:8888/test/test.txt"
|
|
|
|
# Test S3 API
|
|
curl -X PUT "http://localhost:8333/testbucket"
|
|
curl -T test.txt "http://localhost:8333/testbucket/test.txt"
|
|
```
|
|
|
|
### Integration Test Suite
|
|
|
|
```bash
|
|
# Run the provided test suite
|
|
cd test/foundationdb
|
|
make setup
|
|
make test
|
|
```
|
|
|
|
## Performance Tuning
|
|
|
|
### FoundationDB Tuning
|
|
|
|
```bash
|
|
# Configure for high performance
|
|
fdbcli --exec 'configure triple ssd'
|
|
fdbcli --exec 'configure storage_engine=ssd-redwood-1-experimental'
|
|
```
|
|
|
|
### SeaweedFS Configuration
|
|
|
|
```toml
|
|
[foundationdb]
|
|
enabled = true
|
|
cluster_file = "/etc/foundationdb/fdb.cluster"
|
|
timeout = "10s" # Longer timeout for large operations
|
|
max_retry_delay = "2s" # Adjust retry behavior
|
|
directory_prefix = "sw" # Shorter prefix for efficiency
|
|
```
|
|
|
|
### OS-Level Tuning
|
|
|
|
```bash
|
|
# Increase file descriptor limits
|
|
echo "* soft nofile 65536" >> /etc/security/limits.conf
|
|
echo "* hard nofile 65536" >> /etc/security/limits.conf
|
|
|
|
# Adjust network parameters
|
|
echo "net.core.rmem_max = 134217728" >> /etc/sysctl.conf
|
|
echo "net.core.wmem_max = 134217728" >> /etc/sysctl.conf
|
|
sysctl -p
|
|
```
|
|
|
|
## Monitoring and Maintenance
|
|
|
|
### Health Checks
|
|
|
|
```bash
|
|
# FoundationDB cluster health
|
|
fdbcli --exec 'status'
|
|
fdbcli --exec 'status details'
|
|
|
|
# SeaweedFS health
|
|
curl http://localhost:9333/cluster/status
|
|
curl http://localhost:8888/statistics/health
|
|
```
|
|
|
|
### Log Monitoring
|
|
|
|
**FoundationDB Logs:**
|
|
- `/var/log/foundationdb/` (default location)
|
|
- Monitor for errors, warnings, and performance issues
|
|
|
|
**SeaweedFS Logs:**
|
|
```bash
|
|
# Start with verbose logging
|
|
./weed -v=2 server -filer
|
|
```
|
|
|
|
### Backup and Recovery
|
|
|
|
**FoundationDB Backup:**
|
|
```bash
|
|
# Start backup
|
|
fdbbackup start -d file:///path/to/backup -t backup_tag
|
|
|
|
# Monitor backup
|
|
fdbbackup status -t backup_tag
|
|
|
|
# Restore from backup
|
|
fdbrestore start -r file:///path/to/backup -t backup_tag --wait
|
|
```
|
|
|
|
**SeaweedFS Metadata Backup:**
|
|
```bash
|
|
# Export filer metadata
|
|
./weed shell
|
|
> fs.meta.save /path/to/metadata/backup.gz
|
|
```
|
|
|
|
## Troubleshooting
|
|
|
|
### Common Issues
|
|
|
|
1. **Connection Refused**
|
|
- Check FoundationDB service status: `sudo systemctl status foundationdb`
|
|
- Verify cluster file: `cat /etc/foundationdb/fdb.cluster`
|
|
- Check network connectivity: `telnet localhost 4500`
|
|
|
|
2. **API Version Mismatch**
|
|
- Update API version in configuration
|
|
- Rebuild SeaweedFS with matching FDB client library
|
|
|
|
3. **Transaction Conflicts**
|
|
- Reduce transaction scope
|
|
- Implement appropriate retry logic
|
|
- Check for concurrent access patterns
|
|
|
|
4. **Performance Issues**
|
|
- Monitor cluster status: `fdbcli --exec 'status details'`
|
|
- Check data distribution: `fdbcli --exec 'status json'`
|
|
- Verify storage configuration
|
|
|
|
### Debug Mode
|
|
|
|
```bash
|
|
# Enable FoundationDB client tracing
|
|
export FDB_TRACE_ENABLE=1
|
|
export FDB_TRACE_PATH=/tmp/fdb_trace
|
|
|
|
# Start SeaweedFS with debug logging
|
|
./weed -v=3 server -filer
|
|
```
|
|
|
|
### Getting Help
|
|
|
|
1. **FoundationDB Documentation**: https://apple.github.io/foundationdb/
|
|
2. **SeaweedFS Community**: https://github.com/seaweedfs/seaweedfs/discussions
|
|
3. **Issue Reporting**: https://github.com/seaweedfs/seaweedfs/issues
|
|
|
|
For specific FoundationDB filer store issues, include:
|
|
- FoundationDB version and cluster configuration
|
|
- SeaweedFS version and build tags
|
|
- Configuration files (filer.toml)
|
|
- Error messages and logs
|
|
- Steps to reproduce the issue
|