1
0
Fork 0
mirror of https://github.com/chrislusf/seaweedfs synced 2025-08-01 00:12:48 +02:00
seaweedfs/weed/server/volume_server_ui/volume.html
Chris Lu 891a2fb6eb
Admin: misc improvements on admin server and workers. EC now works. (#7055)
* initial design

* added simulation as tests

* reorganized the codebase to move the simulation framework and tests into their own dedicated package

* integration test. ec worker task

* remove "enhanced" reference

* start master, volume servers, filer

Current Status
 Master: Healthy and running (port 9333)
 Filer: Healthy and running (port 8888)
 Volume Servers: All 6 servers running (ports 8080-8085)
🔄 Admin/Workers: Will start when dependencies are ready

* generate write load

* tasks are assigned

* admin start wtih grpc port. worker has its own working directory

* Update .gitignore

* working worker and admin. Task detection is not working yet.

* compiles, detection uses volumeSizeLimitMB from master

* compiles

* worker retries connecting to admin

* build and restart

* rendering pending tasks

* skip task ID column

* sticky worker id

* test canScheduleTaskNow

* worker reconnect to admin

* clean up logs

* worker register itself first

* worker can run ec work and report status

but:
1. one volume should not be repeatedly worked on.
2. ec shards needs to be distributed and source data should be deleted.

* move ec task logic

* listing ec shards

* local copy, ec. Need to distribute.

* ec is mostly working now

* distribution of ec shards needs improvement
* need configuration to enable ec

* show ec volumes

* interval field UI component

* rename

* integration test with vauuming

* garbage percentage threshold

* fix warning

* display ec shard sizes

* fix ec volumes list

* Update ui.go

* show default values

* ensure correct default value

* MaintenanceConfig use ConfigField

* use schema defined defaults

* config

* reduce duplication

* refactor to use BaseUIProvider

* each task register its schema

* checkECEncodingCandidate use ecDetector

* use vacuumDetector

* use volumeSizeLimitMB

* remove

remove

* remove unused

* refactor

* use new framework

* remove v2 reference

* refactor

* left menu can scroll now

* The maintenance manager was not being initialized when no data directory was configured for persistent storage.

* saving config

* Update task_config_schema_templ.go

* enable/disable tasks

* protobuf encoded task configurations

* fix system settings

* use ui component

* remove logs

* interface{} Reduction

* reduce interface{}

* reduce interface{}

* avoid from/to map

* reduce interface{}

* refactor

* keep it DRY

* added logging

* debug messages

* debug level

* debug

* show the log caller line

* use configured task policy

* log level

* handle admin heartbeat response

* Update worker.go

* fix EC rack and dc count

* Report task status to admin server

* fix task logging, simplify interface checking, use erasure_coding constants

* factor in empty volume server during task planning

* volume.list adds disk id

* track disk id also

* fix locking scheduled and manual scanning

* add active topology

* simplify task detector

* ec task completed, but shards are not showing up

* implement ec in ec_typed.go

* adjust log level

* dedup

* implementing ec copying shards and only ecx files

* use disk id when distributing ec shards

🎯 Planning: ActiveTopology creates DestinationPlan with specific TargetDisk
📦 Task Creation: maintenance_integration.go creates ECDestination with DiskId
🚀 Task Execution: EC task passes DiskId in VolumeEcShardsCopyRequest
💾 Volume Server: Receives disk_id and stores shards on specific disk (vs.store.Locations[req.DiskId])
📂 File System: EC shards and metadata land in the exact disk directory planned

* Delete original volume from all locations

* clean up existing shard locations

* local encoding and distributing

* Update docker/admin_integration/EC-TESTING-README.md

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

* check volume id range

* simplify

* fix tests

* fix types

* clean up logs and tests

---------

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
2025-07-30 12:38:03 -07:00

205 lines
6.7 KiB
HTML

<!DOCTYPE html>
<html>
<head>
<title>SeaweedFS {{ .Version }}</title>
<link rel="stylesheet" href="/seaweedfsstatic/bootstrap/3.3.1/css/bootstrap.min.css">
<script type="text/javascript" src="/seaweedfsstatic/javascript/jquery-3.6.0.min.js"></script>
<script type="text/javascript"
src="/seaweedfsstatic/javascript/jquery-sparklines/2.1.2/jquery.sparkline.min.js"></script>
<script type="text/javascript">
$(function () {
var periods = ['second', 'minute', 'hour', 'day'];
for (i = 0; i < periods.length; i++) {
var period = periods[i];
$('.inlinesparkline-' + period).sparkline('html', {
type: 'line',
barColor: 'red',
tooltipSuffix: ' request per ' + period,
});
}
});
</script>
<style>
#jqstooltip {
height: 28px !important;
width: 150px !important;
}
</style>
</head>
<body>
<div class="container">
<div class="page-header">
<h1>
<a href="https://github.com/seaweedfs/seaweedfs"><img src="/seaweedfsstatic/seaweed50x50.png"></img></a>
SeaweedFS <small>{{ .Version }}</small>
</h1>
</div>
<div class="row">
<div class="col-sm-6">
<h2>Disk Stats</h2>
<table class="table table-striped">
<thead>
<tr>
<th>Path</th>
<th>Disk</th>
<th>Total</th>
<th>Free</th>
<th>Usage</th>
</tr>
</thead>
<tbody>
{{ range .DiskStatuses }}
<tr>
<td>{{ .Dir }}</td>
<td>{{ .DiskType }}</td>
<td>{{ bytesToHumanReadable .All }}</td>
<td>{{ bytesToHumanReadable .Free }}</td>
<td>{{ percentFrom .All .Used}}%</td>
</tr>
{{ end }}
</tbody>
</table>
</div>
<div class="col-sm-6">
<h2>System Stats</h2>
<table class="table table-condensed table-striped">
<tr>
<th>Masters</th>
<td>{{.Masters}}</td>
</tr>
<tr>
<th>Weekly # ReadRequests</th>
<td><span class="inlinesparkline-day">{{ .Counters.ReadRequests.WeekCounter.ToList | join }}</span>
</td>
</tr>
<tr>
<th>Daily # ReadRequests</th>
<td><span class="inlinesparkline-hour">{{ .Counters.ReadRequests.DayCounter.ToList | join }}</span>
</td>
</tr>
<tr>
<th>Hourly # ReadRequests</th>
<td><span
class="inlinesparkline-minute">{{ .Counters.ReadRequests.HourCounter.ToList | join }}</span>
</td>
</tr>
<tr>
<th>Last Minute # ReadRequests</th>
<td><span
class="inlinesparkline-second">{{ .Counters.ReadRequests.MinuteCounter.ToList | join }}</span>
</td>
</tr>
{{ range $key, $val := .Stats }}
<tr>
<th>{{ $key }}</th>
<td>{{ $val }}</td>
</tr>
{{ end }}
</table>
</div>
</div>
<div class="row">
<h2>Volumes</h2>
<table class="table table-striped">
<thead>
<tr>
<th>Id</th>
<th>Collection</th>
<th>Disk</th>
<th>Data Size</th>
<th>Files</th>
<th>Trash</th>
<th>TTL</th>
<th>ReadOnly</th>
<th>Version</th>
</tr>
</thead>
<tbody>
{{ range .Volumes }}
<tr>
<td><code>{{ .Id }}</code></td>
<td>{{ .Collection }}</td>
<td>{{ .DiskType }}</td>
<td>{{ bytesToHumanReadable .Size }}</td>
<td>{{ .FileCount }}</td>
<td>{{ .DeleteCount }} / {{bytesToHumanReadable .DeletedByteCount}}</td>
<td>{{ .Ttl }}</td>
<td>{{ .ReadOnly }}</td>
<td>{{ .Version }}</td>
</tr>
{{ end }}
</tbody>
</table>
</div>
{{ if isNotEmpty .RemoteVolumes }}
<div class="row">
<h2>Remote Volumes</h2>
<table class="table table-striped">
<thead>
<tr>
<th>Id</th>
<th>Collection</th>
<th>Size</th>
<th>Files</th>
<th>Trash</th>
<th>Remote</th>
<th>Key</th>
</tr>
</thead>
<tbody>
{{ range .RemoteVolumes }}
<tr>
<td><code>{{ .Id }}</code></td>
<td>{{ .Collection }}</td>
<td>{{ bytesToHumanReadable .Size }}</td>
<td>{{ .FileCount }}</td>
<td>{{ .DeleteCount }} / {{bytesToHumanReadable .DeletedByteCount}}</td>
<td>{{ .RemoteStorageName }}</td>
<td>{{ .RemoteStorageKey }}</td>
</tr>
{{ end }}
</tbody>
</table>
</div>
{{ end }}
{{ if isNotEmpty .EcVolumes }}
<div class="row">
<h2>Erasure Coding Shards</h2>
<table class="table table-striped">
<thead>
<tr>
<th>Id</th>
<th>Collection</th>
<th>Total Size</th>
<th>Shard Details</th>
<th>CreatedAt</th>
</tr>
</thead>
<tbody>
{{ range .EcVolumes }}
<tr>
<td><code>{{ .VolumeId }}</code></td>
<td>{{ .Collection }}</td>
<td>{{ bytesToHumanReadable .Size }}</td>
<td>
{{ range .ShardDetails }}
<span class="label label-info" style="margin-right: 5px;">
{{ .ShardId }}: {{ bytesToHumanReadable .Size }}
</span>
{{ end }}
</td>
<td>{{ .CreatedAt.Format "2006-01-02 15:04" }}</td>
</tr>
{{ end }}
</tbody>
</table>
</div>
{{ end }}
</div>
</body>
</html>