1
0
Fork 0
mirror of https://github.com/chrislusf/seaweedfs synced 2025-09-19 01:30:23 +02:00
seaweedfs/weed/pb/schema_pb/mq_schema.pb.go
Chris Lu a7fdc0d137
Message Queue: Add sql querying (#7185)
* feat: Phase 1 - Add SQL query engine foundation for MQ topics

Implements core SQL infrastructure with metadata operations:

New Components:
- SQL parser integration using github.com/xwb1989/sqlparser
- Query engine framework in weed/query/engine/
- Schema catalog mapping MQ topics to SQL tables
- Interactive SQL CLI command 'weed sql'

Supported Operations:
- SHOW DATABASES (lists MQ namespaces)
- SHOW TABLES (lists MQ topics)
- SQL statement parsing and routing
- Error handling and result formatting

Key Design Decisions:
- MQ namespaces ↔ SQL databases
- MQ topics ↔ SQL tables
- Parquet message storage ready for querying
- Backward-compatible schema evolution support

Testing:
- Unit tests for core engine functionality
- Command integration tests
- Parse error handling validation

Assumptions (documented in code):
- All MQ messages stored in Parquet format
- Schema evolution maintains backward compatibility
- MySQL-compatible SQL syntax via sqlparser
- Single-threaded usage per SQL session

Next Phase: DDL operations (CREATE/ALTER/DROP TABLE)

* feat: Phase 2 - Add DDL operations and real MQ broker integration

Implements comprehensive DDL support for MQ topic management:

New Components:
- Real MQ broker connectivity via BrokerClient
- CREATE TABLE → ConfigureTopic gRPC calls
- DROP TABLE → DeleteTopic operations
- DESCRIBE table → Schema introspection
- SQL type mapping (SQL ↔ MQ schema types)

Enhanced Features:
- Live topic discovery from MQ broker
- Fallback to cached/sample data when broker unavailable
- MySQL-compatible DESCRIBE output
- Schema validation and error handling
- CREATE TABLE with column definitions

Key Infrastructure:
- broker_client.go: gRPC communication with MQ broker
- sql_types.go: Bidirectional SQL/MQ type conversion
- describe.go: Table schema introspection
- Enhanced engine.go: Full DDL routing and execution

Supported SQL Operations:
 SHOW DATABASES, SHOW TABLES (live + fallback)
 CREATE TABLE table_name (col1 INT, col2 VARCHAR(50), ...)
 DROP TABLE table_name
 DESCRIBE table_name / SHOW COLUMNS FROM table_name

Known Limitations:
- SQL parser issues with reserved keywords (e.g., 'timestamp')
- Requires running MQ broker for full functionality
- ALTER TABLE not yet implemented
- DeleteTopic method needs broker-side implementation

Architecture Decisions:
- Broker discovery via filer lock mechanism (same as shell commands)
- Graceful fallback when broker unavailable
- ConfigureTopic for CREATE TABLE with 6 default partitions
- Schema versioning ready for ALTER TABLE support

Testing:
- Unit tests updated with filer address parameter
- Integration tests for DDL operations
- Error handling for connection failures

Next Phase: SELECT query execution with Parquet scanning

* fix: Resolve High Priority TODOs - Real MQ Broker Integration

 COMPLETED HIGH PRIORITY TODOs:

🔧 **Real FilerClient Integration** (engine.go:131)
- Implemented GetFilerClient() method in BrokerClient
- Added filerClientImpl with full FilerClient interface compliance
- Added AdjustedUrl() and GetDataCenter() methods
- Real filerClient connection replaces nil fallback

🔧 **Partition Discovery via MQ Broker** (hybrid_message_scanner.go:116)
- Added ListTopicPartitions() method using topic configuration
- Implemented discoverTopicPartitions() in HybridMessageScanner
- Reads actual partition count from BrokerPartitionAssignments
- Generates proper partition ranges based on topic.PartitionCount

📋 **Technical Fixes:**
- Fixed compilation errors with undefined variables
- Proper error handling with filerClientErr variable
- Corrected ConfigureTopicResponse field usage (BrokerPartitionAssignments vs PartitionCount)
- Complete FilerClient interface implementation

🎯 **Impact:**
- SQL engine now connects to real MQ broker infrastructure
- Actual topic partition discovery instead of hardcoded defaults
- Production-ready broker integration with graceful fallbacks
- Maintains backward compatibility with sample data when broker unavailable

 All tests passing - High priority TODO resolution complete!

Next: Schema-aware message parsing and time filter optimization.

* feat: Time Filter Extraction - Complete Performance Optimization

 FOURTH HIGH PRIORITY TODO COMPLETED!

 **Time Filter Extraction & Push-Down Optimization** (engine.go:198-199)
- Replaced hardcoded StartTimeNs=0, StopTimeNs=0 with intelligent extraction
- Added extractTimeFilters() with recursive WHERE clause analysis
- Smart time column detection (\_timestamp_ns, created_at, timestamp, etc.)
- Comprehensive time value parsing (nanoseconds, ISO dates, datetime formats)
- Operator reversal handling (column op value vs value op column)

🧠 **Intelligent WHERE Clause Processing:**
- AND expressions: Combine time bounds (intersection) 
- OR expressions: Skip extraction (safety) 
- Parentheses: Recursive unwrapping 
- Comparison operators: >, >=, <, <=, = 
- Multiple time formats: nanoseconds, RFC3339, date-only, datetime 

🚀 **Performance Impact:**
- Push-down filtering to hybrid scanner level
- Reduced data scanning at source (live logs + Parquet files)
- Time-based partition pruning potential
- Significant performance gains for time-series queries

📊 **Comprehensive Testing (21 tests passing):**
-  Time filter extraction (6 test scenarios)
-  Time column recognition (case-insensitive)
-  Time value parsing (5 formats)
-  Full integration with SELECT queries
-  Backward compatibility maintained

💡 **Real-World Query Examples:**
Before: Scans ALL data, filters in memory
  SELECT * FROM events WHERE \_timestamp_ns > 1672531200000000000;

After: Scans ONLY relevant time range at source level
  → StartTimeNs=1672531200000000000, StopTimeNs=0
  → Massive performance improvement for large datasets!

🎯 **Production Ready Features:**
- Multiple time column formats supported
- Graceful fallbacks for invalid dates
- OR clause safety (avoids incorrect optimization)
- Comprehensive error handling

**ALL MEDIUM PRIORITY TODOs NOW READY FOR NEXT PHASEtest ./weed/query/engine/ -v* 🎉

* feat: Extended WHERE Operators - Complete Advanced Filtering

 **EXTENDED WHERE OPERATORS IMPLEMENTEDtest ./weed/query/engine/ -v | grep -E PASS

* feat: Enhanced SQL CLI Experience

 COMPLETE ENHANCED CLI IMPLEMENTATION:

🚀 **Multiple Execution Modes:**
- Interactive shell with enhanced prompts and context
- Single query execution: --query 'SQL' --output format
- Batch file processing: --file queries.sql --output csv
- Database context switching: --database dbname

📊 **Multi-Format Output:**
- Table format (ASCII) - default for interactive
- JSON format - structured data for programmatic use
- CSV format - spreadsheet-friendly output
- Smart auto-detection based on execution mode

⚙️ **Enhanced Interactive Shell:**
- Database context switching: USE database_name;
- Output format switching: \format table|json|csv
- Command history tracking (basic implementation)
- Enhanced help with WHERE operator examples
- Contextual prompts: seaweedfs:dbname>

🛠️ **Production Features:**
- Comprehensive error handling (JSON + user-friendly)
- Query execution timing and performance metrics
- 30-second timeout protection with graceful handling
- Real MQ integration with hybrid data scanning

📖 **Complete CLI Interface:**
- Full flag support: --server, --interactive, --file, --output, --database, --query
- Auto-detection of execution mode and output format
- Structured help system with practical examples
- Batch processing with multi-query file support

💡 **Advanced WHERE Integration:**
All extended operators (<=, >=, !=, LIKE, IN) fully supported
across all execution modes and output formats.

🎯 **Usage Examples:**
- weed sql --interactive
- weed sql --query 'SHOW DATABASES' --output json
- weed sql --file queries.sql --output csv
- weed sql --database analytics --interactive

Enhanced CLI experience complete - production ready! 🚀

* Delete test_utils_test.go

* fmt

* integer conversion

* show databases works

* show tables works

* Update describe.go

* actual column types

* Update .gitignore

* scan topic messages

* remove emoji

* support aggregation functions

* column name case insensitive, better auto column names

* fmt

* fix reading system fields

* use parquet statistics for optimization

* remove emoji

* parquet file generate stats

* scan all files

* parquet file generation remember the sources also

* fmt

* sql

* truncate topic

* combine parquet results with live logs

* explain

* explain the execution plan

* add tests

* improve tests

* skip

* use mock for testing

* add tests

* refactor

* fix after refactoring

* detailed logs during explain. Fix bugs on reading live logs.

* fix decoding data

* save source buffer index start for log files

* process buffer from brokers

* filter out already flushed messages

* dedup with buffer start index

* explain with broker buffer

* the parquet file should also remember the first buffer_start attribute from the sources

* parquet file can query messages in broker memory, if log files do not exist

* buffer start stored as 8 bytes

* add jdbc

* add postgres protocol

* Revert "add jdbc"

This reverts commit a6e48b7690.

* hook up seaweed sql engine

* setup integration test for postgres

* rename to "weed db"

* return fast on error

* fix versioning

* address comments

* address some comments

* column name can be on left or right in where conditions

* avoid sample data

* remove sample data

* de-support alter table and drop table

* address comments

* read broker, logs, and parquet files

* Update engine.go

* address some comments

* use schema instead of inferred result types

* fix tests

* fix todo

* fix empty spaces and coercion

* fmt

* change to pg_query_go

* fix tests

* fix tests

* fmt

* fix: Enable CGO in Docker build for pg_query_go dependency

The pg_query_go library requires CGO to be enabled as it wraps the libpg_query C library.
Added gcc and musl-dev dependencies to the Docker build for proper compilation.

* feat: Replace pg_query_go with lightweight SQL parser (no CGO required)

- Remove github.com/pganalyze/pg_query_go/v6 dependency to avoid CGO requirement
- Implement lightweight SQL parser for basic SELECT, SHOW, and DDL statements
- Fix operator precedence in WHERE clause parsing (handle AND/OR before comparisons)
- Support INTEGER, FLOAT, and STRING literals in WHERE conditions
- All SQL engine tests passing with new parser
- PostgreSQL integration tests can now build without CGO

The lightweight parser handles the essential SQL features needed for the
SeaweedFS query engine while maintaining compatibility and avoiding CGO
dependencies that caused Docker build issues.

* feat: Add Parquet logical types to mq_schema.proto

Added support for Parquet logical types in SeaweedFS message queue schema:
- TIMESTAMP: UTC timestamp in microseconds since epoch with timezone flag
- DATE: Date as days since Unix epoch (1970-01-01)
- DECIMAL: Arbitrary precision decimal with configurable precision/scale
- TIME: Time of day in microseconds since midnight

These types enable advanced analytics features:
- Time-based filtering and window functions
- Date arithmetic and year/month/day extraction
- High-precision numeric calculations
- Proper time zone handling for global deployments

Regenerated protobuf Go code with new scalar types and value messages.

* feat: Enable publishers to use Parquet logical types

Enhanced MQ publishers to utilize the new logical types:
- Updated convertToRecordValue() to use TimestampValue instead of string RFC3339
- Added DateValue support for birth_date field (days since epoch)
- Added DecimalValue support for precise_amount field with configurable precision/scale
- Enhanced UserEvent struct with PreciseAmount and BirthDate fields
- Added convertToDecimal() helper using big.Rat for precise decimal conversion
- Updated test data generator to produce varied birth dates (1970-2005) and precise amounts

Publishers now generate structured data with proper logical types:
-  TIMESTAMP: Microsecond precision UTC timestamps
-  DATE: Birth dates as days since Unix epoch
-  DECIMAL: Precise amounts with 18-digit precision, 4-decimal scale

Successfully tested with PostgreSQL integration - all topics created with logical type data.

* feat: Add logical type support to SQL query engine

Extended SQL engine to handle new Parquet logical types:
- Added TimestampValue comparison support (microsecond precision)
- Added DateValue comparison support (days since epoch)
- Added DecimalValue comparison support with string conversion
- Added TimeValue comparison support (microseconds since midnight)
- Enhanced valuesEqual(), valueLessThan(), valueGreaterThan() functions
- Added decimalToString() helper for precise decimal-to-string conversion
- Imported math/big for arbitrary precision decimal handling

The SQL engine can now:
-  Compare TIMESTAMP values for filtering (e.g., WHERE timestamp > 1672531200000000000)
-  Compare DATE values for date-based queries (e.g., WHERE birth_date >= 12345)
-  Compare DECIMAL values for precise financial calculations
-  Compare TIME values for time-of-day filtering

Next: Add YEAR(), MONTH(), DAY() extraction functions for date analytics.

* feat: Add window function foundation with timestamp support

Added comprehensive foundation for SQL window functions with timestamp analytics:

Core Window Function Types:
- WindowSpec with PartitionBy and OrderBy support
- WindowFunction struct for ROW_NUMBER, RANK, LAG, LEAD
- OrderByClause for timestamp-based ordering
- Extended SelectStatement to support WindowFunctions field

Timestamp Analytics Functions:
 ApplyRowNumber() - ROW_NUMBER() OVER (ORDER BY timestamp)
 ExtractYear() - Extract year from TIMESTAMP logical type
 ExtractMonth() - Extract month from TIMESTAMP logical type
 ExtractDay() - Extract day from TIMESTAMP logical type
 FilterByYear() - Filter records by timestamp year

Foundation for Advanced Window Functions:
- LAG/LEAD for time-series access to previous/next values
- RANK/DENSE_RANK for temporal ranking
- FIRST_VALUE/LAST_VALUE for window boundaries
- PARTITION BY support for grouped analytics

This enables sophisticated time-series analytics like:
- SELECT *, ROW_NUMBER() OVER (ORDER BY timestamp) FROM user_events WHERE EXTRACT(YEAR FROM timestamp) = 2024
- Trend analysis over time windows
- Session analytics with LAG/LEAD functions
- Time-based ranking and percentiles

Ready for production time-series analytics with proper timestamp logical type support! 🚀

* fmt

* fix

* fix describe issue

* fix tests, avoid panic

* no more mysql

* timeout client connections

* Update SQL_FEATURE_PLAN.md

* handling errors

* remove sleep

* fix splitting multiple SQLs

* fixes

* fmt

* fix

* Update weed/util/log_buffer/log_buffer.go

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

* Update SQL_FEATURE_PLAN.md

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

* code reuse

* fix

* fix

* feat: Add basic arithmetic operators (+, -, *, /, %) with comprehensive tests

- Implement EvaluateArithmeticExpression with support for all basic operators
- Handle type conversions between int, float, string, and boolean
- Add proper error handling for division/modulo by zero
- Include 14 comprehensive test cases covering all edge cases
- Support mixed type arithmetic (int + float, string numbers, etc.)

All tests passing 

* feat: Add mathematical functions ROUND, CEIL, FLOOR, ABS with comprehensive tests

- Implement ROUND with optional precision parameter
- Add CEIL function for rounding up to nearest integer
- Add FLOOR function for rounding down to nearest integer
- Add ABS function for absolute values with type preservation
- Support all numeric types (int32, int64, float32, double)
- Comprehensive test suite with 20+ test cases covering:
  - Positive/negative numbers
  - Integer/float type preservation
  - Precision handling for ROUND
  - Null value error handling
  - Edge cases (zero, large numbers)

All tests passing 

* feat: Add date/time functions CURRENT_DATE, CURRENT_TIMESTAMP, EXTRACT with comprehensive tests

- Implement CURRENT_DATE returning YYYY-MM-DD format
- Add CURRENT_TIMESTAMP returning TimestampValue with microseconds
- Add CURRENT_TIME returning HH:MM:SS format
- Add NOW() as alias for CURRENT_TIMESTAMP
- Implement comprehensive EXTRACT function supporting:
  - YEAR, MONTH, DAY, HOUR, MINUTE, SECOND
  - QUARTER, WEEK, DOY (day of year), DOW (day of week)
  - EPOCH (Unix timestamp)
- Support multiple input formats:
  - TimestampValue (microseconds)
  - String dates (multiple formats)
  - Unix timestamps (int64 seconds)
- Comprehensive test suite with 15+ test cases covering:
  - All date/time constants
  - Extract from different value types
  - Error handling for invalid inputs
  - Timezone handling

All tests passing 

* feat: Add DATE_TRUNC function with comprehensive tests

- Implement comprehensive DATE_TRUNC function supporting:
  - Time precisions: microsecond, millisecond, second, minute, hour
  - Date precisions: day, week, month, quarter, year, decade, century, millennium
  - Support both singular and plural forms (e.g., 'minute' and 'minutes')
- Enhanced date/time parsing with proper timezone handling:
  - Assume local timezone for non-timezone string formats
  - Support UTC formats with explicit timezone indicators
  - Consistent behavior between parsing and truncation
- Comprehensive test suite with 11 test cases covering:
  - All supported precisions from microsecond to year
  - Multiple input types (TimestampValue, string dates)
  - Edge cases (null values, invalid precisions)
  - Timezone consistency validation

All tests passing 

* feat: Add comprehensive string functions with extensive tests

Implemented String Functions:
- LENGTH: Get string length (supports all value types)
- UPPER/LOWER: Case conversion
- TRIM/LTRIM/RTRIM: Whitespace removal (space, tab, newline, carriage return)
- SUBSTRING: Extract substring with optional length (SQL 1-based indexing)
- CONCAT: Concatenate multiple values (supports mixed types, skips nulls)
- REPLACE: Replace all occurrences of substring
- POSITION: Find substring position (1-based, 0 if not found)
- LEFT/RIGHT: Extract leftmost/rightmost characters
- REVERSE: Reverse string with proper Unicode support

Key Features:
- Robust type conversion (string, int, float, bool, bytes)
- Unicode-safe operations (proper rune handling in REVERSE)
- SQL-compatible indexing (1-based for SUBSTRING, POSITION)
- Comprehensive error handling with descriptive messages
- Mixed-type support (e.g., CONCAT number with string)

Helper Functions:
- valueToString: Convert any schema_pb.Value to string
- valueToInt64: Convert numeric values to int64

Comprehensive test suite with 25+ test cases covering:
- All string functions with typical use cases
- Type conversion scenarios (numbers, booleans)
- Edge cases (empty strings, null values, Unicode)
- Error conditions and boundary testing

All tests passing 

* refactor: Split sql_functions.go into smaller, focused files

**File Structure Before:**
- sql_functions.go (850+ lines)
- sql_functions_test.go (1,205+ lines)

**File Structure After:**
- function_helpers.go (105 lines) - shared utility functions
- arithmetic_functions.go (205 lines) - arithmetic operators & math functions
- datetime_functions.go (170 lines) - date/time functions & constants
- string_functions.go (335 lines) - string manipulation functions
- arithmetic_functions_test.go (560 lines) - tests for arithmetic & math
- datetime_functions_test.go (370 lines) - tests for date/time functions
- string_functions_test.go (270 lines) - tests for string functions

**Benefits:**
 Better organization by functional domain
 Easier to find and maintain specific function types
 Smaller, more manageable file sizes
 Clear separation of concerns
 Improved code readability and navigation
 All tests passing - no functionality lost

**Total:** 7 focused files (1,455 lines) vs 2 monolithic files (2,055+ lines)

This refactoring improves maintainability while preserving all functionality.

* fix: Improve test stability for date/time functions

**Problem:**
- CURRENT_TIMESTAMP test had timing race condition that could cause flaky failures
- CURRENT_DATE test could fail if run exactly at midnight boundary
- Tests were too strict about timing precision without accounting for system variations

**Root Cause:**
- Test captured before/after timestamps and expected function result to be exactly between them
- No tolerance for clock precision differences, NTP adjustments, or system timing variations
- Date boundary race condition around midnight transitions

**Solution:**
 **CURRENT_TIMESTAMP test**: Added 100ms tolerance buffer to account for:
  - Clock precision differences between time.Now() calls
  - System timing variations and NTP corrections
  - Microsecond vs nanosecond precision differences

 **CURRENT_DATE test**: Enhanced to handle midnight boundary crossings:
  - Captures date before and after function call
  - Accepts either date value in case of midnight transition
  - Prevents false failures during overnight test runs

**Testing:**
- Verified with repeated test runs (5x iterations) - all pass consistently
- Full test suite passes - no regressions introduced
- Tests are now robust against timing edge cases

**Impact:**
🚀 **Eliminated flaky test failures** while maintaining function correctness validation
🔧 **Production-ready testing** that works across different system environments
 **CI/CD reliability** - tests won't fail due to timing variations

* heap sort the data sources

* int overflow

* Update README.md

* redirect GetUnflushedMessages to brokers hosting the topic partition

* Update postgres-examples/README.md

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

* clean up

* support limit with offset

* Update SQL_FEATURE_PLAN.md

* limit with offset

* ensure int conversion correctness

* Update weed/query/engine/hybrid_message_scanner.go

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

* avoid closing closed channel

* support string concatenation ||

* int range

* using consts; avoid test data in production binary

* fix tests

* Update SQL_FEATURE_PLAN.md

* fix "use db"

* address comments

* fix comments

* Update mocks_test.go

* comment

* improve docker build

* normal if no partitions found

* fix build docker

* Update SQL_FEATURE_PLAN.md

* upgrade to raft v1.1.4 resolving race in leader

* raft 1.1.5

* Update SQL_FEATURE_PLAN.md

* Revert "raft 1.1.5"

This reverts commit 5f3bdfadbf.

* Revert "upgrade to raft v1.1.4 resolving race in leader"

This reverts commit fa620f0223.

* Fix data race in FUSE GetAttr operation

- Add shared lock to GetAttr when accessing file handle entries
- Prevents concurrent access between Write (ExclusiveLock) and GetAttr (SharedLock)
- Fixes race on entry.Attributes.FileSize field during concurrent operations
- Write operations already use ExclusiveLock, now GetAttr uses SharedLock for consistency

Resolves race condition:
Write at weedfs_file_write.go:62 vs Read at filechunks.go:28

* Update weed/mq/broker/broker_grpc_query.go

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

* clean up

* Update db.go

* limit with offset

* Update Makefile

* fix id*2

* fix math

* fix string function bugs and add tests

* fix string concat

* ensure empty spaces for literals

* add ttl for catalog

* fix time functions

* unused code path

* database qualifier

* refactor

* extract

* recursive functions

* add cockroachdb parser

* postgres only

* test SQLs

* fix tests

* fix count *

* fix where clause

* fix limit offset

* fix  count fast path

* fix tests

* func name

* fix database qualifier

* fix tests

* Update engine.go

* fix tests

* fix jaeger

https://github.com/advisories/GHSA-2w8w-qhg4-f78j

* remove order by, group by, join

* fix extract

* prevent single quote in the string

* skip control messages

* skip control message when converting to parquet files

* psql change database

* remove old code

* remove old parser code

* rename file

* use db

* fix alias

* add alias test

* compare int64

* fix _timestamp_ns comparing

* alias support

* fix fast path count

* rendering data sources tree

* reading data sources

* reading parquet logic types

* convert logic types to parquet

* go mod

* fmt

* skip decimal types

* use UTC

* add warning if broker fails

* add user password file

* support IN

* support INTERVAL

* _ts as timestamp column

* _ts can compare with string

* address comments

* is null / is not null

* go mod

* clean up

* restructure execution plan

* remove extra double quotes

* fix converting logical types to parquet

* decimal

* decimal support

* do not skip decimal logical types

* making row-building schema-aware and alignment-safe

Emit parquet.NullValue() for missing fields to keep row shapes aligned.
Always advance list level and safely handle nil list values.
Add toParquetValueForType(...) to coerce values to match the declared Parquet type (e.g., STRING/BYTES via byte array; numeric/string conversions for INT32/INT64/DOUBLE/FLOAT/BOOL/TIMESTAMP/DATE/TIME).
Keep nil-byte guards for ByteArray.

* tests for growslice

* do not batch

* live logs in sources can be skipped in execution plan

* go mod tidy

* Update fuse-integration.yml

* Update Makefile

* fix deprecated

* fix deprecated

* remove deep-clean all rows

* broker memory count

* fix FieldIndex

---------

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
2025-09-09 01:01:03 -07:00

1405 lines
38 KiB
Go

// Code generated by protoc-gen-go. DO NOT EDIT.
// versions:
// protoc-gen-go v1.36.6
// protoc v5.29.3
// source: weed/pb/mq_schema.proto
package schema_pb
import (
protoreflect "google.golang.org/protobuf/reflect/protoreflect"
protoimpl "google.golang.org/protobuf/runtime/protoimpl"
reflect "reflect"
sync "sync"
unsafe "unsafe"
)
const (
// Verify that this generated code is sufficiently up-to-date.
_ = protoimpl.EnforceVersion(20 - protoimpl.MinVersion)
// Verify that runtime/protoimpl is sufficiently up-to-date.
_ = protoimpl.EnforceVersion(protoimpl.MaxVersion - 20)
)
type OffsetType int32
const (
OffsetType_RESUME_OR_EARLIEST OffsetType = 0
OffsetType_RESET_TO_EARLIEST OffsetType = 5
OffsetType_EXACT_TS_NS OffsetType = 10
OffsetType_RESET_TO_LATEST OffsetType = 15
OffsetType_RESUME_OR_LATEST OffsetType = 20
)
// Enum value maps for OffsetType.
var (
OffsetType_name = map[int32]string{
0: "RESUME_OR_EARLIEST",
5: "RESET_TO_EARLIEST",
10: "EXACT_TS_NS",
15: "RESET_TO_LATEST",
20: "RESUME_OR_LATEST",
}
OffsetType_value = map[string]int32{
"RESUME_OR_EARLIEST": 0,
"RESET_TO_EARLIEST": 5,
"EXACT_TS_NS": 10,
"RESET_TO_LATEST": 15,
"RESUME_OR_LATEST": 20,
}
)
func (x OffsetType) Enum() *OffsetType {
p := new(OffsetType)
*p = x
return p
}
func (x OffsetType) String() string {
return protoimpl.X.EnumStringOf(x.Descriptor(), protoreflect.EnumNumber(x))
}
func (OffsetType) Descriptor() protoreflect.EnumDescriptor {
return file_weed_pb_mq_schema_proto_enumTypes[0].Descriptor()
}
func (OffsetType) Type() protoreflect.EnumType {
return &file_weed_pb_mq_schema_proto_enumTypes[0]
}
func (x OffsetType) Number() protoreflect.EnumNumber {
return protoreflect.EnumNumber(x)
}
// Deprecated: Use OffsetType.Descriptor instead.
func (OffsetType) EnumDescriptor() ([]byte, []int) {
return file_weed_pb_mq_schema_proto_rawDescGZIP(), []int{0}
}
type ScalarType int32
const (
ScalarType_BOOL ScalarType = 0
ScalarType_INT32 ScalarType = 1
ScalarType_INT64 ScalarType = 3
ScalarType_FLOAT ScalarType = 4
ScalarType_DOUBLE ScalarType = 5
ScalarType_BYTES ScalarType = 6
ScalarType_STRING ScalarType = 7
// Parquet logical types for analytics
ScalarType_TIMESTAMP ScalarType = 8 // UTC timestamp (microseconds since epoch)
ScalarType_DATE ScalarType = 9 // Date (days since epoch)
ScalarType_DECIMAL ScalarType = 10 // Arbitrary precision decimal
ScalarType_TIME ScalarType = 11 // Time of day (microseconds)
)
// Enum value maps for ScalarType.
var (
ScalarType_name = map[int32]string{
0: "BOOL",
1: "INT32",
3: "INT64",
4: "FLOAT",
5: "DOUBLE",
6: "BYTES",
7: "STRING",
8: "TIMESTAMP",
9: "DATE",
10: "DECIMAL",
11: "TIME",
}
ScalarType_value = map[string]int32{
"BOOL": 0,
"INT32": 1,
"INT64": 3,
"FLOAT": 4,
"DOUBLE": 5,
"BYTES": 6,
"STRING": 7,
"TIMESTAMP": 8,
"DATE": 9,
"DECIMAL": 10,
"TIME": 11,
}
)
func (x ScalarType) Enum() *ScalarType {
p := new(ScalarType)
*p = x
return p
}
func (x ScalarType) String() string {
return protoimpl.X.EnumStringOf(x.Descriptor(), protoreflect.EnumNumber(x))
}
func (ScalarType) Descriptor() protoreflect.EnumDescriptor {
return file_weed_pb_mq_schema_proto_enumTypes[1].Descriptor()
}
func (ScalarType) Type() protoreflect.EnumType {
return &file_weed_pb_mq_schema_proto_enumTypes[1]
}
func (x ScalarType) Number() protoreflect.EnumNumber {
return protoreflect.EnumNumber(x)
}
// Deprecated: Use ScalarType.Descriptor instead.
func (ScalarType) EnumDescriptor() ([]byte, []int) {
return file_weed_pb_mq_schema_proto_rawDescGZIP(), []int{1}
}
type Topic struct {
state protoimpl.MessageState `protogen:"open.v1"`
Namespace string `protobuf:"bytes,1,opt,name=namespace,proto3" json:"namespace,omitempty"`
Name string `protobuf:"bytes,2,opt,name=name,proto3" json:"name,omitempty"`
unknownFields protoimpl.UnknownFields
sizeCache protoimpl.SizeCache
}
func (x *Topic) Reset() {
*x = Topic{}
mi := &file_weed_pb_mq_schema_proto_msgTypes[0]
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
ms.StoreMessageInfo(mi)
}
func (x *Topic) String() string {
return protoimpl.X.MessageStringOf(x)
}
func (*Topic) ProtoMessage() {}
func (x *Topic) ProtoReflect() protoreflect.Message {
mi := &file_weed_pb_mq_schema_proto_msgTypes[0]
if x != nil {
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
if ms.LoadMessageInfo() == nil {
ms.StoreMessageInfo(mi)
}
return ms
}
return mi.MessageOf(x)
}
// Deprecated: Use Topic.ProtoReflect.Descriptor instead.
func (*Topic) Descriptor() ([]byte, []int) {
return file_weed_pb_mq_schema_proto_rawDescGZIP(), []int{0}
}
func (x *Topic) GetNamespace() string {
if x != nil {
return x.Namespace
}
return ""
}
func (x *Topic) GetName() string {
if x != nil {
return x.Name
}
return ""
}
type Partition struct {
state protoimpl.MessageState `protogen:"open.v1"`
RingSize int32 `protobuf:"varint,1,opt,name=ring_size,json=ringSize,proto3" json:"ring_size,omitempty"`
RangeStart int32 `protobuf:"varint,2,opt,name=range_start,json=rangeStart,proto3" json:"range_start,omitempty"`
RangeStop int32 `protobuf:"varint,3,opt,name=range_stop,json=rangeStop,proto3" json:"range_stop,omitempty"`
UnixTimeNs int64 `protobuf:"varint,4,opt,name=unix_time_ns,json=unixTimeNs,proto3" json:"unix_time_ns,omitempty"`
unknownFields protoimpl.UnknownFields
sizeCache protoimpl.SizeCache
}
func (x *Partition) Reset() {
*x = Partition{}
mi := &file_weed_pb_mq_schema_proto_msgTypes[1]
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
ms.StoreMessageInfo(mi)
}
func (x *Partition) String() string {
return protoimpl.X.MessageStringOf(x)
}
func (*Partition) ProtoMessage() {}
func (x *Partition) ProtoReflect() protoreflect.Message {
mi := &file_weed_pb_mq_schema_proto_msgTypes[1]
if x != nil {
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
if ms.LoadMessageInfo() == nil {
ms.StoreMessageInfo(mi)
}
return ms
}
return mi.MessageOf(x)
}
// Deprecated: Use Partition.ProtoReflect.Descriptor instead.
func (*Partition) Descriptor() ([]byte, []int) {
return file_weed_pb_mq_schema_proto_rawDescGZIP(), []int{1}
}
func (x *Partition) GetRingSize() int32 {
if x != nil {
return x.RingSize
}
return 0
}
func (x *Partition) GetRangeStart() int32 {
if x != nil {
return x.RangeStart
}
return 0
}
func (x *Partition) GetRangeStop() int32 {
if x != nil {
return x.RangeStop
}
return 0
}
func (x *Partition) GetUnixTimeNs() int64 {
if x != nil {
return x.UnixTimeNs
}
return 0
}
type Offset struct {
state protoimpl.MessageState `protogen:"open.v1"`
Topic *Topic `protobuf:"bytes,1,opt,name=topic,proto3" json:"topic,omitempty"`
PartitionOffsets []*PartitionOffset `protobuf:"bytes,2,rep,name=partition_offsets,json=partitionOffsets,proto3" json:"partition_offsets,omitempty"`
unknownFields protoimpl.UnknownFields
sizeCache protoimpl.SizeCache
}
func (x *Offset) Reset() {
*x = Offset{}
mi := &file_weed_pb_mq_schema_proto_msgTypes[2]
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
ms.StoreMessageInfo(mi)
}
func (x *Offset) String() string {
return protoimpl.X.MessageStringOf(x)
}
func (*Offset) ProtoMessage() {}
func (x *Offset) ProtoReflect() protoreflect.Message {
mi := &file_weed_pb_mq_schema_proto_msgTypes[2]
if x != nil {
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
if ms.LoadMessageInfo() == nil {
ms.StoreMessageInfo(mi)
}
return ms
}
return mi.MessageOf(x)
}
// Deprecated: Use Offset.ProtoReflect.Descriptor instead.
func (*Offset) Descriptor() ([]byte, []int) {
return file_weed_pb_mq_schema_proto_rawDescGZIP(), []int{2}
}
func (x *Offset) GetTopic() *Topic {
if x != nil {
return x.Topic
}
return nil
}
func (x *Offset) GetPartitionOffsets() []*PartitionOffset {
if x != nil {
return x.PartitionOffsets
}
return nil
}
type PartitionOffset struct {
state protoimpl.MessageState `protogen:"open.v1"`
Partition *Partition `protobuf:"bytes,1,opt,name=partition,proto3" json:"partition,omitempty"`
StartTsNs int64 `protobuf:"varint,2,opt,name=start_ts_ns,json=startTsNs,proto3" json:"start_ts_ns,omitempty"`
unknownFields protoimpl.UnknownFields
sizeCache protoimpl.SizeCache
}
func (x *PartitionOffset) Reset() {
*x = PartitionOffset{}
mi := &file_weed_pb_mq_schema_proto_msgTypes[3]
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
ms.StoreMessageInfo(mi)
}
func (x *PartitionOffset) String() string {
return protoimpl.X.MessageStringOf(x)
}
func (*PartitionOffset) ProtoMessage() {}
func (x *PartitionOffset) ProtoReflect() protoreflect.Message {
mi := &file_weed_pb_mq_schema_proto_msgTypes[3]
if x != nil {
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
if ms.LoadMessageInfo() == nil {
ms.StoreMessageInfo(mi)
}
return ms
}
return mi.MessageOf(x)
}
// Deprecated: Use PartitionOffset.ProtoReflect.Descriptor instead.
func (*PartitionOffset) Descriptor() ([]byte, []int) {
return file_weed_pb_mq_schema_proto_rawDescGZIP(), []int{3}
}
func (x *PartitionOffset) GetPartition() *Partition {
if x != nil {
return x.Partition
}
return nil
}
func (x *PartitionOffset) GetStartTsNs() int64 {
if x != nil {
return x.StartTsNs
}
return 0
}
type RecordType struct {
state protoimpl.MessageState `protogen:"open.v1"`
Fields []*Field `protobuf:"bytes,1,rep,name=fields,proto3" json:"fields,omitempty"`
unknownFields protoimpl.UnknownFields
sizeCache protoimpl.SizeCache
}
func (x *RecordType) Reset() {
*x = RecordType{}
mi := &file_weed_pb_mq_schema_proto_msgTypes[4]
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
ms.StoreMessageInfo(mi)
}
func (x *RecordType) String() string {
return protoimpl.X.MessageStringOf(x)
}
func (*RecordType) ProtoMessage() {}
func (x *RecordType) ProtoReflect() protoreflect.Message {
mi := &file_weed_pb_mq_schema_proto_msgTypes[4]
if x != nil {
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
if ms.LoadMessageInfo() == nil {
ms.StoreMessageInfo(mi)
}
return ms
}
return mi.MessageOf(x)
}
// Deprecated: Use RecordType.ProtoReflect.Descriptor instead.
func (*RecordType) Descriptor() ([]byte, []int) {
return file_weed_pb_mq_schema_proto_rawDescGZIP(), []int{4}
}
func (x *RecordType) GetFields() []*Field {
if x != nil {
return x.Fields
}
return nil
}
type Field struct {
state protoimpl.MessageState `protogen:"open.v1"`
Name string `protobuf:"bytes,1,opt,name=name,proto3" json:"name,omitempty"`
FieldIndex int32 `protobuf:"varint,2,opt,name=field_index,json=fieldIndex,proto3" json:"field_index,omitempty"`
Type *Type `protobuf:"bytes,3,opt,name=type,proto3" json:"type,omitempty"`
IsRepeated bool `protobuf:"varint,4,opt,name=is_repeated,json=isRepeated,proto3" json:"is_repeated,omitempty"`
IsRequired bool `protobuf:"varint,5,opt,name=is_required,json=isRequired,proto3" json:"is_required,omitempty"`
unknownFields protoimpl.UnknownFields
sizeCache protoimpl.SizeCache
}
func (x *Field) Reset() {
*x = Field{}
mi := &file_weed_pb_mq_schema_proto_msgTypes[5]
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
ms.StoreMessageInfo(mi)
}
func (x *Field) String() string {
return protoimpl.X.MessageStringOf(x)
}
func (*Field) ProtoMessage() {}
func (x *Field) ProtoReflect() protoreflect.Message {
mi := &file_weed_pb_mq_schema_proto_msgTypes[5]
if x != nil {
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
if ms.LoadMessageInfo() == nil {
ms.StoreMessageInfo(mi)
}
return ms
}
return mi.MessageOf(x)
}
// Deprecated: Use Field.ProtoReflect.Descriptor instead.
func (*Field) Descriptor() ([]byte, []int) {
return file_weed_pb_mq_schema_proto_rawDescGZIP(), []int{5}
}
func (x *Field) GetName() string {
if x != nil {
return x.Name
}
return ""
}
func (x *Field) GetFieldIndex() int32 {
if x != nil {
return x.FieldIndex
}
return 0
}
func (x *Field) GetType() *Type {
if x != nil {
return x.Type
}
return nil
}
func (x *Field) GetIsRepeated() bool {
if x != nil {
return x.IsRepeated
}
return false
}
func (x *Field) GetIsRequired() bool {
if x != nil {
return x.IsRequired
}
return false
}
type Type struct {
state protoimpl.MessageState `protogen:"open.v1"`
// Types that are valid to be assigned to Kind:
//
// *Type_ScalarType
// *Type_RecordType
// *Type_ListType
Kind isType_Kind `protobuf_oneof:"kind"`
unknownFields protoimpl.UnknownFields
sizeCache protoimpl.SizeCache
}
func (x *Type) Reset() {
*x = Type{}
mi := &file_weed_pb_mq_schema_proto_msgTypes[6]
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
ms.StoreMessageInfo(mi)
}
func (x *Type) String() string {
return protoimpl.X.MessageStringOf(x)
}
func (*Type) ProtoMessage() {}
func (x *Type) ProtoReflect() protoreflect.Message {
mi := &file_weed_pb_mq_schema_proto_msgTypes[6]
if x != nil {
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
if ms.LoadMessageInfo() == nil {
ms.StoreMessageInfo(mi)
}
return ms
}
return mi.MessageOf(x)
}
// Deprecated: Use Type.ProtoReflect.Descriptor instead.
func (*Type) Descriptor() ([]byte, []int) {
return file_weed_pb_mq_schema_proto_rawDescGZIP(), []int{6}
}
func (x *Type) GetKind() isType_Kind {
if x != nil {
return x.Kind
}
return nil
}
func (x *Type) GetScalarType() ScalarType {
if x != nil {
if x, ok := x.Kind.(*Type_ScalarType); ok {
return x.ScalarType
}
}
return ScalarType_BOOL
}
func (x *Type) GetRecordType() *RecordType {
if x != nil {
if x, ok := x.Kind.(*Type_RecordType); ok {
return x.RecordType
}
}
return nil
}
func (x *Type) GetListType() *ListType {
if x != nil {
if x, ok := x.Kind.(*Type_ListType); ok {
return x.ListType
}
}
return nil
}
type isType_Kind interface {
isType_Kind()
}
type Type_ScalarType struct {
ScalarType ScalarType `protobuf:"varint,1,opt,name=scalar_type,json=scalarType,proto3,enum=schema_pb.ScalarType,oneof"`
}
type Type_RecordType struct {
RecordType *RecordType `protobuf:"bytes,2,opt,name=record_type,json=recordType,proto3,oneof"`
}
type Type_ListType struct {
ListType *ListType `protobuf:"bytes,3,opt,name=list_type,json=listType,proto3,oneof"`
}
func (*Type_ScalarType) isType_Kind() {}
func (*Type_RecordType) isType_Kind() {}
func (*Type_ListType) isType_Kind() {}
type ListType struct {
state protoimpl.MessageState `protogen:"open.v1"`
ElementType *Type `protobuf:"bytes,1,opt,name=element_type,json=elementType,proto3" json:"element_type,omitempty"`
unknownFields protoimpl.UnknownFields
sizeCache protoimpl.SizeCache
}
func (x *ListType) Reset() {
*x = ListType{}
mi := &file_weed_pb_mq_schema_proto_msgTypes[7]
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
ms.StoreMessageInfo(mi)
}
func (x *ListType) String() string {
return protoimpl.X.MessageStringOf(x)
}
func (*ListType) ProtoMessage() {}
func (x *ListType) ProtoReflect() protoreflect.Message {
mi := &file_weed_pb_mq_schema_proto_msgTypes[7]
if x != nil {
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
if ms.LoadMessageInfo() == nil {
ms.StoreMessageInfo(mi)
}
return ms
}
return mi.MessageOf(x)
}
// Deprecated: Use ListType.ProtoReflect.Descriptor instead.
func (*ListType) Descriptor() ([]byte, []int) {
return file_weed_pb_mq_schema_proto_rawDescGZIP(), []int{7}
}
func (x *ListType) GetElementType() *Type {
if x != nil {
return x.ElementType
}
return nil
}
// /////////////////////////
// value definition
// /////////////////////////
type RecordValue struct {
state protoimpl.MessageState `protogen:"open.v1"`
Fields map[string]*Value `protobuf:"bytes,1,rep,name=fields,proto3" json:"fields,omitempty" protobuf_key:"bytes,1,opt,name=key" protobuf_val:"bytes,2,opt,name=value"`
unknownFields protoimpl.UnknownFields
sizeCache protoimpl.SizeCache
}
func (x *RecordValue) Reset() {
*x = RecordValue{}
mi := &file_weed_pb_mq_schema_proto_msgTypes[8]
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
ms.StoreMessageInfo(mi)
}
func (x *RecordValue) String() string {
return protoimpl.X.MessageStringOf(x)
}
func (*RecordValue) ProtoMessage() {}
func (x *RecordValue) ProtoReflect() protoreflect.Message {
mi := &file_weed_pb_mq_schema_proto_msgTypes[8]
if x != nil {
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
if ms.LoadMessageInfo() == nil {
ms.StoreMessageInfo(mi)
}
return ms
}
return mi.MessageOf(x)
}
// Deprecated: Use RecordValue.ProtoReflect.Descriptor instead.
func (*RecordValue) Descriptor() ([]byte, []int) {
return file_weed_pb_mq_schema_proto_rawDescGZIP(), []int{8}
}
func (x *RecordValue) GetFields() map[string]*Value {
if x != nil {
return x.Fields
}
return nil
}
type Value struct {
state protoimpl.MessageState `protogen:"open.v1"`
// Types that are valid to be assigned to Kind:
//
// *Value_BoolValue
// *Value_Int32Value
// *Value_Int64Value
// *Value_FloatValue
// *Value_DoubleValue
// *Value_BytesValue
// *Value_StringValue
// *Value_TimestampValue
// *Value_DateValue
// *Value_DecimalValue
// *Value_TimeValue
// *Value_ListValue
// *Value_RecordValue
Kind isValue_Kind `protobuf_oneof:"kind"`
unknownFields protoimpl.UnknownFields
sizeCache protoimpl.SizeCache
}
func (x *Value) Reset() {
*x = Value{}
mi := &file_weed_pb_mq_schema_proto_msgTypes[9]
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
ms.StoreMessageInfo(mi)
}
func (x *Value) String() string {
return protoimpl.X.MessageStringOf(x)
}
func (*Value) ProtoMessage() {}
func (x *Value) ProtoReflect() protoreflect.Message {
mi := &file_weed_pb_mq_schema_proto_msgTypes[9]
if x != nil {
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
if ms.LoadMessageInfo() == nil {
ms.StoreMessageInfo(mi)
}
return ms
}
return mi.MessageOf(x)
}
// Deprecated: Use Value.ProtoReflect.Descriptor instead.
func (*Value) Descriptor() ([]byte, []int) {
return file_weed_pb_mq_schema_proto_rawDescGZIP(), []int{9}
}
func (x *Value) GetKind() isValue_Kind {
if x != nil {
return x.Kind
}
return nil
}
func (x *Value) GetBoolValue() bool {
if x != nil {
if x, ok := x.Kind.(*Value_BoolValue); ok {
return x.BoolValue
}
}
return false
}
func (x *Value) GetInt32Value() int32 {
if x != nil {
if x, ok := x.Kind.(*Value_Int32Value); ok {
return x.Int32Value
}
}
return 0
}
func (x *Value) GetInt64Value() int64 {
if x != nil {
if x, ok := x.Kind.(*Value_Int64Value); ok {
return x.Int64Value
}
}
return 0
}
func (x *Value) GetFloatValue() float32 {
if x != nil {
if x, ok := x.Kind.(*Value_FloatValue); ok {
return x.FloatValue
}
}
return 0
}
func (x *Value) GetDoubleValue() float64 {
if x != nil {
if x, ok := x.Kind.(*Value_DoubleValue); ok {
return x.DoubleValue
}
}
return 0
}
func (x *Value) GetBytesValue() []byte {
if x != nil {
if x, ok := x.Kind.(*Value_BytesValue); ok {
return x.BytesValue
}
}
return nil
}
func (x *Value) GetStringValue() string {
if x != nil {
if x, ok := x.Kind.(*Value_StringValue); ok {
return x.StringValue
}
}
return ""
}
func (x *Value) GetTimestampValue() *TimestampValue {
if x != nil {
if x, ok := x.Kind.(*Value_TimestampValue); ok {
return x.TimestampValue
}
}
return nil
}
func (x *Value) GetDateValue() *DateValue {
if x != nil {
if x, ok := x.Kind.(*Value_DateValue); ok {
return x.DateValue
}
}
return nil
}
func (x *Value) GetDecimalValue() *DecimalValue {
if x != nil {
if x, ok := x.Kind.(*Value_DecimalValue); ok {
return x.DecimalValue
}
}
return nil
}
func (x *Value) GetTimeValue() *TimeValue {
if x != nil {
if x, ok := x.Kind.(*Value_TimeValue); ok {
return x.TimeValue
}
}
return nil
}
func (x *Value) GetListValue() *ListValue {
if x != nil {
if x, ok := x.Kind.(*Value_ListValue); ok {
return x.ListValue
}
}
return nil
}
func (x *Value) GetRecordValue() *RecordValue {
if x != nil {
if x, ok := x.Kind.(*Value_RecordValue); ok {
return x.RecordValue
}
}
return nil
}
type isValue_Kind interface {
isValue_Kind()
}
type Value_BoolValue struct {
BoolValue bool `protobuf:"varint,1,opt,name=bool_value,json=boolValue,proto3,oneof"`
}
type Value_Int32Value struct {
Int32Value int32 `protobuf:"varint,2,opt,name=int32_value,json=int32Value,proto3,oneof"`
}
type Value_Int64Value struct {
Int64Value int64 `protobuf:"varint,3,opt,name=int64_value,json=int64Value,proto3,oneof"`
}
type Value_FloatValue struct {
FloatValue float32 `protobuf:"fixed32,4,opt,name=float_value,json=floatValue,proto3,oneof"`
}
type Value_DoubleValue struct {
DoubleValue float64 `protobuf:"fixed64,5,opt,name=double_value,json=doubleValue,proto3,oneof"`
}
type Value_BytesValue struct {
BytesValue []byte `protobuf:"bytes,6,opt,name=bytes_value,json=bytesValue,proto3,oneof"`
}
type Value_StringValue struct {
StringValue string `protobuf:"bytes,7,opt,name=string_value,json=stringValue,proto3,oneof"`
}
type Value_TimestampValue struct {
// Parquet logical type values
TimestampValue *TimestampValue `protobuf:"bytes,8,opt,name=timestamp_value,json=timestampValue,proto3,oneof"`
}
type Value_DateValue struct {
DateValue *DateValue `protobuf:"bytes,9,opt,name=date_value,json=dateValue,proto3,oneof"`
}
type Value_DecimalValue struct {
DecimalValue *DecimalValue `protobuf:"bytes,10,opt,name=decimal_value,json=decimalValue,proto3,oneof"`
}
type Value_TimeValue struct {
TimeValue *TimeValue `protobuf:"bytes,11,opt,name=time_value,json=timeValue,proto3,oneof"`
}
type Value_ListValue struct {
// Complex types
ListValue *ListValue `protobuf:"bytes,14,opt,name=list_value,json=listValue,proto3,oneof"`
}
type Value_RecordValue struct {
RecordValue *RecordValue `protobuf:"bytes,15,opt,name=record_value,json=recordValue,proto3,oneof"`
}
func (*Value_BoolValue) isValue_Kind() {}
func (*Value_Int32Value) isValue_Kind() {}
func (*Value_Int64Value) isValue_Kind() {}
func (*Value_FloatValue) isValue_Kind() {}
func (*Value_DoubleValue) isValue_Kind() {}
func (*Value_BytesValue) isValue_Kind() {}
func (*Value_StringValue) isValue_Kind() {}
func (*Value_TimestampValue) isValue_Kind() {}
func (*Value_DateValue) isValue_Kind() {}
func (*Value_DecimalValue) isValue_Kind() {}
func (*Value_TimeValue) isValue_Kind() {}
func (*Value_ListValue) isValue_Kind() {}
func (*Value_RecordValue) isValue_Kind() {}
// Parquet logical type value messages
type TimestampValue struct {
state protoimpl.MessageState `protogen:"open.v1"`
TimestampMicros int64 `protobuf:"varint,1,opt,name=timestamp_micros,json=timestampMicros,proto3" json:"timestamp_micros,omitempty"` // Microseconds since Unix epoch (UTC)
IsUtc bool `protobuf:"varint,2,opt,name=is_utc,json=isUtc,proto3" json:"is_utc,omitempty"` // True if UTC, false if local time
unknownFields protoimpl.UnknownFields
sizeCache protoimpl.SizeCache
}
func (x *TimestampValue) Reset() {
*x = TimestampValue{}
mi := &file_weed_pb_mq_schema_proto_msgTypes[10]
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
ms.StoreMessageInfo(mi)
}
func (x *TimestampValue) String() string {
return protoimpl.X.MessageStringOf(x)
}
func (*TimestampValue) ProtoMessage() {}
func (x *TimestampValue) ProtoReflect() protoreflect.Message {
mi := &file_weed_pb_mq_schema_proto_msgTypes[10]
if x != nil {
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
if ms.LoadMessageInfo() == nil {
ms.StoreMessageInfo(mi)
}
return ms
}
return mi.MessageOf(x)
}
// Deprecated: Use TimestampValue.ProtoReflect.Descriptor instead.
func (*TimestampValue) Descriptor() ([]byte, []int) {
return file_weed_pb_mq_schema_proto_rawDescGZIP(), []int{10}
}
func (x *TimestampValue) GetTimestampMicros() int64 {
if x != nil {
return x.TimestampMicros
}
return 0
}
func (x *TimestampValue) GetIsUtc() bool {
if x != nil {
return x.IsUtc
}
return false
}
type DateValue struct {
state protoimpl.MessageState `protogen:"open.v1"`
DaysSinceEpoch int32 `protobuf:"varint,1,opt,name=days_since_epoch,json=daysSinceEpoch,proto3" json:"days_since_epoch,omitempty"` // Days since Unix epoch (1970-01-01)
unknownFields protoimpl.UnknownFields
sizeCache protoimpl.SizeCache
}
func (x *DateValue) Reset() {
*x = DateValue{}
mi := &file_weed_pb_mq_schema_proto_msgTypes[11]
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
ms.StoreMessageInfo(mi)
}
func (x *DateValue) String() string {
return protoimpl.X.MessageStringOf(x)
}
func (*DateValue) ProtoMessage() {}
func (x *DateValue) ProtoReflect() protoreflect.Message {
mi := &file_weed_pb_mq_schema_proto_msgTypes[11]
if x != nil {
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
if ms.LoadMessageInfo() == nil {
ms.StoreMessageInfo(mi)
}
return ms
}
return mi.MessageOf(x)
}
// Deprecated: Use DateValue.ProtoReflect.Descriptor instead.
func (*DateValue) Descriptor() ([]byte, []int) {
return file_weed_pb_mq_schema_proto_rawDescGZIP(), []int{11}
}
func (x *DateValue) GetDaysSinceEpoch() int32 {
if x != nil {
return x.DaysSinceEpoch
}
return 0
}
type DecimalValue struct {
state protoimpl.MessageState `protogen:"open.v1"`
Value []byte `protobuf:"bytes,1,opt,name=value,proto3" json:"value,omitempty"` // Arbitrary precision decimal as bytes
Precision int32 `protobuf:"varint,2,opt,name=precision,proto3" json:"precision,omitempty"` // Total number of digits
Scale int32 `protobuf:"varint,3,opt,name=scale,proto3" json:"scale,omitempty"` // Number of digits after decimal point
unknownFields protoimpl.UnknownFields
sizeCache protoimpl.SizeCache
}
func (x *DecimalValue) Reset() {
*x = DecimalValue{}
mi := &file_weed_pb_mq_schema_proto_msgTypes[12]
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
ms.StoreMessageInfo(mi)
}
func (x *DecimalValue) String() string {
return protoimpl.X.MessageStringOf(x)
}
func (*DecimalValue) ProtoMessage() {}
func (x *DecimalValue) ProtoReflect() protoreflect.Message {
mi := &file_weed_pb_mq_schema_proto_msgTypes[12]
if x != nil {
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
if ms.LoadMessageInfo() == nil {
ms.StoreMessageInfo(mi)
}
return ms
}
return mi.MessageOf(x)
}
// Deprecated: Use DecimalValue.ProtoReflect.Descriptor instead.
func (*DecimalValue) Descriptor() ([]byte, []int) {
return file_weed_pb_mq_schema_proto_rawDescGZIP(), []int{12}
}
func (x *DecimalValue) GetValue() []byte {
if x != nil {
return x.Value
}
return nil
}
func (x *DecimalValue) GetPrecision() int32 {
if x != nil {
return x.Precision
}
return 0
}
func (x *DecimalValue) GetScale() int32 {
if x != nil {
return x.Scale
}
return 0
}
type TimeValue struct {
state protoimpl.MessageState `protogen:"open.v1"`
TimeMicros int64 `protobuf:"varint,1,opt,name=time_micros,json=timeMicros,proto3" json:"time_micros,omitempty"` // Microseconds since midnight
unknownFields protoimpl.UnknownFields
sizeCache protoimpl.SizeCache
}
func (x *TimeValue) Reset() {
*x = TimeValue{}
mi := &file_weed_pb_mq_schema_proto_msgTypes[13]
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
ms.StoreMessageInfo(mi)
}
func (x *TimeValue) String() string {
return protoimpl.X.MessageStringOf(x)
}
func (*TimeValue) ProtoMessage() {}
func (x *TimeValue) ProtoReflect() protoreflect.Message {
mi := &file_weed_pb_mq_schema_proto_msgTypes[13]
if x != nil {
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
if ms.LoadMessageInfo() == nil {
ms.StoreMessageInfo(mi)
}
return ms
}
return mi.MessageOf(x)
}
// Deprecated: Use TimeValue.ProtoReflect.Descriptor instead.
func (*TimeValue) Descriptor() ([]byte, []int) {
return file_weed_pb_mq_schema_proto_rawDescGZIP(), []int{13}
}
func (x *TimeValue) GetTimeMicros() int64 {
if x != nil {
return x.TimeMicros
}
return 0
}
type ListValue struct {
state protoimpl.MessageState `protogen:"open.v1"`
Values []*Value `protobuf:"bytes,1,rep,name=values,proto3" json:"values,omitempty"`
unknownFields protoimpl.UnknownFields
sizeCache protoimpl.SizeCache
}
func (x *ListValue) Reset() {
*x = ListValue{}
mi := &file_weed_pb_mq_schema_proto_msgTypes[14]
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
ms.StoreMessageInfo(mi)
}
func (x *ListValue) String() string {
return protoimpl.X.MessageStringOf(x)
}
func (*ListValue) ProtoMessage() {}
func (x *ListValue) ProtoReflect() protoreflect.Message {
mi := &file_weed_pb_mq_schema_proto_msgTypes[14]
if x != nil {
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
if ms.LoadMessageInfo() == nil {
ms.StoreMessageInfo(mi)
}
return ms
}
return mi.MessageOf(x)
}
// Deprecated: Use ListValue.ProtoReflect.Descriptor instead.
func (*ListValue) Descriptor() ([]byte, []int) {
return file_weed_pb_mq_schema_proto_rawDescGZIP(), []int{14}
}
func (x *ListValue) GetValues() []*Value {
if x != nil {
return x.Values
}
return nil
}
var File_weed_pb_mq_schema_proto protoreflect.FileDescriptor
const file_weed_pb_mq_schema_proto_rawDesc = "" +
"\n" +
"\x17weed/pb/mq_schema.proto\x12\tschema_pb\"9\n" +
"\x05Topic\x12\x1c\n" +
"\tnamespace\x18\x01 \x01(\tR\tnamespace\x12\x12\n" +
"\x04name\x18\x02 \x01(\tR\x04name\"\x8a\x01\n" +
"\tPartition\x12\x1b\n" +
"\tring_size\x18\x01 \x01(\x05R\bringSize\x12\x1f\n" +
"\vrange_start\x18\x02 \x01(\x05R\n" +
"rangeStart\x12\x1d\n" +
"\n" +
"range_stop\x18\x03 \x01(\x05R\trangeStop\x12 \n" +
"\funix_time_ns\x18\x04 \x01(\x03R\n" +
"unixTimeNs\"y\n" +
"\x06Offset\x12&\n" +
"\x05topic\x18\x01 \x01(\v2\x10.schema_pb.TopicR\x05topic\x12G\n" +
"\x11partition_offsets\x18\x02 \x03(\v2\x1a.schema_pb.PartitionOffsetR\x10partitionOffsets\"e\n" +
"\x0fPartitionOffset\x122\n" +
"\tpartition\x18\x01 \x01(\v2\x14.schema_pb.PartitionR\tpartition\x12\x1e\n" +
"\vstart_ts_ns\x18\x02 \x01(\x03R\tstartTsNs\"6\n" +
"\n" +
"RecordType\x12(\n" +
"\x06fields\x18\x01 \x03(\v2\x10.schema_pb.FieldR\x06fields\"\xa3\x01\n" +
"\x05Field\x12\x12\n" +
"\x04name\x18\x01 \x01(\tR\x04name\x12\x1f\n" +
"\vfield_index\x18\x02 \x01(\x05R\n" +
"fieldIndex\x12#\n" +
"\x04type\x18\x03 \x01(\v2\x0f.schema_pb.TypeR\x04type\x12\x1f\n" +
"\vis_repeated\x18\x04 \x01(\bR\n" +
"isRepeated\x12\x1f\n" +
"\vis_required\x18\x05 \x01(\bR\n" +
"isRequired\"\xb6\x01\n" +
"\x04Type\x128\n" +
"\vscalar_type\x18\x01 \x01(\x0e2\x15.schema_pb.ScalarTypeH\x00R\n" +
"scalarType\x128\n" +
"\vrecord_type\x18\x02 \x01(\v2\x15.schema_pb.RecordTypeH\x00R\n" +
"recordType\x122\n" +
"\tlist_type\x18\x03 \x01(\v2\x13.schema_pb.ListTypeH\x00R\blistTypeB\x06\n" +
"\x04kind\">\n" +
"\bListType\x122\n" +
"\felement_type\x18\x01 \x01(\v2\x0f.schema_pb.TypeR\velementType\"\x96\x01\n" +
"\vRecordValue\x12:\n" +
"\x06fields\x18\x01 \x03(\v2\".schema_pb.RecordValue.FieldsEntryR\x06fields\x1aK\n" +
"\vFieldsEntry\x12\x10\n" +
"\x03key\x18\x01 \x01(\tR\x03key\x12&\n" +
"\x05value\x18\x02 \x01(\v2\x10.schema_pb.ValueR\x05value:\x028\x01\"\xee\x04\n" +
"\x05Value\x12\x1f\n" +
"\n" +
"bool_value\x18\x01 \x01(\bH\x00R\tboolValue\x12!\n" +
"\vint32_value\x18\x02 \x01(\x05H\x00R\n" +
"int32Value\x12!\n" +
"\vint64_value\x18\x03 \x01(\x03H\x00R\n" +
"int64Value\x12!\n" +
"\vfloat_value\x18\x04 \x01(\x02H\x00R\n" +
"floatValue\x12#\n" +
"\fdouble_value\x18\x05 \x01(\x01H\x00R\vdoubleValue\x12!\n" +
"\vbytes_value\x18\x06 \x01(\fH\x00R\n" +
"bytesValue\x12#\n" +
"\fstring_value\x18\a \x01(\tH\x00R\vstringValue\x12D\n" +
"\x0ftimestamp_value\x18\b \x01(\v2\x19.schema_pb.TimestampValueH\x00R\x0etimestampValue\x125\n" +
"\n" +
"date_value\x18\t \x01(\v2\x14.schema_pb.DateValueH\x00R\tdateValue\x12>\n" +
"\rdecimal_value\x18\n" +
" \x01(\v2\x17.schema_pb.DecimalValueH\x00R\fdecimalValue\x125\n" +
"\n" +
"time_value\x18\v \x01(\v2\x14.schema_pb.TimeValueH\x00R\ttimeValue\x125\n" +
"\n" +
"list_value\x18\x0e \x01(\v2\x14.schema_pb.ListValueH\x00R\tlistValue\x12;\n" +
"\frecord_value\x18\x0f \x01(\v2\x16.schema_pb.RecordValueH\x00R\vrecordValueB\x06\n" +
"\x04kind\"R\n" +
"\x0eTimestampValue\x12)\n" +
"\x10timestamp_micros\x18\x01 \x01(\x03R\x0ftimestampMicros\x12\x15\n" +
"\x06is_utc\x18\x02 \x01(\bR\x05isUtc\"5\n" +
"\tDateValue\x12(\n" +
"\x10days_since_epoch\x18\x01 \x01(\x05R\x0edaysSinceEpoch\"X\n" +
"\fDecimalValue\x12\x14\n" +
"\x05value\x18\x01 \x01(\fR\x05value\x12\x1c\n" +
"\tprecision\x18\x02 \x01(\x05R\tprecision\x12\x14\n" +
"\x05scale\x18\x03 \x01(\x05R\x05scale\",\n" +
"\tTimeValue\x12\x1f\n" +
"\vtime_micros\x18\x01 \x01(\x03R\n" +
"timeMicros\"5\n" +
"\tListValue\x12(\n" +
"\x06values\x18\x01 \x03(\v2\x10.schema_pb.ValueR\x06values*w\n" +
"\n" +
"OffsetType\x12\x16\n" +
"\x12RESUME_OR_EARLIEST\x10\x00\x12\x15\n" +
"\x11RESET_TO_EARLIEST\x10\x05\x12\x0f\n" +
"\vEXACT_TS_NS\x10\n" +
"\x12\x13\n" +
"\x0fRESET_TO_LATEST\x10\x0f\x12\x14\n" +
"\x10RESUME_OR_LATEST\x10\x14*\x8a\x01\n" +
"\n" +
"ScalarType\x12\b\n" +
"\x04BOOL\x10\x00\x12\t\n" +
"\x05INT32\x10\x01\x12\t\n" +
"\x05INT64\x10\x03\x12\t\n" +
"\x05FLOAT\x10\x04\x12\n" +
"\n" +
"\x06DOUBLE\x10\x05\x12\t\n" +
"\x05BYTES\x10\x06\x12\n" +
"\n" +
"\x06STRING\x10\a\x12\r\n" +
"\tTIMESTAMP\x10\b\x12\b\n" +
"\x04DATE\x10\t\x12\v\n" +
"\aDECIMAL\x10\n" +
"\x12\b\n" +
"\x04TIME\x10\vB2Z0github.com/seaweedfs/seaweedfs/weed/pb/schema_pbb\x06proto3"
var (
file_weed_pb_mq_schema_proto_rawDescOnce sync.Once
file_weed_pb_mq_schema_proto_rawDescData []byte
)
func file_weed_pb_mq_schema_proto_rawDescGZIP() []byte {
file_weed_pb_mq_schema_proto_rawDescOnce.Do(func() {
file_weed_pb_mq_schema_proto_rawDescData = protoimpl.X.CompressGZIP(unsafe.Slice(unsafe.StringData(file_weed_pb_mq_schema_proto_rawDesc), len(file_weed_pb_mq_schema_proto_rawDesc)))
})
return file_weed_pb_mq_schema_proto_rawDescData
}
var file_weed_pb_mq_schema_proto_enumTypes = make([]protoimpl.EnumInfo, 2)
var file_weed_pb_mq_schema_proto_msgTypes = make([]protoimpl.MessageInfo, 16)
var file_weed_pb_mq_schema_proto_goTypes = []any{
(OffsetType)(0), // 0: schema_pb.OffsetType
(ScalarType)(0), // 1: schema_pb.ScalarType
(*Topic)(nil), // 2: schema_pb.Topic
(*Partition)(nil), // 3: schema_pb.Partition
(*Offset)(nil), // 4: schema_pb.Offset
(*PartitionOffset)(nil), // 5: schema_pb.PartitionOffset
(*RecordType)(nil), // 6: schema_pb.RecordType
(*Field)(nil), // 7: schema_pb.Field
(*Type)(nil), // 8: schema_pb.Type
(*ListType)(nil), // 9: schema_pb.ListType
(*RecordValue)(nil), // 10: schema_pb.RecordValue
(*Value)(nil), // 11: schema_pb.Value
(*TimestampValue)(nil), // 12: schema_pb.TimestampValue
(*DateValue)(nil), // 13: schema_pb.DateValue
(*DecimalValue)(nil), // 14: schema_pb.DecimalValue
(*TimeValue)(nil), // 15: schema_pb.TimeValue
(*ListValue)(nil), // 16: schema_pb.ListValue
nil, // 17: schema_pb.RecordValue.FieldsEntry
}
var file_weed_pb_mq_schema_proto_depIdxs = []int32{
2, // 0: schema_pb.Offset.topic:type_name -> schema_pb.Topic
5, // 1: schema_pb.Offset.partition_offsets:type_name -> schema_pb.PartitionOffset
3, // 2: schema_pb.PartitionOffset.partition:type_name -> schema_pb.Partition
7, // 3: schema_pb.RecordType.fields:type_name -> schema_pb.Field
8, // 4: schema_pb.Field.type:type_name -> schema_pb.Type
1, // 5: schema_pb.Type.scalar_type:type_name -> schema_pb.ScalarType
6, // 6: schema_pb.Type.record_type:type_name -> schema_pb.RecordType
9, // 7: schema_pb.Type.list_type:type_name -> schema_pb.ListType
8, // 8: schema_pb.ListType.element_type:type_name -> schema_pb.Type
17, // 9: schema_pb.RecordValue.fields:type_name -> schema_pb.RecordValue.FieldsEntry
12, // 10: schema_pb.Value.timestamp_value:type_name -> schema_pb.TimestampValue
13, // 11: schema_pb.Value.date_value:type_name -> schema_pb.DateValue
14, // 12: schema_pb.Value.decimal_value:type_name -> schema_pb.DecimalValue
15, // 13: schema_pb.Value.time_value:type_name -> schema_pb.TimeValue
16, // 14: schema_pb.Value.list_value:type_name -> schema_pb.ListValue
10, // 15: schema_pb.Value.record_value:type_name -> schema_pb.RecordValue
11, // 16: schema_pb.ListValue.values:type_name -> schema_pb.Value
11, // 17: schema_pb.RecordValue.FieldsEntry.value:type_name -> schema_pb.Value
18, // [18:18] is the sub-list for method output_type
18, // [18:18] is the sub-list for method input_type
18, // [18:18] is the sub-list for extension type_name
18, // [18:18] is the sub-list for extension extendee
0, // [0:18] is the sub-list for field type_name
}
func init() { file_weed_pb_mq_schema_proto_init() }
func file_weed_pb_mq_schema_proto_init() {
if File_weed_pb_mq_schema_proto != nil {
return
}
file_weed_pb_mq_schema_proto_msgTypes[6].OneofWrappers = []any{
(*Type_ScalarType)(nil),
(*Type_RecordType)(nil),
(*Type_ListType)(nil),
}
file_weed_pb_mq_schema_proto_msgTypes[9].OneofWrappers = []any{
(*Value_BoolValue)(nil),
(*Value_Int32Value)(nil),
(*Value_Int64Value)(nil),
(*Value_FloatValue)(nil),
(*Value_DoubleValue)(nil),
(*Value_BytesValue)(nil),
(*Value_StringValue)(nil),
(*Value_TimestampValue)(nil),
(*Value_DateValue)(nil),
(*Value_DecimalValue)(nil),
(*Value_TimeValue)(nil),
(*Value_ListValue)(nil),
(*Value_RecordValue)(nil),
}
type x struct{}
out := protoimpl.TypeBuilder{
File: protoimpl.DescBuilder{
GoPackagePath: reflect.TypeOf(x{}).PkgPath(),
RawDescriptor: unsafe.Slice(unsafe.StringData(file_weed_pb_mq_schema_proto_rawDesc), len(file_weed_pb_mq_schema_proto_rawDesc)),
NumEnums: 2,
NumMessages: 16,
NumExtensions: 0,
NumServices: 0,
},
GoTypes: file_weed_pb_mq_schema_proto_goTypes,
DependencyIndexes: file_weed_pb_mq_schema_proto_depIdxs,
EnumInfos: file_weed_pb_mq_schema_proto_enumTypes,
MessageInfos: file_weed_pb_mq_schema_proto_msgTypes,
}.Build()
File_weed_pb_mq_schema_proto = out.File
file_weed_pb_mq_schema_proto_goTypes = nil
file_weed_pb_mq_schema_proto_depIdxs = nil
}