Skip to content

Commit

Permalink
copy editing changes to summary (#16880)
Browse files Browse the repository at this point in the history 
Signed-off-by: deepthi <[email protected]>
  • Loading branch information
@deepthi
deepthi authored Oct 4, 2024
1 parent 945124a commit aa7ba63
Showing 1 changed file with 56 additions and 53 deletions.
109 changes: 56 additions & 53 deletions changelog/21.0/21.0.0/summary.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -4,9 +4,9 @@

- **[Major Changes](#major-changes)**
- **[Deprecations and Deletions](#deprecations-and-deletions)**
- [Deprecated VTTablet Flags](#vttablet-flags)
- [Deletion of deprecated metrics](#metric-deletion)
- [VTTablet Flags](#vttablet-flags)
- [Metrics](#deprecations-metrics)
- [Deprecated Metrics](#deprecations-metrics)
- **[Traffic Mirroring](#traffic-mirroring)**
- **[New VTGate Shutdown Behavior](#new-vtgate-shutdown-behavior)**
- **[Tablet Throttler: Multi-Metric support](#tablet-throttler)**
Expand All @@ -18,15 +18,20 @@
- **[Dynamic VReplication Configuration](#dynamic-vreplication-configuration)**
- **[Reference Table Materialization](#reference-table-materialization)**
- **[New VEXPLAIN Modes: TRACE and KEYS](#new-vexplain-modes)**
- **[Errant GTID Detection on Vttablets](#errant-gtid-vttablet)**
- **[Errant GTID Detection on VTTablets](#errant-gtid-vttablet)**

## <a id="major-changes"/>Major Changes

### <a id="deprecations-and-deletions"/>Deprecations and Deletions

#### <a id="vttablet-flags"/>Deprecated VTTablet Flags

- `queryserver-enable-settings-pool` flag, added in `v15`, has been on by default since `v17`.
It is now deprecated and will be removed in a future release.

#### <a id="metric-deletion"/>Deletion of deprecated metrics

The following metrics that were deprecated in the previous release, have now been deleted.
The following VTOrc metrics were deprecated in `v20`. They have now been deleted.

| Metric Name |
|:--------------------------------------------:|
Expand All @@ -43,14 +48,9 @@ The following metrics that were deprecated in the previous release, have now bee
| `planned_reparent_counts` |
| `reparent_shard_operation_timings` |

#### <a id="vttablet-flags"/>VTTablet Flags
#### <a id="deprecations-metrics"/>Deprecated Metrics

- `queryserver-enable-settings-pool` flag, added in v15, has been on by default since v17.
It is now deprecated and will be removed in a future release.

#### <a id="deprecations-metrics"/>Metrics

The following metrics are now deprecated, if provided please use their replacement.
The following metrics are now deprecated and will be deleted in a future release, please use their replacements.

| Component | Metric Name | Replaced By |
|------------|:---------------------:|:-------------------------------:|
Expand All @@ -76,47 +76,44 @@ Mirror rules can be inspected with `GetMirrorRules`.

### <a id="new-vtgate-shutdown-behavior"/>New VTGate Shutdown Behavior

We added a new option to affect the VTGate shutdown process in v21 by using a connection drain timeout rather than the
older activity drain timeout.
The goal of this new behavior, connection draining option, is to disallow new connections when VTGate is shutting down,
but continue allowing existing connections to finish their work until they manually disconnect or until
the `--onterm_timeout` timeout is reached,
without getting a `Server shutdown in progress` error.
We added a new option to VTGate to disallow new connections while VTGate is shutting down,
while allowing existing connections to finish their work until they manually disconnect or until
the `--onterm_timeout` is reached, without getting a `Server shutdown in progress` error.

This new behavior can be enabled by specifying the new `--mysql-server-drain-onterm` flag to VTGate.

See more information about this change by [reading its RFC](https://github.com/vitessio/vitess/issues/15971).
You can find more information about this option in the [RFC](https://github.com/vitessio/vitess/issues/15971).

### <a id="tablet-throttler"/>Tablet Throttler: Multi-Metric support

Up till `v20`, the tablet throttler would only monitor and use a single metric. That would be replication lag, by
default, or could be the result of a custom query. `v21` introduces a major redesign where the throttler monitors and
uses multiple metrics at the same time, including the above two.
Up until `v20`, the tablet throttler would only monitor and use a single metric. That would be replication lag, by
default, or could be the result of a custom query. In this release, we introduce a major redesign so that the throttler
monitors and uses multiple metrics at the same time, including the above two.

Backwards compatible with `v20`, the default behavior in `v21` is to monitor all metrics, but only use `lag` (if the
cutsom query is undefined) or the `cutsom` metric (if the custom query is defined). A `v20` `PRIMARY` is compatible with
The default behavior now is to monitor all metrics, but only use `lag` (if the custom query is undefined) or the `custom`
metric (if the custom query is defined). This is backwards-compatible with `v20`. A `v20` `PRIMARY` is compatible with
a `v21` `REPLICA`, and a `v21` `PRIMARY` is compatible with a `v20` `REPLICA`.

However, with `v21` it is possible to assign any combination of metrics (one or more) for a given app. The throttler
would then accept or reject the app's requests based on the health of _all_ assigned metrics. `v21` comes with a preset
list metrics, expected to be expanded:
However, it is now possible to assign any combination of one or more metrics for a given app. The throttler
would then accept or reject the app's requests based on the health of _all_ assigned metrics. We have provided a pre-defined
list of metrics:

- `lag`: replication lag based on heartbeat injection.
- `threads_running`: concurrent active threads on the MySQL server.
- `loadavg`: per core load average measured on the tablet instance/pod.
- `custom`: the result of a custom query executed on the MySQL server.

Each metric has a factory threshold which can be overridden by the `UpdateThrottlerConfig` command.
Each metric has a default threshold which can be overridden by the `UpdateThrottlerConfig` command.

The throttler also supports the catch-all `"all"` app name, and it is thus possible to assign metrics to _all_ apps.
The throttler also supports the catch-all `"all"` app name, and it is thus possible to assign metrics to **all** apps.
Explicit app to metric assignments will override the catch-all configuration.

Metrics are assigned a default _scope_, which could be `self` (isolated to the tablet) or `shard` (max, aka _worst_
Metrics are assigned a default _scope_, which could be `self` (isolated to the tablet) or `shard` (max, aka **worst**
value among shard tablets). It is further possible to require a different scope for each metric.

### <a id="allow-cross-cell"/>Allow Cross Cell Promotion in PRS

Up until now if the users wanted to promote a replica in a different cell than the current primary
Up until now if the users wanted to promote a replica in a different cell from the current primary
using `PlannedReparentShard`, they had to specify the new primary with the `--new-primary` flag.

We have now added a new flag `--allow-cross-cell-promotion` that lets `PlannedReparentShard` choose a primary in a
Expand All @@ -137,18 +134,18 @@ preferentially routing to tablets in the same cell as the VTGate.
The tablet balancer is enabled by a new flag `--enable-balancer` and configured by `--balancer-vtgate-cells`
and `--balancer-keyspaces`.

See [RFC for details](https://github.com/vitessio/vitess/issues/12241).
See the [RFC ](https://github.com/vitessio/vitess/issues/12241) for more details on the design and configuration of this feature.

### <a id="query-timeout"/>Query Timeout Override

VTGate sends an authoritative query timeout to VTTablet when the `QUERY_TIMEOUT_MS` comment directive,
`query_timeout` session system variable, or `query-timeout` flag is set.
The order of precedence is: `QUERY_TIMEOUT_MS` > `query_timeout` > `query-timeout`.
The order of precedence is: comment directive > session variable > VTGate flag.
VTTablet overrides its default query timeout with the value received from VTGate.
All timeouts are specified in milliseconds.

When a query is executed inside a transaction, this behavior does not apply; instead,
the smaller of the transaction timeout or the query timeout from VTGate is used.
When a query is executed inside a transaction, there is an additional nuance. The actual timeout used will be the smaller
of the transaction timeout and the query timeout.

A query can also be set to have no timeout by using the `QUERY_TIMEOUT_MS` comment directive with a value of `0`.

Expand All @@ -157,52 +154,58 @@ Example usage:

### <a id="new-backup-engine"/>New Backup Engine (EXPERIMENTAL)

We are introducing a backup engine supporting logical backups starting on v21 to support use cases that require something else besides physical backups. This is experimental and is based on the
[MySQL Shell](https://dev.mysql.com/doc/mysql-shell/8.0/en/).
We are introducing a new backup engine for logical backups in order to support use cases that require something other
than physical backups. This feature is experimental and is based on [MySQL Shell](https://dev.mysql.com/doc/mysql-shell/8.0/en/).

The new engine is enabled by using `--backup_engine_implementation=mysqlshell`. There are other options that are required, so [check the docs](https://vitess.io/docs/21.0/user-guides/operating-vitess/backup-and-restore/creating-a-backup/) on which options are required and how to use it.
The new engine is enabled by using `--backup_engine_implementation=mysqlshell`. There are other options that are required,
so please read the [documentation](https://vitess.io/docs/21.0/user-guides/operating-vitess/backup-and-restore/creating-a-backup/) to learn which options are required and how to configure them.

### <a id="dynamic-vreplication-configuration"/>Dynamic VReplication Configuration

Currently many of the configuration options for VReplication Workflows are vttablet flags. This means that any change
requires restarts of vttablets. We now allow these to be overridden while creating a workflow or dynamically once
the workflow is in progress. See https://github.com/vitessio/vitess/pull/16583 for details.
Previously, many of the configuration options for VReplication Workflows had to be provided using VTTablet flags. This
meant that any change to VReplication configuration required restarting VTTablets. We now allow these to be overridden
while creating a workflow or dynamically after the workflow is already in progress.

### <a id="reference-table-materialization"/>Reference Table Materialization

There is a new option in [`Materialize` workflows](https://vitess.io/docs/reference/vreplication/materialize/) to keep
a synced copy of [reference or lookup tables](https://vitess.io/docs/reference/vreplication/reference_tables/)
(countries, states, zip_codes, etc) from an unsharded keyspace, which holds the source of truth for the reference
There is a new option in [`Materialize` workflows](https://vitess.io/docs/reference/vreplication/materialize/) to keep a synced copy of [reference or lookup tables](https://vitess.io/docs/reference/vreplication/reference_tables/)
(countries, states, zip codes, etc) from an unsharded keyspace, which holds the source of truth for the reference
table, to all shards in a sharded keyspace.

### <a id="new-vexplain-modes"/>New VEXPLAIN Modes: TRACE and KEYS

#### VEXPLAIN TRACE

The new TRACE mode for VEXPLAIN provides a detailed execution trace of queries, showing how they're processed through various operators and interactions with tablets. This mode is particularly useful for:
The new `TRACE` mode for `VEXPLAIN` provides a detailed execution trace of queries, showing how they're processed through various
operators and interactions with tablets. This mode is particularly useful for:

- Identifying performance bottlenecks
- Understanding query execution patterns
- Optimizing complex queries
- Debugging unexpected query behavior

TRACE mode runs the query and logs all interactions, returning a JSON representation of the query execution plan with additional statistics like number of calls, average rows processed, and number of shards queried.
`TRACE` mode runs the query and logs all interactions, returning a JSON representation of the query execution plan with additional
statistics like number of calls, average rows processed, and number of shards queried.

#### VEXPLAIN KEYS

The KEYS mode for VEXPLAIN offers a concise summary of query structure, highlighting columns used in joins, filters, and grouping operations. This information is crucial for:
The `KEYS` mode for `VEXPLAIN` offers a concise summary of query structure, highlighting columns used in joins, filters, and
grouping operations. This information is crucial for:

- Identifying potential sharding key candidates
- Optimizing query performance
- Analyzing query patterns to inform database design decisions

KEYS mode analyzes the query structure without executing it, providing JSON output that includes grouping columns, join columns, filter columns (potential candidates for indexes, primary keys, or sharding keys), and the statement type.
`KEYS` mode analyzes the query structure without executing it, providing JSON output that includes grouping columns, join columns,
filter columns (potential candidates for indexes, primary keys, or sharding keys), and the statement type.

These new VEXPLAIN modes enhance Vitess's query analysis capabilities, allowing for more informed decisions about sharding strategies and query optimization.
These new `VEXPLAIN` modes enhance Vitess's query analysis capabilities, allowing for more informed decisions about sharding
strategies and query optimization.

### <a id="errant-gtid-vttablet"/>Errant GTID Detection on Vttablets
### <a id="errant-gtid-vttablet"/>Errant GTID Detection on VTTablets

Vttablets now run an errant GTID detection logic before they join the replication stream. So, if a replica has an errant GTID, it will
not start replicating from the primary. It will fail the call the set its replication source because of the errant GTID. This prevents us
from running into situations from which recovery is very hard.
VTTablets now run an errant GTID detection logic before they join the replication stream. So, if a replica has an errant GTID, it will
not start replicating from the primary. This protects us from running into situations which are very difficult to recover from.

For users running with the vitess operator on kubernetes, this change means that the replicas with errant GTIDs will have broken replication and will report as unready. The users will need to manually clean up these errant replica tablets.
For users running with the vitess-operator on Kubernetes, this change means that replica tablets with errant GTIDs will have broken
replication and will report as unready. Users will need to manually replace and clean up these errant replica tablets.

0 comments on commit aa7ba63

Please sign in to comment.