426 lines
16 KiB
YAML
426 lines
16 KiB
YAML
cpu:
|
|
api_field: cluster.capacityConfig.vcpuCount
|
|
arg_name: cpu
|
|
processor: googlecloudsdk.command_lib.managed_kafka.util:ValidateCPU
|
|
help_text: |
|
|
The number of vCPUs to provision for the cluster. The minimum is 3.
|
|
|
|
memory:
|
|
api_field: cluster.capacityConfig.memoryBytes
|
|
arg_name: memory
|
|
type: googlecloudsdk.core.util.scaled_integer:ParseInteger
|
|
help_text: |
|
|
The memory to provision for the cluster in bytes. The value must be between 1 GiB and 8 GiB
|
|
per vCPU.
|
|
Ex. 1024Mi, 4Gi.
|
|
|
|
subnets:
|
|
arg_name: subnets
|
|
type: "googlecloudsdk.calliope.arg_parsers:ArgList:"
|
|
help_text: |
|
|
A comma-separated list of VPC subnets from which the cluster is accessible. Both broker and bootstrap server
|
|
IP addresses and DNS entries are automatically created in each subnet. Only one subnet per
|
|
network is allowed, and the subnet must be located in the same region as the cluster.
|
|
The project may differ. A minimum of 1 subnet is required. A maximum of 10 subnets can be
|
|
specified. Use commas to separate multiple subnets. The name of the subnet must be in the format
|
|
projects/``PROJECT_ID''/regions/``REGION''/subnetworks/``SUBNET''.
|
|
|
|
labels:
|
|
api_field: cluster.labels
|
|
arg_name: labels
|
|
metavar: KEY=VALUE
|
|
type: "googlecloudsdk.calliope.arg_parsers:ArgDict:"
|
|
help_text: |
|
|
List of label KEY=VALUE pairs to add.
|
|
Keys must start with a lowercase character and contain only hyphens (`-`),
|
|
underscores (```_```), lowercase characters, and numbers. Values must contain only
|
|
hyphens (`-`), underscores (```_```), lowercase characters, and numbers.
|
|
|
|
encryption-key:
|
|
api_field: cluster.gcpConfig.kmsKey
|
|
arg_name: encryption-key
|
|
help_text: |
|
|
The relative resource path of the Cloud KMS key to use for encryption in the form:
|
|
projects/``PROJECT_ID''/locations/``LOCATION''/keyRings/``KEY_RING''/cryptoKeys/``KEY''.
|
|
The key must be located in the same region as the cluster. The key cannot be changed once set.
|
|
|
|
auto-rebalance:
|
|
api_field: cluster.rebalanceConfig.mode
|
|
arg_name: auto-rebalance
|
|
action: store_true
|
|
help_text: |
|
|
Whether the automatic rebalancing is enabled. If automatic rebalancing
|
|
is enabled, topic partitions are rebalanced among brokers when the number of
|
|
CPUs in the cluster changes. Automatic rebalancing is enabled by default.
|
|
Use --no-auto-rebalance to disable this flag.
|
|
choices:
|
|
- arg_value: true
|
|
enum_value: AUTO_REBALANCE_ON_SCALE_UP
|
|
- arg_value: false
|
|
enum_value: NO_REBALANCE
|
|
|
|
mtls-ca-pools:
|
|
arg_name: mtls-ca-pools
|
|
type: "googlecloudsdk.calliope.arg_parsers:ArgList:"
|
|
help_text: |
|
|
A comma-separated list of CA pools from the Google Cloud Certificate Authority Service.
|
|
The root certificates of these CA pools will be installed in the truststore of each broker
|
|
in the cluster for use with mTLS. A maximum of 10 CA pools can be specified.
|
|
CA pools can be in a different project and region than the cluster.
|
|
This command overwrites the entire set of pools currently configured on the cluster.
|
|
If you want to add a new pool to an existing configuration, you must provide the full list of
|
|
both the old and new CA pools in the command.
|
|
Each CA pool must be in the format
|
|
projects/``PROJECT_ID''/locations/``LOCATION''/caPools/``CA_POOL''. Clear the CA pools
|
|
using the `--clear-mtls-ca-pools` flag.
|
|
|
|
clear-mtls-ca-pools:
|
|
arg_name: clear-mtls-ca-pools
|
|
action: store_true
|
|
help_text: |
|
|
Remove all the CA pools from the cluster. This will remove all root certificates
|
|
from the truststore of each broker in the cluster.
|
|
|
|
allow-broker-downscale-on-cluster-upscale:
|
|
api_field: cluster.updateOptions.allowBrokerDownscaleOnClusterUpscale
|
|
arg_name: allow-broker-downscale-on-cluster-upscale
|
|
action: store_true
|
|
default: null
|
|
help_text: |
|
|
If enabled, this setting allows an update operation that could significantly decrease the
|
|
per-broker vCPU and/or memory allocation, which can lead to reduced performance and
|
|
availability. By default, an update operation will fail if it results in a reduction of 10% or
|
|
more to the brokers' vCPU or memory allocation.
|
|
|
|
ssl-principal-mapping-rules:
|
|
api_field: cluster.tlsConfig.sslPrincipalMappingRules
|
|
arg_name: ssl-principal-mapping-rules
|
|
type: str
|
|
help_text: |
|
|
The rules for mapping mTLS certificate Distinguished Names (DNs) to shortened principal names
|
|
for Kafka ACLs. This flag corresponds exactly to the `ssl.principal.mapping.rules` broker config
|
|
and matches the format and syntax defined in the Apache Kafka documentation.
|
|
Setting or modifying this field will trigger a rolling restart of the Kafka brokers
|
|
to apply the change. An empty string means that the default Kafka behavior is used.
|
|
Example: "RULE:^CN=(.*?),OU=ServiceUsers.*$/$1@example.com/,DEFAULT"
|
|
|
|
broker-disk-size-gib:
|
|
api_field: cluster.brokerCapacityConfig.diskSizeGib
|
|
arg_name: broker-disk-size-gib
|
|
help_text: |
|
|
The amount of local disk to provision for each broker in Gibibytes. Minimum: 100 GiB.
|
|
|
|
# Trying to define this as a resouce causes test failures in yaml_command_schema_test so we define
|
|
# it as a flag instead.
|
|
kafka-cluster:
|
|
arg_name: kafka-cluster
|
|
help_text: |
|
|
The resource path of the Kafka cluster to connect to, or the name of the Kafka cluster to
|
|
connect to if the cluster is in the same project as the Connect cluster.
|
|
|
|
connect-cpu:
|
|
api_field: connectCluster.capacityConfig.vcpuCount
|
|
arg_name: cpu
|
|
processor: googlecloudsdk.command_lib.managed_kafka.util:ValidateCPU
|
|
help_text: |
|
|
The number of vCPUs to provision for the cluster. The minimum is 3.
|
|
|
|
connect-memory:
|
|
api_field: connectCluster.capacityConfig.memoryBytes
|
|
arg_name: memory
|
|
type: googlecloudsdk.core.util.scaled_integer:ParseInteger
|
|
help_text: |
|
|
The memory to provision for the cluster in bytes. The value must be between 1 GiB and 8 GiB
|
|
per vCPU.
|
|
Ex. 1024Mi, 4Gi.
|
|
|
|
connect-labels:
|
|
api_field: connectCluster.labels
|
|
arg_name: labels
|
|
metavar: KEY=VALUE
|
|
type: "googlecloudsdk.calliope.arg_parsers:ArgDict:"
|
|
help_text: |
|
|
List of label KEY=VALUE pairs to add.
|
|
Keys must start with a lowercase character and contain only hyphens (`-`),
|
|
underscores (```_```), lowercase characters, and numbers. Values must contain only
|
|
hyphens (`-`), underscores (```_```), lowercase characters, and numbers.
|
|
|
|
clear-connect-labels:
|
|
arg_name: clear-labels
|
|
action: store_true
|
|
help_text: |
|
|
Remove all the labels from the connect cluster.
|
|
|
|
connect-encryption-key:
|
|
api_field: connectCluster.gcpConfig.kmsKey
|
|
arg_name: encryption-key
|
|
help_text: |
|
|
The relative resource path of the Cloud KMS key to use for encryption in the form:
|
|
projects/``PROJECT_ID''/locations/``LOCATION''/keyRings/``KEY_RING''/cryptoKeys/``KEY''.
|
|
The key must be located in the same region as the cluster. The key cannot be changed once set.
|
|
|
|
dns-name:
|
|
arg_name: dns-name
|
|
api_field: connectCluster.gcpConfig.accessConfig.networkConfigs.dnsDomainNames
|
|
repeated: true
|
|
action: append
|
|
help_text: |
|
|
DNS domain name from the subnet's network to be made visible to the Connect Cluster.
|
|
|
|
secret:
|
|
arg_name: secret
|
|
api_field: connectCluster.gcpConfig.secretPaths
|
|
repeated: true
|
|
action: append
|
|
help_text: |
|
|
Secrets to load into workers. Exact SecretVersions from Secret Manager must
|
|
be provided -- aliases are not supported. Up to 32 secrets may be loaded
|
|
into one cluster.
|
|
Format: projects/<project-id>/secrets/<secret-name>/versions/<version-id>
|
|
|
|
connect-primary-subnet:
|
|
arg_name: primary-subnet
|
|
api_field: connectCluster.gcpConfig.accessConfig.networkConfigs.primarySubnet
|
|
help_text: |
|
|
VPC subnet to make available to the Kafka Connect cluster. Structured
|
|
like: projects/{project}/regions/{region}/subnetworks/{subnet_id}.
|
|
The primary subnet is used to create a Private Service Connect (PSC) interface for the Kafka Connect workers.
|
|
It must be located in the same region as the Connect cluster.
|
|
|
|
connect-additional-subnet:
|
|
arg_name: additional-subnet
|
|
repeated: true
|
|
help_text: |
|
|
Additional subnet to make available to the Kafka Connect cluster. Structured
|
|
like: projects/{project}/regions/{region}/subnetworks/{subnet_id}.
|
|
# Custom action since we cannot use both "action: append and action.deprecated"
|
|
action: googlecloudsdk.command_lib.managed_kafka.util:AdditionalSubnetDeprecationBeforeAppendAction
|
|
|
|
partitions:
|
|
api_field: topic.partitionCount
|
|
arg_name: partitions
|
|
help_text: |
|
|
The number of partitions in a topic. You can increase the partition count for a topic, but you
|
|
cannot decrease it. Increasing partitions for a topic that uses a key might change how messages
|
|
are distributed.
|
|
|
|
replication-factor:
|
|
api_field: topic.replicationFactor
|
|
arg_name: replication-factor
|
|
help_text: |
|
|
The number of replicas of each partition. A replication factor of 3 is
|
|
recommended for high availability.
|
|
|
|
configs:
|
|
api_field: topic.configs
|
|
arg_name: configs
|
|
metavar: KEY=VALUE
|
|
type: "googlecloudsdk.calliope.arg_parsers:ArgDict:"
|
|
help_text: |
|
|
Configuration for the topic that are overridden from the cluster defaults.
|
|
The key of the map is a Kafka topic property name, for example:
|
|
`cleanup.policy=compact`,`compression.type=producer`. If you provide a map with a key that
|
|
already exists, only that configuration is updated. If the map contains a key that does not
|
|
exist, the entry is appended to the topic configuration.
|
|
|
|
connectCluster-configs:
|
|
arg_name: configs
|
|
api_field: connectCluster.config
|
|
metavar: KEY=VALUE
|
|
type: "googlecloudsdk.calliope.arg_parsers:ArgDict:"
|
|
help_text: |
|
|
Configuration for the connect cluster that are overridden from the cluster defaults.
|
|
The key of the map is a Kafka topic property name, for example:
|
|
`cleanup.policy=compact`,`compression.type=producer`.
|
|
|
|
connectCluster-config-file:
|
|
arg_name: config-file
|
|
metavar: JSON|YAML|FILE
|
|
type: "googlecloudsdk.calliope.arg_parsers:ArgObject:"
|
|
help_text: |
|
|
The path to the JSON or YAML file containing the configuration that are overridden
|
|
from the cluster or connector defaults. This also supports inline JSON or YAML.
|
|
|
|
connector-configs:
|
|
arg_name: configs
|
|
api_field: connector.configs
|
|
metavar: KEY=VALUE
|
|
type: "googlecloudsdk.calliope.arg_parsers:ArgDict:"
|
|
help_text: |
|
|
Configuration for the connector that are overridden from the connector defaults.
|
|
The key of the map is a Kafka topic property name, for example:
|
|
`cleanup.policy=compact`,`compression.type=producer`.
|
|
|
|
connector-config-file:
|
|
arg_name: config-file
|
|
metavar: JSON|YAML|FILE
|
|
type: "googlecloudsdk.calliope.arg_parsers:ArgObject:"
|
|
help_text: |
|
|
The path to the JSON or YAML file containing the configuration that are overridden
|
|
from the connector defaults. This also supports inline JSON or YAML.
|
|
|
|
task-restart-min-backoff:
|
|
api_field: connector.taskRestartPolicy.minimumBackoff
|
|
arg_name: task-restart-min-backoff
|
|
type: googlecloudsdk.core.util.times:ParseDuration
|
|
processor: googlecloudsdk.core.util.times:FormatDuration
|
|
help_text: |
|
|
The minimum amount of time to wait before retrying a failed task in seconds.
|
|
This sets a lower bound for the backoff delay. The default value is 60s.
|
|
See $ gcloud topic datetimes for information on duration formats.
|
|
task-restart-max-backoff:
|
|
api_field: connector.taskRestartPolicy.maximumBackoff
|
|
arg_name: task-restart-max-backoff
|
|
type: googlecloudsdk.core.util.times:ParseDuration
|
|
processor: googlecloudsdk.core.util.times:FormatDuration
|
|
help_text: |
|
|
The maximum amount of time to wait before retrying a failed task in seconds.
|
|
This sets an upper bound for the backoff delay. The default value is 1800s (30 minutes).
|
|
See $ gcloud topic datetimes for information on duration formats.
|
|
task-retry-disabled:
|
|
api_field: connector.taskRestartPolicy.taskRetryDisabled
|
|
arg_name: task-retry-disabled
|
|
type: bool
|
|
help_text: |
|
|
Disable default task retry policy.
|
|
|
|
clear-secrets:
|
|
arg_name: clear-secrets
|
|
action: store_true
|
|
help_text: |
|
|
Remove all the secrets from the connect cluster.
|
|
|
|
clear-dns-names:
|
|
arg_name: clear-dns-names
|
|
action: store_true
|
|
help_text: |
|
|
Remove all the DNS domain names for the connect cluster.
|
|
|
|
clear-configs:
|
|
arg_name: clear-configs
|
|
action: store_true
|
|
help_text: |
|
|
Remove all the configurations for the topic.
|
|
|
|
# TODO(b/336117815): Provide hard examples and external docs on this flag.
|
|
topics-file:
|
|
arg_name: topics-file
|
|
type: "googlecloudsdk.calliope.arg_parsers:ArgObject:"
|
|
metavar: JSON|YAML|FILE
|
|
help_text: |
|
|
The path to the JSON or YAML file containing the configuration of the topics to be updated for
|
|
the consumer group. This also supports inline JSON or YAML.
|
|
|
|
acl-entry:
|
|
arg_name: acl-entry
|
|
api_field: acl.aclEntries
|
|
help_text: |
|
|
An acl entry that configures access for a principal, for a specific operation on the acl's
|
|
resource pattern. This flag can be repeated.
|
|
|
|
``PRINCIPAL'' is the principal. Specified as Google Cloud account, with the Kafka StandardAuthorizer prefix
|
|
"User:". For example: "User:admin@project.iam.gserviceaccount.com".
|
|
Can be the wildcard "User:```*```" to refer to all users.
|
|
|
|
``OPERATION'' is the operation type. Allowed values are: ALL, READ, WRITE,
|
|
CREATE, DELETE, ALTER, DESCRIBE, CLUSTER_ACTION, DESCRIBE_CONFIGS, ALTER_CONFIGS,
|
|
IDEMPOTENT_WRITE.
|
|
|
|
``PERMISSION-TYPE'' is the permission type. Allowed values are: ALLOW, DENY.
|
|
|
|
``HOST'' is the host. Must be set to "```*```" for Managed Service for Apache Kafka.
|
|
|
|
Example acl-entry:
|
|
"principal=User:admin@project.iam.gserviceaccount.com,operation=ALL,permission-type=ALLOW,host=```*```"
|
|
type:
|
|
arg_dict:
|
|
flatten: false
|
|
spec:
|
|
- api_field: principal
|
|
arg_name: principal
|
|
type: str
|
|
required: true
|
|
- api_field: operation
|
|
arg_name: operation
|
|
type: str
|
|
required: true
|
|
- api_field: permissionType
|
|
arg_name: permission-type
|
|
type: str
|
|
required: true
|
|
- api_field: host
|
|
arg_name: host
|
|
type: str
|
|
required: true
|
|
|
|
acl-entries-from-file:
|
|
arg_name: acl-entries-from-file
|
|
api_field: acl
|
|
type: "googlecloudsdk.calliope.arg_parsers:FileContents:"
|
|
processor: googlecloudsdk.core.yaml:load
|
|
help_text: |
|
|
Path to a JSON or YAML file containing the acl entries to use in the acl.
|
|
|
|
etag:
|
|
arg_name: etag
|
|
api_field: acl.etag
|
|
type: str
|
|
required: true
|
|
help_text: |
|
|
etag returned in the response to a previous create or describe
|
|
command. The etag is used for concurrency control, to ensure that the
|
|
client and server agree on the current set of acl entries in the Kafka
|
|
cluster, before full replacement in the update command.
|
|
|
|
acl-entry-principal:
|
|
api_field: aclEntry.principal
|
|
arg_name: principal
|
|
type: str
|
|
required: true
|
|
help_text: |
|
|
The principal. Specified as Google Cloud account, with the Kafka StandardAuthorizer prefix
|
|
"User:". For example: "User:admin@project.iam.gserviceaccount.com".
|
|
Can be the wildcard "User:```*```" to refer to all users.
|
|
|
|
acl-entry-operation:
|
|
api_field: aclEntry.operation
|
|
arg_name: operation
|
|
type: str
|
|
required: true
|
|
help_text: |
|
|
The operation type. Allowed values are: ALL, READ, WRITE, CREATE, DELETE, ALTER, DESCRIBE,
|
|
CLUSTER_ACTION, DESCRIBE_CONFIGS, ALTER_CONFIGS, IDEMPOTENT_WRITE.
|
|
|
|
See https://kafka.apache.org/documentation/#operations_resources_and_protocols
|
|
for the mapping of operations to Kafka protocols.
|
|
|
|
acl-entry-permission-type:
|
|
api_field: aclEntry.permissionType
|
|
arg_name: permission-type
|
|
type: str
|
|
default: "ALLOW"
|
|
help_text: |
|
|
The permission type. Allowed values are: ALLOW, DENY.
|
|
|
|
acl-entry-host:
|
|
api_field: aclEntry.host
|
|
arg_name: host
|
|
type: str
|
|
default: '*'
|
|
help_text: |
|
|
The host. Must be set to "```*```" for Managed Service for Apache Kafka.
|
|
|
|
full:
|
|
api_field: view
|
|
arg_name: full
|
|
action: store_true
|
|
default: null
|
|
help_text: |
|
|
Show detailed information about individual brokers, such as broker id and zone, as well as the
|
|
Kafka version running on the cluster.
|
|
choices:
|
|
- arg_value: true
|
|
enum_value: CLUSTER_VIEW_FULL
|
|
- arg_value: false
|
|
enum_value: CLUSTER_VIEW_BASIC
|