mmseg.apis

mmseg.datasets

datasets

transforms

mmseg.engine

hooks

class mmseg.engine.hooks.SegVisualizationHook(draw: bool = False, interval: int = 50, show: bool = False, wait_time: float = 0.0, backend_args: Optional[dict] = None)

Segmentation Visualization Hook. Used to visualize validation and testing process prediction results.

In the testing phase:

1. If show is True, it means that only the prediction results are

   visualized without storing data, so vis_backends needs to be excluded.

参数

• draw (bool) – whether to draw prediction results. If it is False, it means that no drawing will be done. Defaults to False.

• interval (int) – The interval of visualization. Defaults to 50.

• show (bool) – Whether to display the drawn image. Default to False.

• wait_time (float) – The interval of show (s). Defaults to 0.

• backend_args (dict, Optional) – Arguments to instantiate a file backend. See https://mmengine.readthedocs.io/en/latest/api/fileio.htm for details. Defaults to None. Notes: mmcv>=2.0.0rc4, mmengine>=0.2.0 required.

after_test_iter(runner: mmengine.runner.runner.Runner, batch_idx: int, data_batch: dict, outputs: Sequence[mmseg.structures.seg_data_sample.SegDataSample]) → None

Run after every testing iterations.

参数

• runner (Runner) – The runner of the testing process.

• batch_idx (int) – The index of the current batch in the val loop.

• data_batch (dict) – Data from dataloader.

• outputs (Sequence[SegDataSample]) – A batch of data samples that contain annotations and predictions.

after_val_iter(runner: mmengine.runner.runner.Runner, batch_idx: int, data_batch: dict, outputs: Sequence[mmseg.structures.seg_data_sample.SegDataSample]) → None

Run after every self.interval validation iterations.

参数

• runner (Runner) – The runner of the validation process.

• batch_idx (int) – The index of the current batch in the val loop.

• data_batch (dict) – Data from dataloader.

• outputs (Sequence[SegDataSample]]) – A batch of data samples that contain annotations and predictions.

optimizers

class mmseg.engine.optimizers.ForceDefaultOptimWrapperConstructor(optim_wrapper_cfg: dict, paramwise_cfg: Optional[dict] = None)

Default constructor with forced optimizer settings.

This constructor extends the default constructor to add an option for forcing default optimizer settings. This is useful for ensuring that certain parameters or layers strictly adhere to pre-defined default settings, regardless of any custom settings specified.

By default, each parameter share the same optimizer settings, and we provide an argument paramwise_cfg to specify parameter-wise settings. It is a dict and may contain various fields like ‘custom_keys’, ‘bias_lr_mult’, etc., as well as the additional field force_default_settings which allows for enforcing default settings on optimizer parameters.

• custom_keys (dict): Specified parameters-wise settings by keys. If one of the keys in custom_keys is a substring of the name of one parameter, then the setting of the parameter will be specified by custom_keys[key] and other setting like bias_lr_mult etc. will be ignored. It should be noted that the aforementioned key is the longest key that is a substring of the name of the parameter. If there are multiple matched keys with the same length, then the key with lower alphabet order will be chosen. custom_keys[key] should be a dict and may contain fields lr_mult and decay_mult. See Example 2 below.

• bias_lr_mult (float): It will be multiplied to the learning rate for all bias parameters (except for those in normalization layers and offset layers of DCN).

• bias_decay_mult (float): It will be multiplied to the weight decay for all bias parameters (except for those in normalization layers, depthwise conv layers, offset layers of DCN).

• norm_decay_mult (float): It will be multiplied to the weight decay for all weight and bias parameters of normalization layers.

• flat_decay_mult (float): It will be multiplied to the weight decay for all one-dimensional parameters

• dwconv_decay_mult (float): It will be multiplied to the weight decay for all weight and bias parameters of depthwise conv layers.

• dcn_offset_lr_mult (float): It will be multiplied to the learning rate for parameters of offset layer in the deformable convs of a model.

• bypass_duplicate (bool): If true, the duplicate parameters would not be added into optimizer. Defaults to False.

• force_default_settings (bool): If true, this will override any custom settings defined by custom_keys and enforce the use of default settings for optimizer parameters like bias_lr_mult. This is particularly useful when you want to ensure that certain layers or parameters adhere strictly to the pre-defined default settings.

注解

1. If the option dcn_offset_lr_mult is used, the constructor will override the effect of bias_lr_mult in the bias of offset layer. So be careful when using both bias_lr_mult and dcn_offset_lr_mult. If you wish to apply both of them to the offset layer in deformable convs, set dcn_offset_lr_mult to the original dcn_offset_lr_mult * bias_lr_mult.

2. If the option dcn_offset_lr_mult is used, the constructor will apply it to all the DCN layers in the model. So be careful when the model contains multiple DCN layers in places other than backbone.

3. When the option force_default_settings is true, it will override any custom settings provided in custom_keys. This ensures that the default settings for the optimizer parameters are used.

参数

• optim_wrapper_cfg (dict) –

  The config dict of the optimizer wrapper.

  Required fields of optim_wrapper_cfg are

  • type: class name of the OptimizerWrapper

  • optimizer: The configuration of optimizer.

  Optional fields of optim_wrapper_cfg are

  • any arguments of the corresponding optimizer wrapper type, e.g., accumulative_counts, clip_grad, etc.

  Required fields of optimizer are

  • type: class name of the optimizer.

  Optional fields of optimizer are

  • any arguments of the corresponding optimizer type, e.g., lr, weight_decay, momentum, etc.

• paramwise_cfg (dict, optional) – Parameter-wise options.

Example 1:

>>> model = torch.nn.modules.Conv1d(1, 1, 1)
>>> optim_wrapper_cfg = dict(
>>>     dict(type='OptimWrapper', optimizer=dict(type='SGD', lr=0.01,
>>>         momentum=0.9, weight_decay=0.0001))
>>> paramwise_cfg = dict(norm_decay_mult=0.)
>>> optim_wrapper_builder = DefaultOptimWrapperConstructor(
>>>     optim_wrapper_cfg, paramwise_cfg)
>>> optim_wrapper = optim_wrapper_builder(model)

Example 2:

>>> # assume model have attribute model.backbone and model.cls_head
>>> optim_wrapper_cfg = dict(type='OptimWrapper', optimizer=dict(
>>>     type='SGD', lr=0.01, weight_decay=0.95))
>>> paramwise_cfg = dict(custom_keys={
>>>     'backbone': dict(lr_mult=0.1, decay_mult=0.9)})
>>> optim_wrapper_builder = DefaultOptimWrapperConstructor(
>>>     optim_wrapper_cfg, paramwise_cfg)
>>> optim_wrapper = optim_wrapper_builder(model)
>>> # Then the `lr` and `weight_decay` for model.backbone is
>>> # (0.01 * 0.1, 0.95 * 0.9). `lr` and `weight_decay` for
>>> # model.cls_head is (0.01, 0.95).

add_params(params: List[dict], module: torch.nn.modules.module.Module, prefix: str = '', is_dcn_module: Optional[Union[int, float]] = None) → None

Add all parameters of module to the params list.

The parameters of the given module will be added to the list of param groups, with specific rules defined by paramwise_cfg.

参数

• params (list[dict]) – A list of param groups, it will be modified in place.

• module (nn.Module) – The module to be added.

• prefix (str) – The prefix of the module

• is_dcn_module (int|float|None) – If the current module is a submodule of DCN, is_dcn_module will be passed to control conv_offset layer’s learning rate. Defaults to None.

class mmseg.engine.optimizers.LayerDecayOptimizerConstructor(optim_wrapper_cfg, paramwise_cfg)

Different learning rates are set for different layers of backbone.

Note: Currently, this optimizer constructor is built for BEiT, and it will be deprecated. Please use LearningRateDecayOptimizerConstructor instead.

class mmseg.engine.optimizers.LearningRateDecayOptimizerConstructor(optim_wrapper_cfg: dict, paramwise_cfg: Optional[dict] = None)

Different learning rates are set for different layers of backbone.

Note: Currently, this optimizer constructor is built for ConvNeXt, BEiT and MAE.

add_params(params, module, **kwargs)

Add all parameters of module to the params list.

The parameters of the given module will be added to the list of param groups, with specific rules defined by paramwise_cfg.

参数

• params (list[dict]) – A list of param groups, it will be modified in place.

• module (nn.Module) – The module to be added.

mmseg.evaluation

metrics

class mmseg.evaluation.metrics.CityscapesMetric(output_dir: str, ignore_index: int = 255, format_only: bool = False, keep_results: bool = False, collect_device: str = 'cpu', prefix: Optional[str] = None, **kwargs)

Cityscapes evaluation metric.

参数

• output_dir (str) – The directory for output prediction

• ignore_index (int) – Index that will be ignored in evaluation. Default: 255.

• format_only (bool) – Only format result for results commit without perform evaluation. It is useful when you want to format the result to a specific format and submit it to the test server. Defaults to False.

• keep_results (bool) – Whether to keep the results. When format_only is True, keep_results must be True. Defaults to False.

• collect_device (str) – Device name used for collecting results from different ranks during distributed training. Must be ‘cpu’ or ‘gpu’. Defaults to ‘cpu’.

• prefix (str, optional) – The prefix that will be added in the metric names to disambiguate homonymous metrics of different evaluators. If prefix is not provided in the argument, self.default_prefix will be used instead. Defaults to None.

compute_metrics(results: list) → Dict[str, float]

Compute the metrics from processed results.

参数

results (list) – Testing results of the dataset.

返回

float]: Cityscapes evaluation results.

返回类型

dict[str

process(data_batch: dict, data_samples: Sequence[dict]) → None

Process one batch of data and data_samples.

The processed results should be stored in self.results, which will be used to computed the metrics when all batches have been processed.

参数

• data_batch (dict) – A batch of data from the dataloader.

• data_samples (Sequence[dict]) – A batch of outputs from the model.

class mmseg.evaluation.metrics.DepthMetric(depth_metrics: Optional[List[str]] = None, min_depth_eval: float = 0.0, max_depth_eval: float = inf, crop_type: Optional[str] = None, depth_scale_factor: float = 1.0, collect_device: str = 'cpu', output_dir: Optional[str] = None, format_only: bool = False, prefix: Optional[str] = None, **kwargs)

Depth estimation evaluation metric.

参数

• depth_metrics (List[str], optional) – List of metrics to compute. If not specified, defaults to all metrics in self.METRICS.

• min_depth_eval (float) – Minimum depth value for evaluation. Defaults to 0.0.

• max_depth_eval (float) – Maximum depth value for evaluation. Defaults to infinity.

• crop_type (str, optional) – Specifies the type of cropping to be used during evaluation. This option can affect how the evaluation mask is generated. Currently, ‘nyu_crop’ is supported, but other types can be added in future. Defaults to None if no cropping should be applied.

• depth_scale_factor (float) – Factor to scale the depth values. Defaults to 1.0.

• collect_device (str) – Device name used for collecting results from different ranks during distributed training. Must be ‘cpu’ or ‘gpu’. Defaults to ‘cpu’.

• output_dir (str) – The directory for output prediction. Defaults to None.

• format_only (bool) – Only format result for results commit without perform evaluation. It is useful when you want to save the result to a specific format and submit it to the test server. Defaults to False.

• prefix (str, optional) – The prefix that will be added in the metric names to disambiguate homonymous metrics of different evaluators. If prefix is not provided in the argument, self.default_prefix will be used instead. Defaults to None.

compute_metrics(results: list) → Dict[str, float]

Compute the metrics from processed results.

参数

results (list) – The processed results of each batch.

返回

The computed metrics. The keys are the names of

the metrics, and the values are corresponding results. The keys are identical with self.metrics.

返回类型

Dict[str, float]

process(data_batch: dict, data_samples: Sequence[dict]) → None

Process one batch of data and data_samples.

The processed results should be stored in self.results, which will be used to compute the metrics when all batches have been processed.

参数

• data_batch (dict) – A batch of data from the dataloader.

• data_samples (Sequence[dict]) – A batch of outputs from the model.

class mmseg.evaluation.metrics.IoUMetric(ignore_index: int = 255, iou_metrics: List[str] = ['mIoU'], nan_to_num: Optional[int] = None, beta: int = 1, collect_device: str = 'cpu', output_dir: Optional[str] = None, format_only: bool = False, prefix: Optional[str] = None, **kwargs)

IoU evaluation metric.

参数

• ignore_index (int) – Index that will be ignored in evaluation. Default: 255.

• iou_metrics (list[str] | str) – Metrics to be calculated, the options includes ‘mIoU’, ‘mDice’ and ‘mFscore’.

• nan_to_num (int, optional) – If specified, NaN values will be replaced by the numbers defined by the user. Default: None.

• beta (int) – Determines the weight of recall in the combined score. Default: 1.

• collect_device (str) – Device name used for collecting results from different ranks during distributed training. Must be ‘cpu’ or ‘gpu’. Defaults to ‘cpu’.

• output_dir (str) – The directory for output prediction. Defaults to None.

• format_only (bool) – Only format result for results commit without perform evaluation. It is useful when you want to save the result to a specific format and submit it to the test server. Defaults to False.

• prefix (str, optional) – The prefix that will be added in the metric names to disambiguate homonymous metrics of different evaluators. If prefix is not provided in the argument, self.default_prefix will be used instead. Defaults to None.

compute_metrics(results: list) → Dict[str, float]

Compute the metrics from processed results.

参数

results (list) – The processed results of each batch.

返回

The computed metrics. The keys are the names of

the metrics, and the values are corresponding results. The key mainly includes aAcc, mIoU, mAcc, mDice, mFscore, mPrecision, mRecall.

返回类型

Dict[str, float]

static intersect_and_union(pred_label: torch._VariableFunctionsClass.tensor, label: torch._VariableFunctionsClass.tensor, num_classes: int, ignore_index: int)

Calculate Intersection and Union.

参数

• pred_label (torch.tensor) – Prediction segmentation map or predict result filename. The shape is (H, W).

• label (torch.tensor) – Ground truth segmentation map or label filename. The shape is (H, W).

• num_classes (int) – Number of categories.

• ignore_index (int) – Index that will be ignored in evaluation.

返回

The intersection of prediction and ground truth

histogram on all classes.

torch.Tensor: The union of prediction and ground truth histogram on

all classes.

torch.Tensor: The prediction histogram on all classes. torch.Tensor: The ground truth histogram on all classes.

返回类型

torch.Tensor

process(data_batch: dict, data_samples: Sequence[dict]) → None

Process one batch of data and data_samples.

The processed results should be stored in self.results, which will be used to compute the metrics when all batches have been processed.

参数

• data_batch (dict) – A batch of data from the dataloader.

• data_samples (Sequence[dict]) – A batch of outputs from the model.

static total_area_to_metrics(total_area_intersect: numpy.ndarray, total_area_union: numpy.ndarray, total_area_pred_label: numpy.ndarray, total_area_label: numpy.ndarray, metrics: List[str] = ['mIoU'], nan_to_num: Optional[int] = None, beta: int = 1)

Calculate evaluation metrics :param total_area_intersect: The intersection of prediction

and ground truth histogram on all classes.

参数

• total_area_union (np.ndarray) – The union of prediction and ground truth histogram on all classes.

• total_area_pred_label (np.ndarray) – The prediction histogram on all classes.

• total_area_label (np.ndarray) – The ground truth histogram on all classes.

• metrics (List[str] | str) – Metrics to be evaluated, ‘mIoU’ and ‘mDice’.

• nan_to_num (int, optional) – If specified, NaN values will be replaced by the numbers defined by the user. Default: None.

• beta (int) – Determines the weight of recall in the combined score. Default: 1.

返回

per category evaluation metrics,

shape (num_classes, ).

返回类型

Dict[str, np.ndarray]

mmseg.models

backbones

decode_heads

segmentors

losses

necks

utils

mmseg.structures

structures

class mmseg.structures.BasePixelSampler(**kwargs)

Base class of pixel sampler.

abstract sample(seg_logit, seg_label)

Placeholder for sample function.

class mmseg.structures.OHEMPixelSampler(context, thresh=None, min_kept=100000)

Online Hard Example Mining Sampler for segmentation.

参数

• context (nn.Module) – The context of sampler, subclass of BaseDecodeHead.

• thresh (float, optional) – The threshold for hard example selection. Below which, are prediction with low confidence. If not specified, the hard examples will be pixels of top min_kept loss. Default: None.

• min_kept (int, optional) – The minimum number of predictions to keep. Default: 100000.

sample(seg_logit, seg_label)

Sample pixels that have high loss or with low prediction confidence.

参数

• seg_logit (torch.Tensor) – segmentation logits, shape (N, C, H, W)

• seg_label (torch.Tensor) – segmentation label, shape (N, 1, H, W)

返回

segmentation weight, shape (N, H, W)

返回类型

torch.Tensor

class mmseg.structures.SegDataSample(*, metainfo: Optional[dict] = None, **kwargs)

A data structure interface of MMSegmentation. They are used as interfaces between different components.

The attributes in SegDataSample are divided into several parts:

• ``gt_sem_seg``(PixelData): Ground truth of semantic segmentation.

• ``pred_sem_seg``(PixelData): Prediction of semantic segmentation.

• ``seg_logits``(PixelData): Predicted logits of semantic segmentation.

实际案例

>>> import torch
>>> import numpy as np
>>> from mmengine.structures import PixelData
>>> from mmseg.structures import SegDataSample

>>> data_sample = SegDataSample()
>>> img_meta = dict(img_shape=(4, 4, 3),
...                 pad_shape=(4, 4, 3))
>>> gt_segmentations = PixelData(metainfo=img_meta)
>>> gt_segmentations.data = torch.randint(0, 2, (1, 4, 4))
>>> data_sample.gt_sem_seg = gt_segmentations
>>> assert 'img_shape' in data_sample.gt_sem_seg.metainfo_keys()
>>> data_sample.gt_sem_seg.shape
(4, 4)
>>> print(data_sample)

<SegDataSample(

META INFORMATION

DATA FIELDS gt_sem_seg: <PixelData(

META INFORMATION img_shape: (4, 4, 3) pad_shape: (4, 4, 3)

DATA FIELDS data: tensor([[[1, 1, 1, 0],

[1, 0, 1, 1], [1, 1, 1, 1], [0, 1, 0, 1]]])

) at 0x1c2b4156460>

) at 0x1c2aae44d60>

>>> data_sample = SegDataSample()
>>> gt_sem_seg_data = dict(sem_seg=torch.rand(1, 4, 4))
>>> gt_sem_seg = PixelData(**gt_sem_seg_data)
>>> data_sample.gt_sem_seg = gt_sem_seg
>>> assert 'gt_sem_seg' in data_sample
>>> assert 'sem_seg' in data_sample.gt_sem_seg

mmseg.structures.build_pixel_sampler(cfg, **default_args)

Build pixel sampler for segmentation map.

sampler

class mmseg.structures.sampler.BasePixelSampler(**kwargs)

Base class of pixel sampler.

abstract sample(seg_logit, seg_label)

Placeholder for sample function.

class mmseg.structures.sampler.OHEMPixelSampler(context, thresh=None, min_kept=100000)

Online Hard Example Mining Sampler for segmentation.

参数

• context (nn.Module) – The context of sampler, subclass of BaseDecodeHead.

• thresh (float, optional) – The threshold for hard example selection. Below which, are prediction with low confidence. If not specified, the hard examples will be pixels of top min_kept loss. Default: None.

• min_kept (int, optional) – The minimum number of predictions to keep. Default: 100000.

sample(seg_logit, seg_label)

Sample pixels that have high loss or with low prediction confidence.

参数

• seg_logit (torch.Tensor) – segmentation logits, shape (N, C, H, W)

• seg_label (torch.Tensor) – segmentation label, shape (N, 1, H, W)

返回

segmentation weight, shape (N, H, W)

返回类型

torch.Tensor

mmseg.structures.sampler.build_pixel_sampler(cfg, **default_args)

Build pixel sampler for segmentation map.

mmseg.visualization

mmseg.utils