mmseg.apis¶
mmseg.engine¶
hooks¶
- class mmseg.engine.hooks.SegVisualizationHook(draw: bool = False, interval: int = 50, show: bool = False, wait_time: float = 0.0, backend_args: Optional[dict] = None)[源代码]¶
Segmentation Visualization Hook. Used to visualize validation and testing process prediction results.
In the testing phase:
- If
show
is True, it means that only the prediction results are visualized without storing data, so
vis_backends
needs to be excluded.
- If
- 参数
draw (bool) – whether to draw prediction results. If it is False, it means that no drawing will be done. Defaults to False.
interval (int) – The interval of visualization. Defaults to 50.
show (bool) – Whether to display the drawn image. Default to False.
wait_time (float) – The interval of show (s). Defaults to 0.
backend_args (dict, Optional) – Arguments to instantiate a file backend. See https://mmengine.readthedocs.io/en/latest/api/fileio.htm for details. Defaults to None. Notes: mmcv>=2.0.0rc4, mmengine>=0.2.0 required.
- after_test_iter(runner: mmengine.runner.runner.Runner, batch_idx: int, data_batch: dict, outputs: Sequence[mmseg.structures.seg_data_sample.SegDataSample]) → None[源代码]¶
Run after every testing iterations.
- 参数
runner (
Runner
) – The runner of the testing process.batch_idx (int) – The index of the current batch in the val loop.
data_batch (dict) – Data from dataloader.
outputs (Sequence[
SegDataSample
]) – A batch of data samples that contain annotations and predictions.
- after_val_iter(runner: mmengine.runner.runner.Runner, batch_idx: int, data_batch: dict, outputs: Sequence[mmseg.structures.seg_data_sample.SegDataSample]) → None[源代码]¶
Run after every
self.interval
validation iterations.- 参数
runner (
Runner
) – The runner of the validation process.batch_idx (int) – The index of the current batch in the val loop.
data_batch (dict) – Data from dataloader.
outputs (Sequence[
SegDataSample
]]) – A batch of data samples that contain annotations and predictions.
optimizers¶
- class mmseg.engine.optimizers.ForceDefaultOptimWrapperConstructor(optim_wrapper_cfg: dict, paramwise_cfg: Optional[dict] = None)[源代码]¶
Default constructor with forced optimizer settings.
This constructor extends the default constructor to add an option for forcing default optimizer settings. This is useful for ensuring that certain parameters or layers strictly adhere to pre-defined default settings, regardless of any custom settings specified.
By default, each parameter share the same optimizer settings, and we provide an argument
paramwise_cfg
to specify parameter-wise settings. It is a dict and may contain various fields like ‘custom_keys’, ‘bias_lr_mult’, etc., as well as the additional field force_default_settings which allows for enforcing default settings on optimizer parameters.custom_keys
(dict): Specified parameters-wise settings by keys. If one of the keys incustom_keys
is a substring of the name of one parameter, then the setting of the parameter will be specified bycustom_keys[key]
and other setting likebias_lr_mult
etc. will be ignored. It should be noted that the aforementionedkey
is the longest key that is a substring of the name of the parameter. If there are multiple matched keys with the same length, then the key with lower alphabet order will be chosen.custom_keys[key]
should be a dict and may contain fieldslr_mult
anddecay_mult
. See Example 2 below.bias_lr_mult
(float): It will be multiplied to the learning rate for all bias parameters (except for those in normalization layers and offset layers of DCN).bias_decay_mult
(float): It will be multiplied to the weight decay for all bias parameters (except for those in normalization layers, depthwise conv layers, offset layers of DCN).norm_decay_mult
(float): It will be multiplied to the weight decay for all weight and bias parameters of normalization layers.flat_decay_mult
(float): It will be multiplied to the weight decay for all one-dimensional parametersdwconv_decay_mult
(float): It will be multiplied to the weight decay for all weight and bias parameters of depthwise conv layers.dcn_offset_lr_mult
(float): It will be multiplied to the learning rate for parameters of offset layer in the deformable convs of a model.bypass_duplicate
(bool): If true, the duplicate parameters would not be added into optimizer. Defaults to False.force_default_settings
(bool): If true, this will override any custom settings defined bycustom_keys
and enforce the use of default settings for optimizer parameters likebias_lr_mult
. This is particularly useful when you want to ensure that certain layers or parameters adhere strictly to the pre-defined default settings.
注解
1. If the option
dcn_offset_lr_mult
is used, the constructor will override the effect ofbias_lr_mult
in the bias of offset layer. So be careful when using bothbias_lr_mult
anddcn_offset_lr_mult
. If you wish to apply both of them to the offset layer in deformable convs, setdcn_offset_lr_mult
to the originaldcn_offset_lr_mult
*bias_lr_mult
.2. If the option
dcn_offset_lr_mult
is used, the constructor will apply it to all the DCN layers in the model. So be careful when the model contains multiple DCN layers in places other than backbone.3. When the option
force_default_settings
is true, it will override any custom settings provided incustom_keys
. This ensures that the default settings for the optimizer parameters are used.- 参数
optim_wrapper_cfg (dict) –
The config dict of the optimizer wrapper.
Required fields of
optim_wrapper_cfg
aretype
: class name of the OptimizerWrapperoptimizer
: The configuration of optimizer.
Optional fields of
optim_wrapper_cfg
areany arguments of the corresponding optimizer wrapper type, e.g., accumulative_counts, clip_grad, etc.
Required fields of
optimizer
aretype: class name of the optimizer.
Optional fields of
optimizer
areany arguments of the corresponding optimizer type, e.g., lr, weight_decay, momentum, etc.
paramwise_cfg (dict, optional) – Parameter-wise options.
- Example 1:
>>> model = torch.nn.modules.Conv1d(1, 1, 1) >>> optim_wrapper_cfg = dict( >>> dict(type='OptimWrapper', optimizer=dict(type='SGD', lr=0.01, >>> momentum=0.9, weight_decay=0.0001)) >>> paramwise_cfg = dict(norm_decay_mult=0.) >>> optim_wrapper_builder = DefaultOptimWrapperConstructor( >>> optim_wrapper_cfg, paramwise_cfg) >>> optim_wrapper = optim_wrapper_builder(model)
- Example 2:
>>> # assume model have attribute model.backbone and model.cls_head >>> optim_wrapper_cfg = dict(type='OptimWrapper', optimizer=dict( >>> type='SGD', lr=0.01, weight_decay=0.95)) >>> paramwise_cfg = dict(custom_keys={ >>> 'backbone': dict(lr_mult=0.1, decay_mult=0.9)}) >>> optim_wrapper_builder = DefaultOptimWrapperConstructor( >>> optim_wrapper_cfg, paramwise_cfg) >>> optim_wrapper = optim_wrapper_builder(model) >>> # Then the `lr` and `weight_decay` for model.backbone is >>> # (0.01 * 0.1, 0.95 * 0.9). `lr` and `weight_decay` for >>> # model.cls_head is (0.01, 0.95).
- add_params(params: List[dict], module: torch.nn.modules.module.Module, prefix: str = '', is_dcn_module: Optional[Union[int, float]] = None) → None[源代码]¶
Add all parameters of module to the params list.
The parameters of the given module will be added to the list of param groups, with specific rules defined by paramwise_cfg.
- 参数
params (list[dict]) – A list of param groups, it will be modified in place.
module (nn.Module) – The module to be added.
prefix (str) – The prefix of the module
is_dcn_module (int|float|None) – If the current module is a submodule of DCN, is_dcn_module will be passed to control conv_offset layer’s learning rate. Defaults to None.
- class mmseg.engine.optimizers.LayerDecayOptimizerConstructor(optim_wrapper_cfg, paramwise_cfg)[源代码]¶
Different learning rates are set for different layers of backbone.
Note: Currently, this optimizer constructor is built for BEiT, and it will be deprecated. Please use
LearningRateDecayOptimizerConstructor
instead.
- class mmseg.engine.optimizers.LearningRateDecayOptimizerConstructor(optim_wrapper_cfg: dict, paramwise_cfg: Optional[dict] = None)[源代码]¶
Different learning rates are set for different layers of backbone.
Note: Currently, this optimizer constructor is built for ConvNeXt, BEiT and MAE.
- add_params(params, module, **kwargs)[源代码]¶
Add all parameters of module to the params list.
The parameters of the given module will be added to the list of param groups, with specific rules defined by paramwise_cfg.
- 参数
params (list[dict]) – A list of param groups, it will be modified in place.
module (nn.Module) – The module to be added.
mmseg.evaluation¶
metrics¶
- class mmseg.evaluation.metrics.CityscapesMetric(output_dir: str, ignore_index: int = 255, format_only: bool = False, keep_results: bool = False, collect_device: str = 'cpu', prefix: Optional[str] = None, **kwargs)[源代码]¶
Cityscapes evaluation metric.
- 参数
output_dir (str) – The directory for output prediction
ignore_index (int) – Index that will be ignored in evaluation. Default: 255.
format_only (bool) – Only format result for results commit without perform evaluation. It is useful when you want to format the result to a specific format and submit it to the test server. Defaults to False.
keep_results (bool) – Whether to keep the results. When
format_only
is True,keep_results
must be True. Defaults to False.collect_device (str) – Device name used for collecting results from different ranks during distributed training. Must be ‘cpu’ or ‘gpu’. Defaults to ‘cpu’.
prefix (str, optional) – The prefix that will be added in the metric names to disambiguate homonymous metrics of different evaluators. If prefix is not provided in the argument, self.default_prefix will be used instead. Defaults to None.
- compute_metrics(results: list) → Dict[str, float][源代码]¶
Compute the metrics from processed results.
- 参数
results (list) – Testing results of the dataset.
- 返回
float]: Cityscapes evaluation results.
- 返回类型
dict[str
- process(data_batch: dict, data_samples: Sequence[dict]) → None[源代码]¶
Process one batch of data and data_samples.
The processed results should be stored in
self.results
, which will be used to computed the metrics when all batches have been processed.- 参数
data_batch (dict) – A batch of data from the dataloader.
data_samples (Sequence[dict]) – A batch of outputs from the model.
- class mmseg.evaluation.metrics.DepthMetric(depth_metrics: Optional[List[str]] = None, min_depth_eval: float = 0.0, max_depth_eval: float = inf, crop_type: Optional[str] = None, depth_scale_factor: float = 1.0, collect_device: str = 'cpu', output_dir: Optional[str] = None, format_only: bool = False, prefix: Optional[str] = None, **kwargs)[源代码]¶
Depth estimation evaluation metric.
- 参数
depth_metrics (List[str], optional) – List of metrics to compute. If not specified, defaults to all metrics in self.METRICS.
min_depth_eval (float) – Minimum depth value for evaluation. Defaults to 0.0.
max_depth_eval (float) – Maximum depth value for evaluation. Defaults to infinity.
crop_type (str, optional) – Specifies the type of cropping to be used during evaluation. This option can affect how the evaluation mask is generated. Currently, ‘nyu_crop’ is supported, but other types can be added in future. Defaults to None if no cropping should be applied.
depth_scale_factor (float) – Factor to scale the depth values. Defaults to 1.0.
collect_device (str) – Device name used for collecting results from different ranks during distributed training. Must be ‘cpu’ or ‘gpu’. Defaults to ‘cpu’.
output_dir (str) – The directory for output prediction. Defaults to None.
format_only (bool) – Only format result for results commit without perform evaluation. It is useful when you want to save the result to a specific format and submit it to the test server. Defaults to False.
prefix (str, optional) – The prefix that will be added in the metric names to disambiguate homonymous metrics of different evaluators. If prefix is not provided in the argument, self.default_prefix will be used instead. Defaults to None.
- compute_metrics(results: list) → Dict[str, float][源代码]¶
Compute the metrics from processed results.
- 参数
results (list) – The processed results of each batch.
- 返回
- The computed metrics. The keys are the names of
the metrics, and the values are corresponding results. The keys are identical with self.metrics.
- 返回类型
Dict[str, float]
- process(data_batch: dict, data_samples: Sequence[dict]) → None[源代码]¶
Process one batch of data and data_samples.
The processed results should be stored in
self.results
, which will be used to compute the metrics when all batches have been processed.- 参数
data_batch (dict) – A batch of data from the dataloader.
data_samples (Sequence[dict]) – A batch of outputs from the model.
- class mmseg.evaluation.metrics.IoUMetric(ignore_index: int = 255, iou_metrics: List[str] = ['mIoU'], nan_to_num: Optional[int] = None, beta: int = 1, collect_device: str = 'cpu', output_dir: Optional[str] = None, format_only: bool = False, prefix: Optional[str] = None, **kwargs)[源代码]¶
IoU evaluation metric.
- 参数
ignore_index (int) – Index that will be ignored in evaluation. Default: 255.
iou_metrics (list[str] | str) – Metrics to be calculated, the options includes ‘mIoU’, ‘mDice’ and ‘mFscore’.
nan_to_num (int, optional) – If specified, NaN values will be replaced by the numbers defined by the user. Default: None.
beta (int) – Determines the weight of recall in the combined score. Default: 1.
collect_device (str) – Device name used for collecting results from different ranks during distributed training. Must be ‘cpu’ or ‘gpu’. Defaults to ‘cpu’.
output_dir (str) – The directory for output prediction. Defaults to None.
format_only (bool) – Only format result for results commit without perform evaluation. It is useful when you want to save the result to a specific format and submit it to the test server. Defaults to False.
prefix (str, optional) – The prefix that will be added in the metric names to disambiguate homonymous metrics of different evaluators. If prefix is not provided in the argument, self.default_prefix will be used instead. Defaults to None.
- compute_metrics(results: list) → Dict[str, float][源代码]¶
Compute the metrics from processed results.
- 参数
results (list) – The processed results of each batch.
- 返回
- The computed metrics. The keys are the names of
the metrics, and the values are corresponding results. The key mainly includes aAcc, mIoU, mAcc, mDice, mFscore, mPrecision, mRecall.
- 返回类型
Dict[str, float]
- static intersect_and_union(pred_label: torch._VariableFunctionsClass.tensor, label: torch._VariableFunctionsClass.tensor, num_classes: int, ignore_index: int)[源代码]¶
Calculate Intersection and Union.
- 参数
pred_label (torch.tensor) – Prediction segmentation map or predict result filename. The shape is (H, W).
label (torch.tensor) – Ground truth segmentation map or label filename. The shape is (H, W).
num_classes (int) – Number of categories.
ignore_index (int) – Index that will be ignored in evaluation.
- 返回
- The intersection of prediction and ground truth
histogram on all classes.
- torch.Tensor: The union of prediction and ground truth histogram on
all classes.
torch.Tensor: The prediction histogram on all classes. torch.Tensor: The ground truth histogram on all classes.
- 返回类型
torch.Tensor
- process(data_batch: dict, data_samples: Sequence[dict]) → None[源代码]¶
Process one batch of data and data_samples.
The processed results should be stored in
self.results
, which will be used to compute the metrics when all batches have been processed.- 参数
data_batch (dict) – A batch of data from the dataloader.
data_samples (Sequence[dict]) – A batch of outputs from the model.
- static total_area_to_metrics(total_area_intersect: numpy.ndarray, total_area_union: numpy.ndarray, total_area_pred_label: numpy.ndarray, total_area_label: numpy.ndarray, metrics: List[str] = ['mIoU'], nan_to_num: Optional[int] = None, beta: int = 1)[源代码]¶
Calculate evaluation metrics :param total_area_intersect: The intersection of prediction
and ground truth histogram on all classes.
- 参数
total_area_union (np.ndarray) – The union of prediction and ground truth histogram on all classes.
total_area_pred_label (np.ndarray) – The prediction histogram on all classes.
total_area_label (np.ndarray) – The ground truth histogram on all classes.
metrics (List[str] | str) – Metrics to be evaluated, ‘mIoU’ and ‘mDice’.
nan_to_num (int, optional) – If specified, NaN values will be replaced by the numbers defined by the user. Default: None.
beta (int) – Determines the weight of recall in the combined score. Default: 1.
- 返回
- per category evaluation metrics,
shape (num_classes, ).
- 返回类型
Dict[str, np.ndarray]
mmseg.structures¶
structures¶
- class mmseg.structures.OHEMPixelSampler(context, thresh=None, min_kept=100000)[源代码]¶
Online Hard Example Mining Sampler for segmentation.
- 参数
context (nn.Module) – The context of sampler, subclass of
BaseDecodeHead
.thresh (float, optional) – The threshold for hard example selection. Below which, are prediction with low confidence. If not specified, the hard examples will be pixels of top
min_kept
loss. Default: None.min_kept (int, optional) – The minimum number of predictions to keep. Default: 100000.
- class mmseg.structures.SegDataSample(*, metainfo: Optional[dict] = None, **kwargs)[源代码]¶
A data structure interface of MMSegmentation. They are used as interfaces between different components.
The attributes in
SegDataSample
are divided into several parts:实际案例
>>> import torch >>> import numpy as np >>> from mmengine.structures import PixelData >>> from mmseg.structures import SegDataSample
>>> data_sample = SegDataSample() >>> img_meta = dict(img_shape=(4, 4, 3), ... pad_shape=(4, 4, 3)) >>> gt_segmentations = PixelData(metainfo=img_meta) >>> gt_segmentations.data = torch.randint(0, 2, (1, 4, 4)) >>> data_sample.gt_sem_seg = gt_segmentations >>> assert 'img_shape' in data_sample.gt_sem_seg.metainfo_keys() >>> data_sample.gt_sem_seg.shape (4, 4) >>> print(data_sample)
<SegDataSample(
META INFORMATION
DATA FIELDS gt_sem_seg: <PixelData(
META INFORMATION img_shape: (4, 4, 3) pad_shape: (4, 4, 3)
DATA FIELDS data: tensor([[[1, 1, 1, 0],
[1, 0, 1, 1], [1, 1, 1, 1], [0, 1, 0, 1]]])
) at 0x1c2b4156460>
) at 0x1c2aae44d60>
>>> data_sample = SegDataSample() >>> gt_sem_seg_data = dict(sem_seg=torch.rand(1, 4, 4)) >>> gt_sem_seg = PixelData(**gt_sem_seg_data) >>> data_sample.gt_sem_seg = gt_sem_seg >>> assert 'gt_sem_seg' in data_sample >>> assert 'sem_seg' in data_sample.gt_sem_seg
sampler¶
- class mmseg.structures.sampler.OHEMPixelSampler(context, thresh=None, min_kept=100000)[源代码]¶
Online Hard Example Mining Sampler for segmentation.
- 参数
context (nn.Module) – The context of sampler, subclass of
BaseDecodeHead
.thresh (float, optional) – The threshold for hard example selection. Below which, are prediction with low confidence. If not specified, the hard examples will be pixels of top
min_kept
loss. Default: None.min_kept (int, optional) – The minimum number of predictions to keep. Default: 100000.