华为开源自研AI框架昇思MindSpore应用案例：ICNet用于及时的语义分割 ...

 ICNet用于及时的语义分割
  ICNet 被广泛应用于及时的语义分割范畴。它在处理图像数据时，能够以较高的服从进行语义分割操作，为相关范畴的研究和实际应用提供了有力的支持。ICNet 的及时性使其在浩繁场景中都具有很大的上风，例如在视频处理、自动驾驶等对及时性要求较高的范畴，ICNet 能够快速准确地对图像进行语义分割，为后续的决策和处理提供关键信息。
    
   如果你对MindSpore感爱好，可以关注昇思MindSpore社区

​​​

​​​
一、环境准备

1.进入ModelArts官网

云平台资助用户快速创建和摆设模子，管理全周期AI工作流，选择下面的云平台以开始使用昇思MindSpore，获取安装下令，安装MindSpore2.0.0-alpha版本，可以在昇思教程中进入ModelArts官网

​​​
选择下方CodeLab立刻体验

​​​
等待环境搭建完成

​​​
2.使用CodeLab体验Notebook实例

选择ModelArts Upload Files上传Git文件，地址为GitHub - yfjcode/ICNet: mindspore icnet model

​​​

选择Kernel环境

​​​
切换至GPU环境，切换成第一个限时免费

​​​
进入昇思MindSpore官网，点击上方的安装

​​​
获取安装下令

​​​
回到Notebook中，在第一块代码前加入下令

​​​
  

1. conda update -n base -c defaults conda

复制代码

​​​
安装MindSpore 2.0 GPU版本
  

1. conda install mindspore=2.0.0a0 -c mindspore -c conda-forge

复制代码

​​​
安装mindvision
  

1. pip install mindvision

复制代码

​​​
安装下载download
  

1. pip install download

复制代码

​​​

二、应用体验


1.模子准备

根据原作者提示
   环境准备与数据读取 本案例基于MindSpore-CPU版本实现，在CPU上完成模子训练。
  案例实现所使用的数据:Cityscape Dataset Website
  为了下载数据集，我们起首需要在Cityscapes数据集官网进行注册，而且最好使用edu教诲邮箱进行注册，此后等待几天，就可以下载数据集了，这里我们下载了两个文件：gtFine_trainvaltest.zip和leftImg8bit_trainvaltest.zip (11GB)。
  下载完成后，我们对数据集压缩文件进行解压，文件的目录布局如下所示。
  由于我们是在CPU上跑得，本来数据集有1个多G，全部拿来跑得话，很轻易掉卡，故我们就选择一个都会的一些图片完成。
  起首要处理数据，生成对应的.mindrecord 和 .mindrecord.db文件
  需要注意的是，在生成这两个文件之前，我们要创建一个文件夹，用cityscapes_mindrecord命名，放在cityscapes文件夹的同级目录下： 而且要保持cityscapes_mindrecord文件夹内里为空
  下面是构建数据集的代码：注意，要保持cityscapes_mindrecord文件夹内里为空，报错可能是文件夹已经有文件了,文件夹地址为：/home/ma-user/work/ICNet/data/cityscapes_mindrecord
  需要删掉/data/cityscapes_mindrecord文件

​
删掉文件后，需要修改路径，删掉/home/ma-user/work/ICNet，用./替换，之后直接运行代码块即可

2. """Prepare Cityscapes dataset"""
3. import os
4. import random
5. import argparse
6. import numpy as np
7. from PIL import Image
8. from PIL import ImageOps
9. from PIL import ImageFilter
10. import mindspore.dataset as de
11. from mindspore.mindrecord import FileWriter
12. import mindspore.dataset.vision as transforms
13. import mindspore.dataset.transforms as tc
16. def _get_city_pairs(folder, split='train'):
17.     """Return two path arrays of data set img and mask"""
19.     def get_path_pairs(image_folder, masks_folder):
20.         image_paths = []
21.         masks_paths = []
22.         for root, _, files in os.walk(image_folder):
23.             for filename in files:
24.                 if filename.endswith('.png'):
25.                     imgpath = os.path.join(root, filename)
26.                     foldername = os.path.basename(os.path.dirname(imgpath))
27.                     maskname = filename.replace('leftImg8bit', 'gtFine_labelIds')
28.                     maskpath = os.path.join(masks_folder, foldername, maskname)
29.                     if os.path.isfile(imgpath) and os.path.isfile(maskpath):
30.                         image_paths.append(imgpath)
31.                         masks_paths.append(maskpath)
32.                     else:
33.                         print('cannot find the mask or image:', imgpath, maskpath)
34.         print('Found {} images in the folder {}'.format(len(image_paths), image_folder))
35.         return image_paths, masks_paths
37.     if split in ('train', 'val'):
38.         # "./Cityscapes/leftImg8bit/train" or "./Cityscapes/leftImg8bit/val"
39.         img_folder = os.path.join(folder, 'leftImg8bit/' + split)
40.         # "./Cityscapes/gtFine/train" or "./Cityscapes/gtFine/val"
41.         mask_folder = os.path.join(folder, 'gtFine/' + split)
42.         # The order of img_paths and mask_paths is one-to-one correspondence
43.         img_paths, mask_paths = get_path_pairs(img_folder, mask_folder)
44.         return img_paths, mask_paths
47. def _sync_transform(img, mask):
48.     """img and mask augmentation"""
49.     a = random.Random()
50.     a.seed(1234)
51.     base_size = 1024
52.     crop_size = 960
54.     # random mirror
55.     if random.random() < 0.5:
56.         img = img.transpose(Image.FLIP_LEFT_RIGHT)
57.         mask = mask.transpose(Image.FLIP_LEFT_RIGHT)
58.     crop_size = crop_size
59.     # random scale (short edge)
60.     short_size = random.randint(int(base_size * 0.5), int(base_size * 2.0))
61.     w, h = img.size
62.     if h > w:
63.         ow = short_size
64.         oh = int(1.0 * h * ow / w)
65.     else:
66.         oh = short_size
67.         ow = int(1.0 * w * oh / h)
68.     img = img.resize((ow, oh), Image.BILINEAR)
69.     mask = mask.resize((ow, oh), Image.NEAREST)
70.     # pad crop
71.     if short_size < crop_size:
72.         padh = crop_size - oh if oh < crop_size else 0
73.         padw = crop_size - ow if ow < crop_size else 0
74.         img = ImageOps.expand(img, border=(0, 0, padw, padh), fill=0)
75.         mask = ImageOps.expand(mask, border=(0, 0, padw, padh), fill=0)
76.     # random crop crop_size
77.     w, h = img.size
78.     x1 = random.randint(0, w - crop_size)
79.     y1 = random.randint(0, h - crop_size)
80.     img = img.crop((x1, y1, x1 + crop_size, y1 + crop_size))
81.     mask = mask.crop((x1, y1, x1 + crop_size, y1 + crop_size))
82.     # gaussian blur as in PSP
83.     if random.random() < 0.5:
84.         img = img.filter(ImageFilter.GaussianBlur(radius=random.random()))
85.     # final transform
86.     output = _img_mask_transform(img, mask)
88.     return output
91. def _class_to_index(mask):
92.     """class to index"""
93.     # Reference:
94.     # https://github.com/mcordts/cityscapesScripts/blob/master/cityscapesscripts/helpers/labels.py
95.     _key = np.array([-1, -1, -1, -1, -1, -1,
96.                      -1, -1, 0, 1, -1, -1,
97.                      2, 3, 4, -1, -1, -1,
98.                      5, -1, 6, 7, 8, 9,
99.                      10, 11, 12, 13, 14, 15,
100.                      -1, -1, 16, 17, 18])
101.     # [-1, ..., 33]
102.     _mapping = np.array(range(-1, len(_key) - 1)).astype('int32')
104.     # assert the value
105.     values = np.unique(mask)
106.     for value in values:
107.         assert value in _mapping
108.     # Get the index of each pixel value in the mask corresponding to _mapping
109.     index = np.digitize(mask.ravel(), _mapping, right=True)
110.     # According to the above index, according to _key, get the corresponding
111.     return _key[index].reshape(mask.shape)
114. def _img_transform(img):
115.     return np.array(img)
118. def _mask_transform(mask):
119.     target = _class_to_index(np.array(mask).astype('int32'))
120.     return np.array(target).astype('int32')
123. def _img_mask_transform(img, mask):
124.     """img and mask transform"""
125.     input_transform = tc.Compose([
126.         transforms.ToTensor(),
127.         transforms.Normalize((0.485, 0.456, 0.406), (0.229, 0.224, 0.225), is_hwc=False)])
128.     img = _img_transform(img)
129.     mask = _mask_transform(mask)
130.     img = input_transform(img)
132.     img = np.array(img).astype(np.float32)
133.     mask = np.array(mask).astype(np.float32)
135.     return (img, mask)
138. def data_to_mindrecord_img(prefix='cityscapes-2975.mindrecord', file_num=1,
139.                            root='./', split='train', mindrecord_dir="./"):
140.     """to mindrecord"""
141.     mindrecord_path = os.path.join(mindrecord_dir, prefix)
143.     writter = FileWriter(mindrecord_path, file_num)
145.     img_paths, mask_paths = _get_city_pairs(root, split)
147.     cityscapes_json = {
148.         "images": {"type": "int32", "shape": [1024, 2048, 3]},
149.         "mask": {"type": "int32", "shape": [1024, 2048]},
150.     }
152.     writter.add_schema(cityscapes_json, "cityscapes_json")
154.     images_files_num = len(img_paths)
155.     for index in range(images_files_num):
156.         img = Image.open(img_paths[index]).convert('RGB')
157.         img = np.array(img, dtype=np.int32)
159.         mask = Image.open(mask_paths[index])
160.         mask = np.array(mask, dtype=np.int32)
162.         row = {"images": img, "mask": mask}
163.         # print("images",img, "mask", mask)
164.         # print("images_files_num，index， img_paths[index],mask_paths[index]",images_files_num,index,img_paths[index],mask_paths[index])
165.         if (index + 1) % 10 == 0:
166.             print("writing {}/{} into mindrecord".format(index + 1, images_files_num))
167.         writter.write_raw_data([row])
168.     writter.commit()
171. def get_Image_crop_nor(img, mask):
172.     image = np.uint8(img)
173.     mask = np.uint8(mask)
174.     image = Image.fromarray(image)
175.     mask = Image.fromarray(mask)
177.     output = _sync_transform(image, mask)
179.     return output
182. def create_icnet_dataset(mindrecord_file, batch_size=16, device_num=1, rank_id=0):
183.     """create dataset for training"""
184.     a = random.Random()
185.     a.seed(1234)
186.     ds = de.MindDataset(mindrecord_file, columns_list=["images", "mask"],
187.                         num_shards=device_num, shard_id=rank_id, shuffle=True)
188.     ds = ds.map(operations=get_Image_crop_nor, input_columns=["images", "mask"], output_columns=["image", "masks"])
190.     ds = ds.batch(batch_size=batch_size, drop_remainder=False)
192.     return ds
195. dataset_path="./data/cityscapes/"
196. mindrecord_path="./data/cityscapes_mindrecord/"
198. data_to_mindrecord_img(root=dataset_path, mindrecord_dir=mindrecord_path)
199. # if __name__ == '__main__':
200. #     parser = argparse.ArgumentParser(description="dataset_to_mindrecord")
201. #     parser.add_argument("--dataset_path", type=str, default="/home/ma-user/work/ICNet/data/cityscapes/", help="dataset path")
202. #     parser.add_argument("--mindrecord_path", type=str, default="/home/ma-user/work/ICNet/data/cityscapes_mindrecord/",
203. #                         help="mindrecord_path")
205. #     args_opt = parser.parse_args()
206. #     data_to_mindrecord_img(root=args_opt.dataset_path, mindrecord_dir=args_opt.mindrecord_path)

复制代码

可以看到已经生成的对应的数据集文件，然后我们创建稍后用到的数据

注意修改路径

1. prefix = 'cityscapes-2975.mindrecord'
2. train_mindrecord_dir="/home/ma-user/work/ICNet/data/cityscapes_mindrecord"
3. train_train_batch_size_percard=4
4. device_num=1
5. rank_id=0
7. mindrecord_dir = train_mindrecord_dir
8. mindrecord_file = os.path.join(mindrecord_dir, prefix)
9. print("mindrecord_file",mindrecord_file)
10. # print("cfg['train'][‘’train_batch_size_percard‘]",cfg['train']["train_batch_size_percard"])
12. dataset = create_icnet_dataset(mindrecord_file, batch_size=train_train_batch_size_percard,
13.                                    device_num=device_num, rank_id=rank_id)
14. print(dataset)

复制代码

​
2.模子构建

创建需要训练模子的一些参数：(这里只是展示，不运行，具体参数运行在背面)

   1.Model
  model: name: "icnet" backbone: "resnet50v1" base_size: 1024 # during augmentation, shorter size will be resized between [base_size0.5, base_size2.0] crop_size: 960 # end of augmentation, crop to training
  2.Optimizer
  optimizer: init_lr: 0.02 momentum: 0.9 weight_decay: 0.0001
  3.Training
  train: train_batch_size_percard: 4 valid_batch_size: 1 cityscapes_root: "/data/cityscapes/" epochs: 10 val_epoch: 1 # run validation every val-epoch ckpt_dir: "./ckpt/" # ckpt and training log will be saved here mindrecord_dir: '/home/ma-user/work/ICNet/data/cityscapes_mindrecord' pretrained_model_path: '/home/ma-user/work/ICNet/root/cacheckpt/resnet50-icnet-150_2.ckpt' save_checkpoint_epochs: 5 keep_checkpoint_max: 10
  4.Valid
  test: ckpt_path: "" # set the pretrained model path correctly
  ​注意修改路径

1. train_epochs=10
2. train_data_size = dataset.get_dataset_size()
3. print("data_size", train_data_size)
4. epoch = train_epochs
5. project_path="/home/ma-user/work/ICNet/"
6. train_pretrained_model_path="/home/ma-user/work/ICNet/root/cacheckpt/resnet50-icnet-150_2.ckpt"

复制代码

​

1. import mindspore as ms
2. import mindspore.nn as nn
3. import mindspore.ops as ops
4. from src.loss import ICNetLoss
5. from src.models.resnet50_v1 import get_resnet50v1b
7. __all__ = ['ICNetdc']
9. class ICNetdc(nn.Cell):
10.     """Image Cascade Network"""
12.     def __init__(self, nclass=19, pretrained_path="", istraining=True, norm_layer=nn.SyncBatchNorm):
13.         super(ICNetdc, self).__init__()
14.         self.conv_sub1 = nn.SequentialCell(
15.             _ConvBNReLU(3, 32, 3, 2, norm_layer=norm_layer),
16.             _ConvBNReLU(32, 32, 3, 2, norm_layer=norm_layer),
17.             _ConvBNReLU(32, 64, 3, 2, norm_layer=norm_layer)
18.         )
19.         self.istraining = istraining
20.         self.ppm = PyramidPoolingModule()
22.         self.backbone = SegBaseModel(root=pretrained_path, istraining=istraining)
24.         self.head = _ICHead(nclass, norm_layer=norm_layer)
26.         self.loss = ICNetLoss()
28.         self.resize_bilinear = nn.ResizeBilinear()
30.         self.__setattr__('exclusive', ['conv_sub1', 'head'])
32.     def construct(self, x, y):
33.         """ICNet_construct"""
34.         if x.shape[0] != 1:
35.             x = x.squeeze()
36.         # sub 1
37.         x_sub1 = self.conv_sub1(x)
39.         h, w = x.shape[2:]
40.         # sub 2
41.         x_sub2 = self.resize_bilinear(x, size=(h / 2, w / 2))
42.         _, x_sub2, _, _ = self.backbone(x_sub2)
44.         # sub 4
45.         _, _, _, x_sub4 = self.backbone(x)
46.         # add PyramidPoolingModule
47.         x_sub4 = self.ppm(x_sub4)
49.         output = self.head(x_sub1, x_sub2, x_sub4)
51.         if self.istraining:
52.             outputs = self.loss(output, y)
53.         else:
54.             outputs = output
55.         return outputs
57. class PyramidPoolingModule(nn.Cell):
58.     """PPM"""
60.     def __init__(self, pyramids=None):
61.         super(PyramidPoolingModule, self).__init__()
62.         self.avgpool = ops.ReduceMean(keep_dims=True)
63.         self.pool2 = nn.AvgPool2d(kernel_size=15, stride=15)
64.         self.pool3 = nn.AvgPool2d(kernel_size=10, stride=10)
65.         self.pool6 = nn.AvgPool2d(kernel_size=5, stride=5)
66.         self.resize_bilinear = nn.ResizeBilinear()
68.     def construct(self, x):
69.         """ppm_construct"""
70.         feat = x
71.         height, width = x.shape[2:]
73.         x1 = self.avgpool(x, (2, 3))
74.         x1 = self.resize_bilinear(x1, size=(height, width), align_corners=True)
75.         feat = feat + x1
77.         x2 = self.pool2(x)
78.         x2 = self.resize_bilinear(x2, size=(height, width), align_corners=True)
79.         feat = feat + x2
81.         x3 = self.pool3(x)
82.         x3 = self.resize_bilinear(x3, size=(height, width), align_corners=True)
83.         feat = feat + x3
85.         x6 = self.pool6(x)
86.         x6 = self.resize_bilinear(x6, size=(height, width), align_corners=True)
87.         feat = feat + x6
89.         return feat
92. class _ICHead(nn.Cell):
93.     """Head"""
95.     def __init__(self, nclass, norm_layer=nn.SyncBatchNorm, **kwargs):
96.         super(_ICHead, self).__init__()
97.         self.cff_12 = CascadeFeatureFusion12(128, 64, 128, nclass, norm_layer, **kwargs)
98.         self.cff_24 = CascadeFeatureFusion24(2048, 512, 128, nclass, norm_layer, **kwargs)
100.         self.conv_cls = nn.Conv2d(128, nclass, 1, has_bias=False)
101.         self.outputs = list()
102.         self.resize_bilinear = nn.ResizeBilinear()
104.     def construct(self, x_sub1, x_sub2, x_sub4):
105.         """Head_construct"""
106.         outputs = self.outputs
107.         x_cff_24, x_24_cls = self.cff_24(x_sub4, x_sub2)
109.         x_cff_12, x_12_cls = self.cff_12(x_cff_24, x_sub1)
111.         h1, w1 = x_cff_12.shape[2:]
112.         up_x2 = self.resize_bilinear(x_cff_12, size=(h1 * 2, w1 * 2),
113.                                      align_corners=True)
114.         up_x2 = self.conv_cls(up_x2)
115.         h2, w2 = up_x2.shape[2:]
117.         up_x8 = self.resize_bilinear(up_x2, size=(h2 * 4, w2 * 4),
118.                                      align_corners=True)  # scale_factor=4,
119.         outputs.append(up_x8)
120.         outputs.append(up_x2)
121.         outputs.append(x_12_cls)
122.         outputs.append(x_24_cls)
124.         return outputs
127. class _ConvBNReLU(nn.Cell):
128.     """ConvBNRelu"""
130.     def __init__(self, in_channels, out_channels, kernel_size=3, stride=2, padding=1, dilation=1,
131.                  groups=1, norm_layer=nn.SyncBatchNorm, bias=False, **kwargs):
132.         super(_ConvBNReLU, self).__init__()
133.         self.conv = nn.Conv2d(in_channels, out_channels, kernel_size, stride, pad_mode='pad', padding=padding,
134.                               dilation=dilation,
135.                               group=1, has_bias=False)
136.         self.bn = norm_layer(out_channels, momentum=0.1)
137.         self.relu = nn.ReLU()
139.     def construct(self, x):
140.         x = self.conv(x)
141.         x = self.bn(x)
142.         x = self.relu(x)
143.         return x
146. class CascadeFeatureFusion12(nn.Cell):
147.     """CFF Unit"""
149.     def __init__(self, low_channels, high_channels, out_channels, nclass, norm_layer=nn.SyncBatchNorm, **kwargs):
150.         super(CascadeFeatureFusion12, self).__init__()
151.         self.conv_low = nn.SequentialCell(
152.             nn.Conv2d(low_channels, out_channels, 3, pad_mode='pad', padding=2, dilation=2, has_bias=False),
153.             norm_layer(out_channels, momentum=0.1)
154.         )
155.         self.conv_high = nn.SequentialCell(
156.             nn.Conv2d(high_channels, out_channels, kernel_size=1, has_bias=False),
157.             norm_layer(out_channels, momentum=0.1)
158.         )
159.         self.conv_low_cls = nn.Conv2d(in_channels=out_channels, out_channels=nclass, kernel_size=1, has_bias=False)
160.         self.resize_bilinear = nn.ResizeBilinear()
162.         self.scalar_cast = ops.ScalarCast()
164.         self.relu = ms.nn.ReLU()
166.     def construct(self, x_low, x_high):
167.         """cff_construct"""
168.         h, w = x_high.shape[2:]
169.         x_low = self.resize_bilinear(x_low, size=(h, w), align_corners=True)
170.         x_low = self.conv_low(x_low)
172.         x_high = self.conv_high(x_high)
173.         x = x_low + x_high
175.         x = self.relu(x)
176.         x_low_cls = self.conv_low_cls(x_low)
178.         return x, x_low_cls
181. class CascadeFeatureFusion24(nn.Cell):
182.     """CFF Unit"""
184.     def __init__(self, low_channels, high_channels, out_channels, nclass, norm_layer=nn.SyncBatchNorm, **kwargs):
185.         super(CascadeFeatureFusion24, self).__init__()
186.         self.conv_low = nn.SequentialCell(
187.             nn.Conv2d(low_channels, out_channels, 3, pad_mode='pad', padding=2, dilation=2, has_bias=False),
188.             norm_layer(out_channels, momentum=0.1)
189.         )
190.         self.conv_high = nn.SequentialCell(
191.             nn.Conv2d(high_channels, out_channels, kernel_size=1, has_bias=False),
192.             norm_layer(out_channels, momentum=0.1)
193.         )
194.         self.conv_low_cls = nn.Conv2d(in_channels=out_channels, out_channels=nclass, kernel_size=1, has_bias=False)
196.         self.resize_bilinear = nn.ResizeBilinear()
197.         self.relu = ms.nn.ReLU()
199.     def construct(self, x_low, x_high):
200.         """ccf_construct"""
201.         h, w = x_high.shape[2:]
203.         x_low = self.resize_bilinear(x_low, size=(h, w), align_corners=True)
204.         x_low = self.conv_low(x_low)
206.         x_high = self.conv_high(x_high)
207.         x = x_low + x_high
209.         x = self.relu(x)
210.         x_low_cls = self.conv_low_cls(x_low)
212.         return x, x_low_cls
215. class SegBaseModel(nn.Cell):
216.     """Base Model for Semantic Segmentation"""
218.     def __init__(self, nclass=19, backbone='resnet50', root="", istraining=False):
219.         super(SegBaseModel, self).__init__()
220.         self.nclass = nclass
221.         if backbone == 'resnet50':
222.             self.pretrained = get_resnet50v1b(ckpt_root=root, istraining=istraining)
224.     def construct(self, x):
225.         """forwarding pre-trained network"""
226.         x = self.pretrained.conv1(x)
227.         x = self.pretrained.bn1(x)
228.         x = self.pretrained.relu(x)
229.         x = self.pretrained.maxpool(x)
230.         c1 = self.pretrained.layer1(x)
231.         c2 = self.pretrained.layer2(c1)
232.         c3 = self.pretrained.layer3(c2)
233.         c4 = self.pretrained.layer4(c3)
235.         return c1, c2, c3, c4

复制代码

1. def poly_lr(base_lr, decay_steps, total_steps, end_lr=0.0001, power=0.9):
2.     for i in range(total_steps):
3.         step_ = min(i, decay_steps)
4.         yield (base_lr - end_lr) * ((1.0 - step_ / decay_steps) ** power) + end_lr

复制代码

1. optimizer_init_lr=0.02
2. optimizer_weight_decay = 0.0001
3. optimizer_momentum= 0.9
4. train_save_checkpoint_epochs=5
5. train_keep_checkpoint_max = 10
6. rank_id = 0
7. device_id = 0
8. device_num =1
10. # from src.lr_scheduler import poly_lr
11. import os
12. import sys
13. import logging
14. import argparse
15. # import yaml
16. import mindspore.nn as nn
17. from mindspore import Model
18. from mindspore import context
19. from mindspore import set_seed
20. from mindspore.context import ParallelMode
21. from mindspore.communication import init
22. from mindspore.train.callback import CheckpointConfig
23. from mindspore.train.callback import ModelCheckpoint
24. from mindspore.train.callback import LossMonitor
25. from mindspore.train.callback import TimeMonitor
27. iters_per_epoch = train_data_size
28. total_train_steps = iters_per_epoch * epoch
29. base_lr = optimizer_init_lr
30. iter_lr = poly_lr(base_lr, total_train_steps, total_train_steps, end_lr=0.0, power=0.9)
32. network = ICNetdc(pretrained_path=train_pretrained_model_path, norm_layer=nn.BatchNorm2d)
34. optim = nn.SGD(params=network.trainable_params(), learning_rate=iter_lr, momentum=optimizer_momentum,
35.                    weight_decay=optimizer_weight_decay)
37. model = Model(network, optimizer=optim, metrics=None)
39. config_ck_train = CheckpointConfig(save_checkpoint_steps=iters_per_epoch * train_save_checkpoint_epochs,
40.                                        keep_checkpoint_max=train_keep_checkpoint_max)
41. ckpoint_cb_train = ModelCheckpoint(prefix='ICNet', directory=project_path + 'ckpt' + str(device_id),
42.                                        config=config_ck_train)
43. time_cb_train = TimeMonitor(data_size=dataset.get_dataset_size())
44. loss_cb_train = LossMonitor()
45. print("train begins------------------------------")
46. model.train(epoch=epoch, train_dataset=dataset, callbacks=[ckpoint_cb_train, loss_cb_train, time_cb_train],
47.                 dataset_sink_mode=True)

复制代码

3.模子验证

1. import os
2. import time
3. import sys
4. import argparse
5. import yaml
6. import numpy as np
7. from PIL import Image
8. import mindspore.ops as ops
9. from mindspore import load_param_into_net
10. from mindspore import load_checkpoint
11. from mindspore import Tensor
12. import mindspore.dataset.vision as vision
13. from src.models import ICNet
14. from src.metric import SegmentationMetric
15. from src.logger import SetupLogger
18. class Evaluator:
19.     """evaluate"""
21.     def __init__(self):
22.         # self.cfg = config
24.         # get valid dataset images and targets
25.         self.image_paths, self.mask_paths = _get_city_pairs(dataset_path, "val")
26.         # self.image_paths,
27.         # self.mask_paths
28.         
29.         # create network
30.         # self.model = ICNetdc(nclass=19, pretrained_path=train_pretrained_model_path, norm_layer=nn.BatchNorm2d,istraining=False)
31.         self.model = ICNet(nclass=19, pretrained_path=train_pretrained_model_path, istraining=False)
33.         # load ckpt
34.         checkpoint_path="/home/ma-user/work/ICNet/ckpt0/ICNet-10_1.ckpt"
35.         ckpt_file_name = checkpoint_path
36.         param_dict = load_checkpoint(ckpt_file_name)
37.         load_param_into_net(self.model, param_dict)
39.         # evaluation metrics
40.         self.metric = SegmentationMetric(19)
42.     def eval(self):
43.         """evaluate"""
44.         self.metric.reset()
45.         model = self.model
46.         model = model.set_train(False)
48.         logger.info("Start validation, Total sample: {:d}".format(len(self.image_paths)))
49.         list_time = []
51.         for i in range(len(self.image_paths)):
52.             image = Image.open(self.image_paths[i]).convert('RGB')  # image shape: (W,H,3)
53.             mask = Image.open(self.mask_paths[i])  # mask shape: (W,H)
55.             image = self._img_transform(image)  # image shape: (3,H,W) [0,1]
56.             mask = self._mask_transform(mask)  # mask shape: (H,w)
58.             image = Tensor(image)
60.             expand_dims = ops.ExpandDims()
61.             image = expand_dims(image, 0)
63.             start_time = time.time()
64.             output = model(image)
65.             end_time = time.time()
66.             step_time = end_time - start_time
68.             output = output.asnumpy()
69.             mask = np.expand_dims(mask.asnumpy(), axis=0)
70.             self.metric.update(output, mask)
71.             list_time.append(step_time)
73.         mIoU, pixAcc = self.metric.get()
75.         average_time = sum(list_time) / len(list_time)
77.         print("avgmiou", mIoU)
78.         print("avg_pixacc", pixAcc)
79.         print("avgtime", average_time)
81.     def _img_transform(self, image):
82.         """img_transform"""
83.         to_tensor = vision.ToTensor()
84.         normalize = vision.Normalize([.485, .456, .406], [.229, .224, .225], is_hwc=False)
85.         image = to_tensor(image)
86.         image = normalize(image)
87.         return image
89.     def _mask_transform(self, mask):
90.         mask = self._class_to_index(np.array(mask).astype('int32'))
91.         return Tensor(np.array(mask).astype('int32'))  # torch.LongTensor
93.     def _class_to_index(self, mask):
94.         """assert the value"""
95.         values = np.unique(mask)
96.         self._key = np.array([-1, -1, -1, -1, -1, -1,
97.                               -1, -1, 0, 1, -1, -1,
98.                               2, 3, 4, -1, -1, -1,
99.                               5, -1, 6, 7, 8, 9,
100.                               10, 11, 12, 13, 14, 15,
101.                               -1, -1, 16, 17, 18])
102.         self._mapping = np.array(range(-1, len(self._key) - 1)).astype('int32')
103.         for value in values:
104.             assert value in self._mapping
105.         # Get the index of each pixel value in the mask corresponding to _mapping
106.         index = np.digitize(mask.ravel(), self._mapping, right=True)
107.         # According to the above index index, according to _key, the corresponding mask image is obtained
108.         return self._key[index].reshape(mask.shape)
111. def _get_city_pairs(folder, split='train'):
112.     """get dataset img_mask_path_pairs"""
114.     def get_path_pairs(image_folder, mask_folder):
115.         img_paths = []
116.         mask_paths = []
117.         for root, _, files in os.walk(image_folder):
118.             for filename in files:
119.                 if filename.endswith('.png'):
120.                     imgpath = os.path.join(root, filename)
121.                     foldername = os.path.basename(os.path.dirname(imgpath))
122.                     maskname = filename.replace('leftImg8bit', 'gtFine_labelIds')
123.                     maskpath = os.path.join(mask_folder, foldername, maskname)
124.                     if os.path.isfile(imgpath) and os.path.isfile(maskpath):
125.                         img_paths.append(imgpath)
126.                         mask_paths.append(maskpath)
127.                     else:
128.                         print('cannot find the mask or image:', imgpath, maskpath)
129.         print('Found {} images in the folder {}'.format(len(img_paths), image_folder))
130.         return img_paths, mask_paths
132.     if split in ('train', 'val', 'test'):
133.         # "./Cityscapes/leftImg8bit/train" or "./Cityscapes/leftImg8bit/val"
134.         img_folder = os.path.join(folder, 'leftImg8bit/' + split)
135.         # "./Cityscapes/gtFine/train" or "./Cityscapes/gtFine/val"
136.         mask_folder = os.path.join(folder, 'gtFine/' + split)
138.         img_paths, mask_paths = get_path_pairs(img_folder, mask_folder)
139.         return img_paths, mask_paths

复制代码

1. train_ckpt_dir="./ckpt/"
2. model_name="icnet"
3. model_backbone="resnet50v1"
4. checkpoint_path="./ckpt0/ICNet-10_1.ckpt"
6. logger = SetupLogger(name="semantic_segmentation",
7.                          save_dir=train_ckpt_dir,
8.                          distributed_rank=0,
9.                          filename='{}_{}_evaluate_log.txt'.format(model_name,model_backbone))
11. evaluator = Evaluator()
12. evaluator.eval()

复制代码

最后根据路径的图片获取语义分割文本

免责声明：如果侵犯了您的权益，请联系站长，我们会及时删除侵权内容，谢谢合作！更多信息从访问主页：qidao123.com:ToB企服之家，中国第一个企服评测及商务社交产业平台。