各位哥哥姐姐弟弟妹妹各人好,我是干饭王刘姐,主业干饭,主业2.0计算机研究生在读。
和我一起来改进yolov8变身计算机大牛吧!
本文中的论文笔记都是刘姐亲身整理,原创整理哦~
一、FasterNet简介
论文地址
https://export.arxiv.org/pdf/2303.03667v1.pdf
代码地址
https://github.com/JierunChen/FasterNet
论文内容(原创整理)
前述
- 提出了一种新的部门卷积(PConv),更有效地提取空间特征,同时减少冗余计算和内存访问。
- MobileNets,ShuffleNets 和GhostNet 等利用 dependency卷积(DWConv)和/或组卷积(GConv)来提取空间特征。然而,在努力减少FLOP的过程中,操作员常常遭受增长的存储器访问的副作用。MicroNet 进一步分解和稀疏化网络,将其FLOP推到极低的程度。
- 延迟Latency = FLOPs/FLOPS, FLOPS是每秒浮点运算的缩写,作为有效计算速率的度量。
- 本文旨在通过开辟一种简朴而快速有效的运算符来消除这种差异,该运算符可以在降低FLOPs的环境下保持高FLOPS。
- 从本质上讲,PConv具有比通例Conv更低的FLOPS,而具有比DWConv/GConv更高的FLOPS。换句话说,PConv更好地利用了设备上的计算能力。
- 基于DWConv
贡献
- 指出了实现更高FLOPS的重要性,而不但仅是为了更快的神经网络而减少FLOPS。·
- 引入了一个简朴而快速有效的算子PConv,它很有可能取代现有的首选项DWConv。·
- 推出FasterNet,它可以在GPU,CPU和ARM处置惩罚器等各种设备上顺利运行,速率普遍很快。·
- 对各种任务举行了广泛的实验,并验证了PConv和FasterNet的高速和有效性。
具体
- 它只是在一部门输入通道上应用通例Conv举行空间特征提取,其余通道保持不变。
- PConv的FLOPs为:hwkkcp*cp。
- 内存访问量:hw2cp+kkcpcp。
- PConv后接PWConv(逐点卷积)
- FasterNet,这是一种新的神经网络家族,运行速率快,对许多视觉任务非常有效。
显著效果
时间进步巨大50%+,GFLOPs减少60%+
改进过程
FasterNet核心代码(添加到ultralytics/nn/modules/block.py):
- # --------------------------FasterNet----------------------------
- from timm.models.layers import DropPath
- class Partial_conv3(nn.Module):
- def __init__(self, dim, n_div, forward):
- super().__init__()
- self.dim_conv3 = dim // n_div
- self.dim_untouched = dim - self.dim_conv3
- self.partial_conv3 = nn.Conv2d(self.dim_conv3, self.dim_conv3, 3, 1, 1, bias=False)
- if forward == 'slicing':
- self.forward = self.forward_slicing
- elif forward == 'split_cat':
- self.forward = self.forward_split_cat
- else:
- raise NotImplementedError
- def forward_slicing(self, x):
- # only for inference
- x = x.clone() # !!! Keep the original input intact for the residual connection later
- x[:, :self.dim_conv3, :, :] = self.partial_conv3(x[:, :self.dim_conv3, :, :])
- return x
- def forward_split_cat(self, x):
- # for training/inference
- x1, x2 = torch.split(x, [self.dim_conv3, self.dim_untouched], dim=1)
- x1 = self.partial_conv3(x1)
- x = torch.cat((x1, x2), 1)
- return x
- class MLPBlock(nn.Module):
- def __init__(self,
- dim,
- n_div,
- mlp_ratio,
- drop_path,
- layer_scale_init_value,
- act_layer,
- norm_layer,
- pconv_fw_type
- ):
- super().__init__()
- self.dim = dim
- self.mlp_ratio = mlp_ratio
- self.drop_path = DropPath(drop_path) if drop_path > 0. else nn.Identity()
- self.n_div = n_div
- mlp_hidden_dim = int(dim * mlp_ratio)
- mlp_layer = [
- nn.Conv2d(dim, mlp_hidden_dim, 1, bias=False),
- norm_layer(mlp_hidden_dim),
- act_layer(),
- nn.Conv2d(mlp_hidden_dim, dim, 1, bias=False)
- ]
- self.mlp = nn.Sequential(*mlp_layer)
- self.spatial_mixing = Partial_conv3(
- dim,
- n_div,
- pconv_fw_type
- )
- if layer_scale_init_value > 0:
- self.layer_scale = nn.Parameter(layer_scale_init_value * torch.ones((dim)), requires_grad=True)
- self.forward = self.forward_layer_scale
- else:
- self.forward = self.forward
- def forward(self, x):
- shortcut = x
- x = self.spatial_mixing(x)
- x = shortcut + self.drop_path(self.mlp(x))
- return x
- def forward_layer_scale(self, x):
- shortcut = x
- x = self.spatial_mixing(x)
- x = shortcut + self.drop_path(
- self.layer_scale.unsqueeze(-1).unsqueeze(-1) * self.mlp(x))
- return x
- class BasicStage(nn.Module):
- def __init__(self,
- dim,
- depth=1,
- n_div=4,
- mlp_ratio=2,
- layer_scale_init_value=0,
- norm_layer=nn.BatchNorm2d,
- act_layer=nn.ReLU,
- pconv_fw_type='split_cat'
- ):
- super().__init__()
- dpr = [x.item()
- for x in torch.linspace(0, 0.0, sum([1, 2, 8, 2]))]
- blocks_list = [
- MLPBlock(
- dim=dim,
- n_div=n_div,
- mlp_ratio=mlp_ratio,
- drop_path=dpr[i],
- layer_scale_init_value=layer_scale_init_value,
- norm_layer=norm_layer,
- act_layer=act_layer,
- pconv_fw_type=pconv_fw_type
- )
- for i in range(depth)
- ]
- self.blocks = nn.Sequential(*blocks_list)
- def forward(self, x):
- x = self.blocks(x)
- return x
- class PatchEmbed_FasterNet(nn.Module):
- def __init__(self, in_chans, embed_dim, patch_size, patch_stride, norm_layer=nn.BatchNorm2d):
- super().__init__()
- self.proj = nn.Conv2d(in_chans, embed_dim, kernel_size=patch_size, stride=patch_stride, bias=False)
- if norm_layer is not None:
- self.norm = norm_layer(embed_dim)
- else:
- self.norm = nn.Identity()
- def forward(self, x):
- x = self.norm(self.proj(x))
- return x
- def fuseforward(self, x):
- x = self.proj(x)
- return x
- class PatchMerging_FasterNet(nn.Module):
- def __init__(self, dim, out_dim, k, patch_stride2, norm_layer=nn.BatchNorm2d):
- super().__init__()
- self.reduction = nn.Conv2d(dim, out_dim, kernel_size=k, stride=patch_stride2, bias=False)
- if norm_layer is not None:
- self.norm = norm_layer(out_dim)
- else:
- self.norm = nn.Identity()
- def forward(self, x):
- x = self.norm(self.reduction(x))
- return x
- def fuseforward(self, x):
- x = self.reduction(x)
- return x
复制代码 并在ultralytics/nn/modules/block.py中最上方“all”中引用‘BasicStage’, ‘PatchEmbed_FasterNet’, ‘PatchMerging_FasterNet’
在ultralytics/nn/modules/init.py中
- from .block import (....,BasicStage,PatchEmbed_FasterNet,PatchMerging_FasterNet)
复制代码 在 ultralytics/nn/tasks.py 上方
- from ultralytics.nn.modules import (....BasicStage, PatchEmbed_FasterNet, PatchMerging_FasterNet)
复制代码 在parse_model解析函数中添加如下代码:
- if m in (... BasicStage, PatchEmbed_FasterNet, PatchMerging_FasterNet):
复制代码 和
- elif m in [BasicStage]:
- args.pop(1)
复制代码 在 ultralytics/nn/tasks.py 中的self.model.modules()后面添加
- if type(m) is PatchEmbed_FasterNet:
- m.proj = fuse_conv_and_bn(m.proj, m.norm)
- delattr(m, 'norm') # remove BN
- m.forward = m.fuseforward
- if type(m) is PatchMerging_FasterNet:
- m.reduction = fuse_conv_and_bn(m.reduction, m.norm)
- delattr(m, 'norm') # remove BN
- m.forward = m.fuseforward
复制代码 在ultralytics/cfg/models/v8文件夹下新建yolov8-FasterNet.yaml文件
[code]# Ultralytics YOLO |