【TVM教程】为 NVIDIA GPU 主动调度神经网络

Apache TVM 是一个深度的深度学习编译框架，适用于 CPU、GPU 和各种机器学习加快芯片。更多 TVM 中文文档可访问 →https://tvm.hyper.ai/
作者：Lianmin Zheng
针对特定装备和工作负载的主动调优对于获得最佳性能至关重要。本文介绍如何使用 auto-scheduler 为 NVIDIA GPU 调优整个神经网络。
为主动调优神经网络，需要将网络划分为小的子图并独立调优。每个子图被视为一个搜刮任务，任务调度器对时间举行切片并动态地为这些任务分配时间资源，并预测每个任务对端到端执行时间的影响，优先思量最能淘汰执行时间的任务。
对于每个子图，使用 tvm/python/topi 中的盘算声明来获取张量表达式情势的盘算 DAG。然后用 auto-scheduler 来构建这个 DAG 的搜刮空间，并搜刮合适的调度（低级优化）。
与基于 template 的 AutoTVM（依赖手动 template 来定义搜刮空间的） 不同，auto-scheduler 无需任何调度 template。换言之，auto-scheduler 只使用 tvm/python/topi 中的盘算声明，不使用现有的调度 template。
注意，本教程无法在 Windows 或最新版本的 macOS 上运行。如需运行，请将本教程的主体放在 if __name__ == "__main__": 代码块中。

1. import numpy as np
3. import tvm
4. from tvm import relay, auto_scheduler
5. import tvm.relay.testing
6. from tvm.contrib import graph_executor

复制代码

定义网络​

起首，要用 Relay 前端 API 定义网络。可以从 tvm.relay.testing 加载一些预定义的网络。也可以从 MXNet、ONNX、PyTorch 和 TensorFlow 加载模子（参见 前端教程）。
对于卷积神经网络，尽管 auto-scheduler 可以在任何布局下正常运行，但通过 NHWC 布局实现的性能最佳。auto-scheduler 对 NHWC 布局举行了很多优化，因此推荐将模子转换为 NHWC 布局，从而得以使用 auto-scheduler。可用 ConvertLayout pass 在 TVM 中举行布局转换。

1. def get_network(name, batch_size, layout="NHWC", dtype="float32"):
2. """Get the symbol definition and random weight of a network"""
4. # auto-scheduler 更适合 NHWC 布局
5. if layout == "NHWC":
6.         image_shape = (224, 224, 3)
7. elif layout == "NCHW":
8.         image_shape = (3, 224, 224)
9. else:
10. raise ValueError("Invalid layout: " + layout)
12.     input_shape = (batch_size,) + image_shape
13.     output_shape = (batch_size, 1000)
15. if name.startswith("resnet-"):
16.         n_layer = int(name.split("-")[1])
17.         mod, params = relay.testing.resnet.get_workload(
18.             num_layers=n_layer,
19.             batch_size=batch_size,
20.             layout=layout,
21.             dtype=dtype,
22.             image_shape=image_shape,
23. )
24. elif name.startswith("resnet3d-"):
25.         n_layer = int(name.split("-")[1])
26.         mod, params = relay.testing.resnet.get_workload(
27.             num_layers=n_layer,
28.             batch_size=batch_size,
29.             layout=layout,
30.             dtype=dtype,
31.             image_shape=image_shape,
32. )
33. elif name == "mobilenet":
34.         mod, params = relay.testing.mobilenet.get_workload(
35.             batch_size=batch_size, layout=layout, dtype=dtype, image_shape=image_shape
36. )
37. elif name == "squeezenet_v1.1":
38. assert layout == "NCHW", "squeezenet_v1.1 only supports NCHW layout"
39.         mod, params = relay.testing.squeezenet.get_workload(
40.             version="1.1",
41.             batch_size=batch_size,
42.             dtype=dtype,
43.             image_shape=image_shape,
44. )
45. elif name == "inception_v3":
46.         input_shape = (batch_size, 3, 299, 299) if layout == "NCHW" else (batch_size, 299, 299, 3)
47.         mod, params = relay.testing.inception_v3.get_workload(batch_size=batch_size, dtype=dtype)
48. elif name == "mxnet":
49. # MXNet 模型的示例
50. from mxnet.gluon.model_zoo.vision import get_model
52. assert layout == "NCHW"
54.         block = get_model("resnet18_v1", pretrained=True)
55.         mod, params = relay.frontend.from_mxnet(block, shape={"data": input_shape}, dtype=dtype)
56.         net = mod["main"]
57.         net = relay.Function(
58.             net.params, relay.nn.softmax(net.body), None, net.type_params, net.attrs
59. )
60.         mod = tvm.IRModule.from_expr(net)
62. return mod, params, input_shape, output_shape
64. # 定义神经网络和编译目标
65. network = "resnet-18"
66. batch_size = 1
67. layout = "NHWC"
68. target = tvm.target.Target("cuda")
69. dtype = "float32"
70. log_file = "%s-%s-B%d-%s.json" % (network, layout, batch_size, target.kind.name)

复制代码

提取搜刮任务​

接下来，从网络中提取搜刮任务及其权重。任务的权重是任务的子图在整个网络中出现的次数。通过使用权重，可以将网络的端到端耽误近似为 sum(latency[t] * weight[t])，其中 latency[t] 是任务的耽误，而 weight[t] 是任务的权重，任务调度器仅针对该目标举行优化。

1. # 从网络中提取任务
2. print("Extract tasks...")
3. mod, params, input_shape, output_shape = get_network(network, batch_size, layout, dtype=dtype)
4. tasks, task_weights = auto_scheduler.extract_tasks(mod["main"], params, target)
6. for idx, task in enumerate(tasks):
7. print("========== Task %d  (workload key: %s) ==========" % (idx, task.workload_key))
8. print(task.compute_dag)

复制代码

输出结果：

1. Extract tasks...
2. /workspace/python/tvm/driver/build_module.py:268: UserWarning: target_host parameter is going to be deprecated. Please pass in tvm.target.Target(target, host=target_host) instead.
3. "target_host parameter is going to be deprecated. "
4. ========== Task 0 (workload key: ["8654f16aeddf785bad9f028164b3a48d", [1, 56, 56, 64], [1, 1, 64, 64], [1, 56, 56, 64]]) ==========
5. placeholder = PLACEHOLDER [1, 56, 56, 64]
6. pad_temp(i0, i1, i2, i3) = placeholder[i0, i1, i2, i3]
7. placeholder = PLACEHOLDER [1, 1, 64, 64]
8. conv2d_nhwc(nn, yy, xx, ff) += (pad_temp[nn, (yy + ry), (xx + rx), rc]*placeholder[ry, rx, rc, ff])
10. ========== Task 1 (workload key: ["c4500b4e2fd04e695c32d2f31bbdc14a", [1, 28, 28, 128], [4, 4, 128, 128], [1, 28, 28, 128], [1, 1, 1, 128], [1, 28, 28, 128]]) ==========
11. placeholder = PLACEHOLDER [1, 28, 28, 128]
12. data_pad(i0, i1, i2, i3) = tir.if_then_else(((((i1 >= 1) && (i1 < 29)) && (i2 >= 1)) && (i2 < 29)), placeholder[i0, (i1 - 1), (i2 - 1), i3], 0f)
13. input_tile(eps, nu, p, ci) = data_pad[floordiv(p, 196), ((floormod(floordiv(p, 14), 14)*2) + eps), ((floormod(p, 14)*2) + nu), ci]
14. B(i, j) = select(((floormod(i, 4) == 3) && (floormod(j, 4) == 3)), 1f, select(((floormod(i, 4) == 3) && (floormod(j, 4) == 2)),  ..(OMITTED).. ormod(i, 4) == 0) && (floormod(j, 4) == 1)), 0f, select(((floormod(i, 4) == 0) && (floormod(j, 4) == 0)), 1f, 0f))))))))))))))))
15. data_pack(eps, nu, p, ci) += ((input_tile[r_a, r_b, p, ci]*B[r_a, eps])*B[r_b, nu])
16. placeholder = PLACEHOLDER [4, 4, 128, 128]
17. bgemm(eps, nu, p, co) += (data_pack[eps, nu, p, ci]*placeholder[eps, nu, co, ci])
18. A(i, j) = select(((floormod(i, 4) == 3) && (floormod(j, 2) == 1)), 1f, select(((floormod(i, 4) == 3) && (floormod(j, 2) == 0)),  ..(OMITTED).. ct(((floormod(i, 4) == 0) && (floormod(j, 2) == 1)), 0f, select(((floormod(i, 4) == 0) && (floormod(j, 2) == 0)), 1f, 0f))))))))
19. inverse(vh, vw, p, co) += ((bgemm[r_a, r_b, p, co]*A[r_a, vh])*A[r_b, vw])
20. conv2d_winograd(n, h, w, co) = inverse[floormod(h, 2), floormod(w, 2), ((((n*14)*14) + (floordiv(h, 2)*14)) + floordiv(w, 2)), co]
21. placeholder = PLACEHOLDER [1, 28, 28, 128]
22. T_add(ax0, ax1, ax2, ax3) = (conv2d_winograd[ax0, ax1, ax2, ax3] + placeholder[ax0, ax1, ax2, ax3])
23. placeholder = PLACEHOLDER [1, 1, 1, 128]
24. T_add(ax0, ax1, ax2, ax3) = (T_add[ax0, ax1, ax2, ax3] + placeholder[ax0, 0, 0, ax3])
25. T_relu(ax0, ax1, ax2, ax3) = max(T_add[ax0, ax1, ax2, ax3], 0f)
27. ========== Task 2 (workload key: ["06f578e6519a86e85028eecf4de64b25", [1, 56, 56, 64], [1, 1, 64, 128], [1, 28, 28, 128]]) ==========
28. placeholder = PLACEHOLDER [1, 56, 56, 64]
29. pad_temp(i0, i1, i2, i3) = placeholder[i0, i1, i2, i3]
30. placeholder = PLACEHOLDER [1, 1, 64, 128]
31. conv2d_nhwc(nn, yy, xx, ff) += (pad_temp[nn, ((yy*2) + ry), ((xx*2) + rx), rc]*placeholder[ry, rx, rc, ff])
33. ========== Task 3 (workload key: ["b8b52b9be9df6102466a22a014c44c1f", [1, 14, 14, 256], [4, 4, 256, 256], [1, 1, 1, 256], [1, 14, 14, 256]]) ==========
34. placeholder = PLACEHOLDER [1, 14, 14, 256]
35. data_pad(i0, i1, i2, i3) = tir.if_then_else(((((i1 >= 1) && (i1 < 15)) && (i2 >= 1)) && (i2 < 15)), placeholder[i0, (i1 - 1), (i2 - 1), i3], 0f)
36. input_tile(eps, nu, p, ci) = data_pad[floordiv(p, 49), ((floormod(floordiv(p, 7), 7)*2) + eps), ((floormod(p, 7)*2) + nu), ci]
37. B(i, j) = select(((floormod(i, 4) == 3) && (floormod(j, 4) == 3)), 1f, select(((floormod(i, 4) == 3) && (floormod(j, 4) == 2)),  ..(OMITTED).. ormod(i, 4) == 0) && (floormod(j, 4) == 1)), 0f, select(((floormod(i, 4) == 0) && (floormod(j, 4) == 0)), 1f, 0f))))))))))))))))
38. data_pack(eps, nu, p, ci) += ((input_tile[r_a, r_b, p, ci]*B[r_a, eps])*B[r_b, nu])
39. placeholder = PLACEHOLDER [4, 4, 256, 256]
40. bgemm(eps, nu, p, co) += (data_pack[eps, nu, p, ci]*placeholder[eps, nu, co, ci])
41. A(i, j) = select(((floormod(i, 4) == 3) && (floormod(j, 2) == 1)), 1f, select(((floormod(i, 4) == 3) && (floormod(j, 2) == 0)),  ..(OMITTED).. ct(((floormod(i, 4) == 0) && (floormod(j, 2) == 1)), 0f, select(((floormod(i, 4) == 0) && (floormod(j, 2) == 0)), 1f, 0f))))))))
42. inverse(vh, vw, p, co) += ((bgemm[r_a, r_b, p, co]*A[r_a, vh])*A[r_b, vw])
43. conv2d_winograd(n, h, w, co) = inverse[floormod(h, 2), floormod(w, 2), ((((n*7)*7) + (floordiv(h, 2)*7)) + floordiv(w, 2)), co]
44. placeholder = PLACEHOLDER [1, 1, 1, 256]
45. T_add(ax0, ax1, ax2, ax3) = (conv2d_winograd[ax0, ax1, ax2, ax3] + placeholder[ax0, 0, 0, ax3])
46. T_relu(ax0, ax1, ax2, ax3) = max(T_add[ax0, ax1, ax2, ax3], 0f)
48. ========== Task 4 (workload key: ["e4cdf917b876dbdd64488c3818d9c141", [1, 28, 28, 128], [4, 4, 128, 128], [1, 1, 1, 128], [1, 28, 28, 128]]) ==========
49. placeholder = PLACEHOLDER [1, 28, 28, 128]
50. data_pad(i0, i1, i2, i3) = tir.if_then_else(((((i1 >= 1) && (i1 < 29)) && (i2 >= 1)) && (i2 < 29)), placeholder[i0, (i1 - 1), (i2 - 1), i3], 0f)
51. input_tile(eps, nu, p, ci) = data_pad[floordiv(p, 196), ((floormod(floordiv(p, 14), 14)*2) + eps), ((floormod(p, 14)*2) + nu), ci]
52. B(i, j) = select(((floormod(i, 4) == 3) && (floormod(j, 4) == 3)), 1f, select(((floormod(i, 4) == 3) && (floormod(j, 4) == 2)),  ..(OMITTED).. ormod(i, 4) == 0) && (floormod(j, 4) == 1)), 0f, select(((floormod(i, 4) == 0) && (floormod(j, 4) == 0)), 1f, 0f))))))))))))))))
53. data_pack(eps, nu, p, ci) += ((input_tile[r_a, r_b, p, ci]*B[r_a, eps])*B[r_b, nu])
54. placeholder = PLACEHOLDER [4, 4, 128, 128]
55. bgemm(eps, nu, p, co) += (data_pack[eps, nu, p, ci]*placeholder[eps, nu, co, ci])
56. A(i, j) = select(((floormod(i, 4) == 3) && (floormod(j, 2) == 1)), 1f, select(((floormod(i, 4) == 3) && (floormod(j, 2) == 0)),  ..(OMITTED).. ct(((floormod(i, 4) == 0) && (floormod(j, 2) == 1)), 0f, select(((floormod(i, 4) == 0) && (floormod(j, 2) == 0)), 1f, 0f))))))))
57. inverse(vh, vw, p, co) += ((bgemm[r_a, r_b, p, co]*A[r_a, vh])*A[r_b, vw])
58. conv2d_winograd(n, h, w, co) = inverse[floormod(h, 2), floormod(w, 2), ((((n*14)*14) + (floordiv(h, 2)*14)) + floordiv(w, 2)), co]
59. placeholder = PLACEHOLDER [1, 1, 1, 128]
60. T_add(ax0, ax1, ax2, ax3) = (conv2d_winograd[ax0, ax1, ax2, ax3] + placeholder[ax0, 0, 0, ax3])
61. T_relu(ax0, ax1, ax2, ax3) = max(T_add[ax0, ax1, ax2, ax3], 0f)
63. ========== Task 5 (workload key: ["d730bcd28f0920f6b97245e2a11bd8d6", [1, 7, 7, 512], [4, 4, 512, 512], [1, 7, 7, 512], [1, 7, 7, 512]]) ==========
64. placeholder = PLACEHOLDER [1, 7, 7, 512]
65. data_pad(i0, i1, i2, i3) = tir.if_then_else(((((i1 >= 1) && (i1 < 8)) && (i2 >= 1)) && (i2 < 8)), placeholder[i0, (i1 - 1), (i2 - 1), i3], 0f)
66. input_tile(eps, nu, p, ci) = data_pad[floordiv(p, 16), ((floormod(floordiv(p, 4), 4)*2) + eps), ((floormod(p, 4)*2) + nu), ci]
67. B(i, j) = select(((floormod(i, 4) == 3) && (floormod(j, 4) == 3)), 1f, select(((floormod(i, 4) == 3) && (floormod(j, 4) == 2)),  ..(OMITTED).. ormod(i, 4) == 0) && (floormod(j, 4) == 1)), 0f, select(((floormod(i, 4) == 0) && (floormod(j, 4) == 0)), 1f, 0f))))))))))))))))
68. data_pack(eps, nu, p, ci) += ((input_tile[r_a, r_b, p, ci]*B[r_a, eps])*B[r_b, nu])
69. placeholder = PLACEHOLDER [4, 4, 512, 512]
70. bgemm(eps, nu, p, co) += (data_pack[eps, nu, p, ci]*placeholder[eps, nu, co, ci])
71. A(i, j) = select(((floormod(i, 4) == 3) && (floormod(j, 2) == 1)), 1f, select(((floormod(i, 4) == 3) && (floormod(j, 2) == 0)),  ..(OMITTED).. ct(((floormod(i, 4) == 0) && (floormod(j, 2) == 1)), 0f, select(((floormod(i, 4) == 0) && (floormod(j, 2) == 0)), 1f, 0f))))))))
72. inverse(vh, vw, p, co) += ((bgemm[r_a, r_b, p, co]*A[r_a, vh])*A[r_b, vw])
73. conv2d_winograd(n, h, w, co) = inverse[floormod(h, 2), floormod(w, 2), ((((n*4)*4) + (floordiv(h, 2)*4)) + floordiv(w, 2)), co]
74. placeholder = PLACEHOLDER [1, 7, 7, 512]
75. T_add(ax0, ax1, ax2, ax3) = (conv2d_winograd[ax0, ax1, ax2, ax3] + placeholder[ax0, ax1, ax2, ax3])
77. ========== Task 6 (workload key: ["b818b53148cd450f86569dfc3e04cb8a", [1, 56, 56, 64], [6, 6, 64, 64], [1, 1, 1, 64], [1, 56, 56, 64]]) ==========
78. placeholder = PLACEHOLDER [1, 56, 56, 64]
79. data_pad(i0, i1, i2, i3) = tir.if_then_else(((((i1 >= 1) && (i1 < 57)) && (i2 >= 1)) && (i2 < 57)), placeholder[i0, (i1 - 1), (i2 - 1), i3], 0f)
80. input_tile(eps, nu, p, ci) = data_pad[floordiv(p, 196), ((floormod(floordiv(p, 14), 14)*4) + eps), ((floormod(p, 14)*4) + nu), ci]
81. B(i, j) = select(((floormod(i, 6) == 5) && (floormod(j, 6) == 5)), 1f, select(((floormod(i, 6) == 5) && (floormod(j, 6) == 4)),  ..(OMITTED).. (floormod(j, 6) == 1)), 0f, select(((floormod(i, 6) == 0) && (floormod(j, 6) == 0)), 1f, 0f))))))))))))))))))))))))))))))))))))
82. data_pack(eps, nu, p, ci) += ((input_tile[r_a, r_b, p, ci]*B[r_a, eps])*B[r_b, nu])
83. placeholder = PLACEHOLDER [6, 6, 64, 64]
84. bgemm(eps, nu, p, co) += (data_pack[eps, nu, p, ci]*placeholder[eps, nu, co, ci])
85. A(i, j) = select(((floormod(i, 6) == 5) && (floormod(j, 4) == 3)), 1f, select(((floormod(i, 6) == 5) && (floormod(j, 4) == 2)),  ..(OMITTED).. 6) == 0) && (floormod(j, 4) == 1)), 0f, select(((floormod(i, 6) == 0) && (floormod(j, 4) == 0)), 1f, 0f))))))))))))))))))))))))
86. inverse(vh, vw, p, co) += ((bgemm[r_a, r_b, p, co]*A[r_a, vh])*A[r_b, vw])
87. conv2d_winograd(n, h, w, co) = inverse[floormod(h, 4), floormod(w, 4), ((((n*14)*14) + (floordiv(h, 4)*14)) + floordiv(w, 4)), co]
88. placeholder = PLACEHOLDER [1, 1, 1, 64]
89. T_add(ax0, ax1, ax2, ax3) = (conv2d_winograd[ax0, ax1, ax2, ax3] + placeholder[ax0, 0, 0, ax3])
90. T_relu(ax0, ax1, ax2, ax3) = max(T_add[ax0, ax1, ax2, ax3], 0f)
92. ========== Task 7 (workload key: ["ad6cecbf5d85cb1cda3c2bb7af170211", [1, 7, 7, 512], [4, 4, 512, 512], [1, 7, 7, 512], [1, 1, 1, 512], [1, 1, 1, 512], [1, 7, 7, 512]]) ==========
93. placeholder = PLACEHOLDER [1, 7, 7, 512]
94. data_pad(i0, i1, i2, i3) = tir.if_then_else(((((i1 >= 1) && (i1 < 8)) && (i2 >= 1)) && (i2 < 8)), placeholder[i0, (i1 - 1), (i2 - 1), i3], 0f)
95. input_tile(eps, nu, p, ci) = data_pad[floordiv(p, 16), ((floormod(floordiv(p, 4), 4)*2) + eps), ((floormod(p, 4)*2) + nu), ci]
96. B(i, j) = select(((floormod(i, 4) == 3) && (floormod(j, 4) == 3)), 1f, select(((floormod(i, 4) == 3) && (floormod(j, 4) == 2)),  ..(OMITTED).. ormod(i, 4) == 0) && (floormod(j, 4) == 1)), 0f, select(((floormod(i, 4) == 0) && (floormod(j, 4) == 0)), 1f, 0f))))))))))))))))
97. data_pack(eps, nu, p, ci) += ((input_tile[r_a, r_b, p, ci]*B[r_a, eps])*B[r_b, nu])
98. placeholder = PLACEHOLDER [4, 4, 512, 512]
99. bgemm(eps, nu, p, co) += (data_pack[eps, nu, p, ci]*placeholder[eps, nu, co, ci])
100. A(i, j) = select(((floormod(i, 4) == 3) && (floormod(j, 2) == 1)), 1f, select(((floormod(i, 4) == 3) && (floormod(j, 2) == 0)),  ..(OMITTED).. ct(((floormod(i, 4) == 0) && (floormod(j, 2) == 1)), 0f, select(((floormod(i, 4) == 0) && (floormod(j, 2) == 0)), 1f, 0f))))))))
101. inverse(vh, vw, p, co) += ((bgemm[r_a, r_b, p, co]*A[r_a, vh])*A[r_b, vw])
102. conv2d_winograd(n, h, w, co) = inverse[floormod(h, 2), floormod(w, 2), ((((n*4)*4) + (floordiv(h, 2)*4)) + floordiv(w, 2)), co]
103. placeholder = PLACEHOLDER [1, 7, 7, 512]
104. T_add(ax0, ax1, ax2, ax3) = (conv2d_winograd[ax0, ax1, ax2, ax3] + placeholder[ax0, ax1, ax2, ax3])
105. placeholder = PLACEHOLDER [1, 1, 1, 512]
106. T_multiply(ax0, ax1, ax2, ax3) = (T_add[ax0, ax1, ax2, ax3]*placeholder[ax0, 0, 0, ax3])
107. placeholder = PLACEHOLDER [1, 1, 1, 512]
108. T_add(ax0, ax1, ax2, ax3) = (T_multiply[ax0, ax1, ax2, ax3] + placeholder[ax0, 0, 0, ax3])
109. T_relu(ax0, ax1, ax2, ax3) = max(T_add[ax0, ax1, ax2, ax3], 0f)
111. ========== Task 8 (workload key: ["f3b6c10fcc6ce01ff01add933e4d21e9", [1, 14, 14, 256], [4, 4, 256, 256], [1, 14, 14, 256], [1, 1, 1, 256], [1, 14, 14, 256]]) ==========
112. placeholder = PLACEHOLDER [1, 14, 14, 256]
113. data_pad(i0, i1, i2, i3) = tir.if_then_else(((((i1 >= 1) && (i1 < 15)) && (i2 >= 1)) && (i2 < 15)), placeholder[i0, (i1 - 1), (i2 - 1), i3], 0f)
114. input_tile(eps, nu, p, ci) = data_pad[floordiv(p, 49), ((floormod(floordiv(p, 7), 7)*2) + eps), ((floormod(p, 7)*2) + nu), ci]
115. B(i, j) = select(((floormod(i, 4) == 3) && (floormod(j, 4) == 3)), 1f, select(((floormod(i, 4) == 3) && (floormod(j, 4) == 2)),  ..(OMITTED).. ormod(i, 4) == 0) && (floormod(j, 4) == 1)), 0f, select(((floormod(i, 4) == 0) && (floormod(j, 4) == 0)), 1f, 0f))))))))))))))))
116. data_pack(eps, nu, p, ci) += ((input_tile[r_a, r_b, p, ci]*B[r_a, eps])*B[r_b, nu])
117. placeholder = PLACEHOLDER [4, 4, 256, 256]
118. bgemm(eps, nu, p, co) += (data_pack[eps, nu, p, ci]*placeholder[eps, nu, co, ci])
119. A(i, j) = select(((floormod(i, 4) == 3) && (floormod(j, 2) == 1)), 1f, select(((floormod(i, 4) == 3) && (floormod(j, 2) == 0)),  ..(OMITTED).. ct(((floormod(i, 4) == 0) && (floormod(j, 2) == 1)), 0f, select(((floormod(i, 4) == 0) && (floormod(j, 2) == 0)), 1f, 0f))))))))
120. inverse(vh, vw, p, co) += ((bgemm[r_a, r_b, p, co]*A[r_a, vh])*A[r_b, vw])
121. conv2d_winograd(n, h, w, co) = inverse[floormod(h, 2), floormod(w, 2), ((((n*7)*7) + (floordiv(h, 2)*7)) + floordiv(w, 2)), co]
122. placeholder = PLACEHOLDER [1, 14, 14, 256]
123. T_add(ax0, ax1, ax2, ax3) = (conv2d_winograd[ax0, ax1, ax2, ax3] + placeholder[ax0, ax1, ax2, ax3])
124. placeholder = PLACEHOLDER [1, 1, 1, 256]
125. T_add(ax0, ax1, ax2, ax3) = (T_add[ax0, ax1, ax2, ax3] + placeholder[ax0, 0, 0, ax3])
126. T_relu(ax0, ax1, ax2, ax3) = max(T_add[ax0, ax1, ax2, ax3], 0f)
128. ========== Task 9 (workload key: ["d7b65649a4dd54becea0a52aabbc5af5", [1, 1000], [1, 1000]]) ==========
129. placeholder = PLACEHOLDER [1, 1000]
130. T_softmax_maxelem(i0) max= placeholder[i0, k]
131. T_softmax_exp(i0, i1) = tir.exp((placeholder[i0, i1] - T_softmax_maxelem[i0]))
132. T_softmax_expsum(i0) += T_softmax_exp[i0, k]
133. T_softmax_norm(i0, i1) = (T_softmax_exp[i0, i1]/T_softmax_expsum[i0])
135. ========== Task 10 (workload key: ["69115f188984ae34ede37c3b8ca40b43", [1, 7, 7, 512], [1, 1, 1, 512]]) ==========
136. placeholder = PLACEHOLDER [1, 7, 7, 512]
137. tensor(ax0, ax1, ax2, ax3) += placeholder[ax0, ((ax1*7) + rv0), ((ax2*7) + rv1), ax3]
138. tensor(ax0, ax1, ax2, ax3) = (tensor[ax0, ax1, ax2, ax3]/(float32((select((bool)1, ((ax1 + 1)*7), (((ax1 + 1)*7) + 1)) - (ax1*7)))*float32((select((bool)1, ((ax2 + 1)*7), (((ax2 + 1)*7) + 1)) - (ax2*7)))))
140. ========== Task 11 (workload key: ["3a69f9fbc63760d99e36b4c17b3bfc57", [1, 7, 7, 512], [4, 4, 512, 512], [1, 1, 1, 512], [1, 7, 7, 512]]) ==========
141. placeholder = PLACEHOLDER [1, 7, 7, 512]
142. data_pad(i0, i1, i2, i3) = tir.if_then_else(((((i1 >= 1) && (i1 < 8)) && (i2 >= 1)) && (i2 < 8)), placeholder[i0, (i1 - 1), (i2 - 1), i3], 0f)
143. input_tile(eps, nu, p, ci) = data_pad[floordiv(p, 16), ((floormod(floordiv(p, 4), 4)*2) + eps), ((floormod(p, 4)*2) + nu), ci]
144. B(i, j) = select(((floormod(i, 4) == 3) && (floormod(j, 4) == 3)), 1f, select(((floormod(i, 4) == 3) && (floormod(j, 4) == 2)),  ..(OMITTED).. ormod(i, 4) == 0) && (floormod(j, 4) == 1)), 0f, select(((floormod(i, 4) == 0) && (floormod(j, 4) == 0)), 1f, 0f))))))))))))))))
145. data_pack(eps, nu, p, ci) += ((input_tile[r_a, r_b, p, ci]*B[r_a, eps])*B[r_b, nu])
146. placeholder = PLACEHOLDER [4, 4, 512, 512]
147. bgemm(eps, nu, p, co) += (data_pack[eps, nu, p, ci]*placeholder[eps, nu, co, ci])
148. A(i, j) = select(((floormod(i, 4) == 3) && (floormod(j, 2) == 1)), 1f, select(((floormod(i, 4) == 3) && (floormod(j, 2) == 0)),  ..(OMITTED).. ct(((floormod(i, 4) == 0) && (floormod(j, 2) == 1)), 0f, select(((floormod(i, 4) == 0) && (floormod(j, 2) == 0)), 1f, 0f))))))))
149. inverse(vh, vw, p, co) += ((bgemm[r_a, r_b, p, co]*A[r_a, vh])*A[r_b, vw])
150. conv2d_winograd(n, h, w, co) = inverse[floormod(h, 2), floormod(w, 2), ((((n*4)*4) + (floordiv(h, 2)*4)) + floordiv(w, 2)), co]
151. placeholder = PLACEHOLDER [1, 1, 1, 512]
152. T_add(ax0, ax1, ax2, ax3) = (conv2d_winograd[ax0, ax1, ax2, ax3] + placeholder[ax0, 0, 0, ax3])
153. T_relu(ax0, ax1, ax2, ax3) = max(T_add[ax0, ax1, ax2, ax3], 0f)
155. ========== Task 12 (workload key: ["06f578e6519a86e85028eecf4de64b25", [1, 28, 28, 128], [1, 1, 128, 256], [1, 14, 14, 256]]) ==========
156. placeholder = PLACEHOLDER [1, 28, 28, 128]
157. pad_temp(i0, i1, i2, i3) = placeholder[i0, i1, i2, i3]
158. placeholder = PLACEHOLDER [1, 1, 128, 256]
159. conv2d_nhwc(nn, yy, xx, ff) += (pad_temp[nn, ((yy*2) + ry), ((xx*2) + rx), rc]*placeholder[ry, rx, rc, ff])
161. ========== Task 13 (workload key: ["96daaa9daa1b41bc383b7c05ce8b58de", [1, 14, 14, 256], [3, 3, 256, 512], [1, 1, 1, 512], [1, 7, 7, 512]]) ==========
162. placeholder = PLACEHOLDER [1, 14, 14, 256]
163. pad_temp(i0, i1, i2, i3) = tir.if_then_else(((((i1 >= 1) && (i1 < 15)) && (i2 >= 1)) && (i2 < 15)), placeholder[i0, (i1 - 1), (i2 - 1), i3], 0f)
164. placeholder = PLACEHOLDER [3, 3, 256, 512]
165. conv2d_nhwc(nn, yy, xx, ff) += (pad_temp[nn, ((yy*2) + ry), ((xx*2) + rx), rc]*placeholder[ry, rx, rc, ff])
166. placeholder = PLACEHOLDER [1, 1, 1, 512]
167. T_add(ax0, ax1, ax2, ax3) = (conv2d_nhwc[ax0, ax1, ax2, ax3] + placeholder[ax0, 0, 0, ax3])
168. T_relu(ax0, ax1, ax2, ax3) = max(T_add[ax0, ax1, ax2, ax3], 0f)
170. ========== Task 14 (workload key: ["dac19035dd5fe9424ee8617421b9c817", [1, 28, 28, 128], [4, 4, 128, 128], [1, 28, 28, 128], [1, 28, 28, 128]]) ==========
171. placeholder = PLACEHOLDER [1, 28, 28, 128]
172. data_pad(i0, i1, i2, i3) = tir.if_then_else(((((i1 >= 1) && (i1 < 29)) && (i2 >= 1)) && (i2 < 29)), placeholder[i0, (i1 - 1), (i2 - 1), i3], 0f)
173. input_tile(eps, nu, p, ci) = data_pad[floordiv(p, 196), ((floormod(floordiv(p, 14), 14)*2) + eps), ((floormod(p, 14)*2) + nu), ci]
174. B(i, j) = select(((floormod(i, 4) == 3) && (floormod(j, 4) == 3)), 1f, select(((floormod(i, 4) == 3) && (floormod(j, 4) == 2)),  ..(OMITTED).. ormod(i, 4) == 0) && (floormod(j, 4) == 1)), 0f, select(((floormod(i, 4) == 0) && (floormod(j, 4) == 0)), 1f, 0f))))))))))))))))
175. data_pack(eps, nu, p, ci) += ((input_tile[r_a, r_b, p, ci]*B[r_a, eps])*B[r_b, nu])
176. placeholder = PLACEHOLDER [4, 4, 128, 128]
177. bgemm(eps, nu, p, co) += (data_pack[eps, nu, p, ci]*placeholder[eps, nu, co, ci])
178. A(i, j) = select(((floormod(i, 4) == 3) && (floormod(j, 2) == 1)), 1f, select(((floormod(i, 4) == 3) && (floormod(j, 2) == 0)),  ..(OMITTED).. ct(((floormod(i, 4) == 0) && (floormod(j, 2) == 1)), 0f, select(((floormod(i, 4) == 0) && (floormod(j, 2) == 0)), 1f, 0f))))))))
179. inverse(vh, vw, p, co) += ((bgemm[r_a, r_b, p, co]*A[r_a, vh])*A[r_b, vw])
180. conv2d_winograd(n, h, w, co) = inverse[floormod(h, 2), floormod(w, 2), ((((n*14)*14) + (floordiv(h, 2)*14)) + floordiv(w, 2)), co]
181. placeholder = PLACEHOLDER [1, 28, 28, 128]
182. T_add(ax0, ax1, ax2, ax3) = (conv2d_winograd[ax0, ax1, ax2, ax3] + placeholder[ax0, ax1, ax2, ax3])
184. ========== Task 15 (workload key: ["96daaa9daa1b41bc383b7c05ce8b58de", [1, 28, 28, 128], [3, 3, 128, 256], [1, 1, 1, 256], [1, 14, 14, 256]]) ==========
185. placeholder = PLACEHOLDER [1, 28, 28, 128]
186. pad_temp(i0, i1, i2, i3) = tir.if_then_else(((((i1 >= 1) && (i1 < 29)) && (i2 >= 1)) && (i2 < 29)), placeholder[i0, (i1 - 1), (i2 - 1), i3], 0f)
187. placeholder = PLACEHOLDER [3, 3, 128, 256]
188. conv2d_nhwc(nn, yy, xx, ff) += (pad_temp[nn, ((yy*2) + ry), ((xx*2) + rx), rc]*placeholder[ry, rx, rc, ff])
189. placeholder = PLACEHOLDER [1, 1, 1, 256]
190. T_add(ax0, ax1, ax2, ax3) = (conv2d_nhwc[ax0, ax1, ax2, ax3] + placeholder[ax0, 0, 0, ax3])
191. T_relu(ax0, ax1, ax2, ax3) = max(T_add[ax0, ax1, ax2, ax3], 0f)
193. ========== Task 16 (workload key: ["1e3c4211ffd2f2db91078ae4d04b779d", [1, 56, 56, 64], [6, 6, 64, 64], [1, 56, 56, 64], [1, 1, 1, 64], [1, 56, 56, 64]]) ==========
194. placeholder = PLACEHOLDER [1, 56, 56, 64]
195. data_pad(i0, i1, i2, i3) = tir.if_then_else(((((i1 >= 1) && (i1 < 57)) && (i2 >= 1)) && (i2 < 57)), placeholder[i0, (i1 - 1), (i2 - 1), i3], 0f)
196. input_tile(eps, nu, p, ci) = data_pad[floordiv(p, 196), ((floormod(floordiv(p, 14), 14)*4) + eps), ((floormod(p, 14)*4) + nu), ci]
197. B(i, j) = select(((floormod(i, 6) == 5) && (floormod(j, 6) == 5)), 1f, select(((floormod(i, 6) == 5) && (floormod(j, 6) == 4)),  ..(OMITTED).. (floormod(j, 6) == 1)), 0f, select(((floormod(i, 6) == 0) && (floormod(j, 6) == 0)), 1f, 0f))))))))))))))))))))))))))))))))))))
198. data_pack(eps, nu, p, ci) += ((input_tile[r_a, r_b, p, ci]*B[r_a, eps])*B[r_b, nu])
199. placeholder = PLACEHOLDER [6, 6, 64, 64]
200. bgemm(eps, nu, p, co) += (data_pack[eps, nu, p, ci]*placeholder[eps, nu, co, ci])
201. A(i, j) = select(((floormod(i, 6) == 5) && (floormod(j, 4) == 3)), 1f, select(((floormod(i, 6) == 5) && (floormod(j, 4) == 2)),  ..(OMITTED).. 6) == 0) && (floormod(j, 4) == 1)), 0f, select(((floormod(i, 6) == 0) && (floormod(j, 4) == 0)), 1f, 0f))))))))))))))))))))))))
202. inverse(vh, vw, p, co) += ((bgemm[r_a, r_b, p, co]*A[r_a, vh])*A[r_b, vw])
203. conv2d_winograd(n, h, w, co) = inverse[floormod(h, 4), floormod(w, 4), ((((n*14)*14) + (floordiv(h, 4)*14)) + floordiv(w, 4)), co]
204. placeholder = PLACEHOLDER [1, 56, 56, 64]
205. T_add(ax0, ax1, ax2, ax3) = (conv2d_winograd[ax0, ax1, ax2, ax3] + placeholder[ax0, ax1, ax2, ax3])
206. placeholder = PLACEHOLDER [1, 1, 1, 64]
207. T_add(ax0, ax1, ax2, ax3) = (T_add[ax0, ax1, ax2, ax3] + placeholder[ax0, 0, 0, ax3])
208. T_relu(ax0, ax1, ax2, ax3) = max(T_add[ax0, ax1, ax2, ax3], 0f)
210. ========== Task 17 (workload key: ["96daaa9daa1b41bc383b7c05ce8b58de", [1, 224, 224, 3], [7, 7, 3, 64], [1, 1, 1, 64], [1, 112, 112, 64]]) ==========
211. placeholder = PLACEHOLDER [1, 224, 224, 3]
212. pad_temp(i0, i1, i2, i3) = tir.if_then_else(((((i1 >= 3) && (i1 < 227)) && (i2 >= 3)) && (i2 < 227)), placeholder[i0, (i1 - 3), (i2 - 3), i3], 0f)
213. placeholder = PLACEHOLDER [7, 7, 3, 64]
214. conv2d_nhwc(nn, yy, xx, ff) += (pad_temp[nn, ((yy*2) + ry), ((xx*2) + rx), rc]*placeholder[ry, rx, rc, ff])
215. placeholder = PLACEHOLDER [1, 1, 1, 64]
216. T_add(ax0, ax1, ax2, ax3) = (conv2d_nhwc[ax0, ax1, ax2, ax3] + placeholder[ax0, 0, 0, ax3])
217. T_relu(ax0, ax1, ax2, ax3) = max(T_add[ax0, ax1, ax2, ax3], 0f)
219. ========== Task 18 (workload key: ["3ea73fb9b0364374730d09e068821f95", [1, 56, 56, 64], [6, 6, 64, 64], [1, 56, 56, 64], [1, 56, 56, 64]]) ==========
220. placeholder = PLACEHOLDER [1, 56, 56, 64]
221. data_pad(i0, i1, i2, i3) = tir.if_then_else(((((i1 >= 1) && (i1 < 57)) && (i2 >= 1)) && (i2 < 57)), placeholder[i0, (i1 - 1), (i2 - 1), i3], 0f)
222. input_tile(eps, nu, p, ci) = data_pad[floordiv(p, 196), ((floormod(floordiv(p, 14), 14)*4) + eps), ((floormod(p, 14)*4) + nu), ci]
223. B(i, j) = select(((floormod(i, 6) == 5) && (floormod(j, 6) == 5)), 1f, select(((floormod(i, 6) == 5) && (floormod(j, 6) == 4)),  ..(OMITTED).. (floormod(j, 6) == 1)), 0f, select(((floormod(i, 6) == 0) && (floormod(j, 6) == 0)), 1f, 0f))))))))))))))))))))))))))))))))))))
224. data_pack(eps, nu, p, ci) += ((input_tile[r_a, r_b, p, ci]*B[r_a, eps])*B[r_b, nu])
225. placeholder = PLACEHOLDER [6, 6, 64, 64]
226. bgemm(eps, nu, p, co) += (data_pack[eps, nu, p, ci]*placeholder[eps, nu, co, ci])
227. A(i, j) = select(((floormod(i, 6) == 5) && (floormod(j, 4) == 3)), 1f, select(((floormod(i, 6) == 5) && (floormod(j, 4) == 2)),  ..(OMITTED).. 6) == 0) && (floormod(j, 4) == 1)), 0f, select(((floormod(i, 6) == 0) && (floormod(j, 4) == 0)), 1f, 0f))))))))))))))))))))))))
228. inverse(vh, vw, p, co) += ((bgemm[r_a, r_b, p, co]*A[r_a, vh])*A[r_b, vw])
229. conv2d_winograd(n, h, w, co) = inverse[floormod(h, 4), floormod(w, 4), ((((n*14)*14) + (floordiv(h, 4)*14)) + floordiv(w, 4)), co]
230. placeholder = PLACEHOLDER [1, 56, 56, 64]
231. T_add(ax0, ax1, ax2, ax3) = (conv2d_winograd[ax0, ax1, ax2, ax3] + placeholder[ax0, ax1, ax2, ax3])
233. ========== Task 19 (workload key: ["d374e472bd9d8164892b9e28a0a8cb59", [1, 14, 14, 256], [4, 4, 256, 256], [1, 14, 14, 256], [1, 14, 14, 256]]) ==========
234. placeholder = PLACEHOLDER [1, 14, 14, 256]
235. data_pad(i0, i1, i2, i3) = tir.if_then_else(((((i1 >= 1) && (i1 < 15)) && (i2 >= 1)) && (i2 < 15)), placeholder[i0, (i1 - 1), (i2 - 1), i3], 0f)
236. input_tile(eps, nu, p, ci) = data_pad[floordiv(p, 49), ((floormod(floordiv(p, 7), 7)*2) + eps), ((floormod(p, 7)*2) + nu), ci]
237. B(i, j) = select(((floormod(i, 4) == 3) && (floormod(j, 4) == 3)), 1f, select(((floormod(i, 4) == 3) && (floormod(j, 4) == 2)),  ..(OMITTED).. ormod(i, 4) == 0) && (floormod(j, 4) == 1)), 0f, select(((floormod(i, 4) == 0) && (floormod(j, 4) == 0)), 1f, 0f))))))))))))))))
238. data_pack(eps, nu, p, ci) += ((input_tile[r_a, r_b, p, ci]*B[r_a, eps])*B[r_b, nu])
239. placeholder = PLACEHOLDER [4, 4, 256, 256]
240. bgemm(eps, nu, p, co) += (data_pack[eps, nu, p, ci]*placeholder[eps, nu, co, ci])
241. A(i, j) = select(((floormod(i, 4) == 3) && (floormod(j, 2) == 1)), 1f, select(((floormod(i, 4) == 3) && (floormod(j, 2) == 0)),  ..(OMITTED).. ct(((floormod(i, 4) == 0) && (floormod(j, 2) == 1)), 0f, select(((floormod(i, 4) == 0) && (floormod(j, 2) == 0)), 1f, 0f))))))))
242. inverse(vh, vw, p, co) += ((bgemm[r_a, r_b, p, co]*A[r_a, vh])*A[r_b, vw])
243. conv2d_winograd(n, h, w, co) = inverse[floormod(h, 2), floormod(w, 2), ((((n*7)*7) + (floordiv(h, 2)*7)) + floordiv(w, 2)), co]
244. placeholder = PLACEHOLDER [1, 14, 14, 256]
245. T_add(ax0, ax1, ax2, ax3) = (conv2d_winograd[ax0, ax1, ax2, ax3] + placeholder[ax0, ax1, ax2, ax3])
247. ========== Task 20 (workload key: ["64b98c71af70a904fdbb81d7d4188d84", [1, 112, 112, 64], [1, 1, 1, 64], [1, 56, 56, 64]]) ==========
248. placeholder = PLACEHOLDER [1, 112, 112, 64]
249. pad_temp(ax0, ax1, ax2, ax3) = tir.if_then_else(((((ax1 >= 1) && (ax1 < 113)) && (ax2 >= 1)) && (ax2 < 113)), placeholder[ax0, (ax1 - 1), (ax2 - 1), ax3], -3.40282e+38f)
250. tensor(ax0, ax1, ax2, ax3) max= pad_temp[ax0, ((ax1*2) + rv0), ((ax2*2) + rv1), ax3]
251. placeholder = PLACEHOLDER [1, 1, 1, 64]
252. T_add(ax0, ax1, ax2, ax3) = (tensor[ax0, ax1, ax2, ax3] + placeholder[ax0, 0, 0, ax3])
253. T_relu(ax0, ax1, ax2, ax3) = max(T_add[ax0, ax1, ax2, ax3], 0f)
255. ========== Task 21 (workload key: ["06f578e6519a86e85028eecf4de64b25", [1, 14, 14, 256], [1, 1, 256, 512], [1, 7, 7, 512]]) ==========
256. placeholder = PLACEHOLDER [1, 14, 14, 256]
257. pad_temp(i0, i1, i2, i3) = placeholder[i0, i1, i2, i3]
258. placeholder = PLACEHOLDER [1, 1, 256, 512]
259. conv2d_nhwc(nn, yy, xx, ff) += (pad_temp[nn, ((yy*2) + ry), ((xx*2) + rx), rc]*placeholder[ry, rx, rc, ff])
261. ========== Task 22 (workload key: ["7d44c6e3c81cd80f61ff2265b2bae89a", [1, 512], [1000, 512], [1, 1000], [1, 1000]]) ==========
262. placeholder = PLACEHOLDER [1, 512]
263. placeholder = PLACEHOLDER [1000, 512]
264. T_matmul_NT(i, j) += (placeholder[i, k]*placeholder[j, k])
265. placeholder = PLACEHOLDER [1, 1000]
266. T_add(ax0, ax1) = (T_matmul_NT[ax0, ax1] + placeholder[ax0, ax1])
268. ========== Task 23 (workload key: ["96daaa9daa1b41bc383b7c05ce8b58de", [1, 56, 56, 64], [3, 3, 64, 128], [1, 1, 1, 128], [1, 28, 28, 128]]) ==========
269. placeholder = PLACEHOLDER [1, 56, 56, 64]
270. pad_temp(i0, i1, i2, i3) = tir.if_then_else(((((i1 >= 1) && (i1 < 57)) && (i2 >= 1)) && (i2 < 57)), placeholder[i0, (i1 - 1), (i2 - 1), i3], 0f)
271. placeholder = PLACEHOLDER [3, 3, 64, 128]
272. conv2d_nhwc(nn, yy, xx, ff) += (pad_temp[nn, ((yy*2) + ry), ((xx*2) + rx), rc]*placeholder[ry, rx, rc, ff])
273. placeholder = PLACEHOLDER [1, 1, 1, 128]
274. T_add(ax0, ax1, ax2, ax3) = (conv2d_nhwc[ax0, ax1, ax2, ax3] + placeholder[ax0, 0, 0, ax3])
275. T_relu(ax0, ax1, ax2, ax3) = max(T_add[ax0, ax1, ax2, ax3], 0f)

复制代码

开始调优​

接下来为调优和启动搜刮任务设置一些选项

• measure_ctx 启动不同的测试过程以提供隔离。在测试期间保护主进程免受 GPU 瓦解并制止其他 runtime 冲突。
  
• min_repeat_ms 定义每次测试中一次“重复”的最短持续时间，可以预热 GPU 以获得正确测试结果，通常，推荐设置值 >= 300 ms。
  
• num_measure_trials 是调优期间可以使用的测试次数（根据自己的时间预算调解这个参数），若要快速演示，可将其设置为较小的数字（比方 200）。推荐将其设置为 900 * len(tasks) 左右，以便使搜刮收敛。好比 resnet-18 有 24 个任务，所以可以设置为 20000。
  
• 此外，使用 RecordToFile 将测试记录转储到日志文件中，测试记录可用于历史最佳查询、规复搜刮以及举行后续分析。
  
• 更多参数参见 auto_scheduler.TuningOptions，auto_scheduler.LocalRunner。
  

1. def run_tuning():
2. print("Begin tuning...")
3.     measure_ctx = auto_scheduler.LocalRPCMeasureContext(repeat=1, min_repeat_ms=300, timeout=10)
5.     tuner = auto_scheduler.TaskScheduler(tasks, task_weights)
6.     tune_option = auto_scheduler.TuningOptions(
7.         num_measure_trials=200, # 将此更改为 20000 以达到最佳性能
8.         runner=measure_ctx.runner,
9.         measure_callbacks=[auto_scheduler.RecordToFile(log_file)],
10. )
12.     tuner.tune(tune_option)
14. # 不在网页服务器中运行调优，因为它需要的时间太长。
15. # 取消注释运行下面行。
16. # run_tuning()

复制代码

  备注
解释调优过程中打印的信息
在调优过程中，控制台上会打印很多用于调试的信息，最重要的信息是任务调度步调的输出，下表是输出示例。

  ------------------------------ [ Task Scheduler ]

  | ID | Latency (ms) | Speed (GFLOPS) | Trials |

  | 0 | 0.005 | 0.88 | 64 |
| 1 | 0.010 | 99.10 | 64 |
| 2 | 0.006 | 0.00 | 64 |
| 3 | 0.145 | 979.78 | 384 |
| 4 | 0.130 | 1097.02 | 384 |
| 5 | 0.143 | 992.69 | 384 |
| 6 | 0.076 | 1526.86 | 192 |
| 7 | 0.115 | 999.44 | 320 |
| 8 | 0.079 | 1449.39 | 320 |
| 9 | 0.122 | 938.73 | 384 |
| 10 | 0.063 | 1832.98 | 192 |
| 11 | 0.072 | 1763.62 | 256 |
| 12 | 0.062 | 2036.40 | 192 |
| 13 | 0.068 | 1874.44 | 192 |
| 14 | 0.049 | 2346.50 | 128 |
| 15 | 0.076 | 1694.31 | 256 |
| 16 | 0.067 | 1933.30 | 448 |
| 17 | 0.076 | 1680.90 | 256 |
| 18 | 0.022 | 98.43 | 64 |
| 19 | 0.076 | 3112.55 | 192 |
| 20 | 0.013 | 2026.44 | 64 |
| 21 | 0.011 | 1136.69 | 64 |
| 22 | 0.013 | 992.47 | 64 |
| 23 | 0.020 | 627.56 | 64 |

  Estimated total latency: 1.587 ms Trials: 4992 Used time : 13296 s Next ID: 3
  此表列出了所有任务的耽误和（预估）速度，还列出了所有任务的测试分配。末了一行打印了这些任务的总加权耽误，可以大略估计网络的端到端执行时间。末了一行还打印了测试试验的总数、主动调优所花费的总时间以及下一个要调优的任务的 ID。
还有一些「tvm::Error」错误，因为 auto-scheduler 会尝试一些无效的调度。若调优继承运行，则可以忽略这些错误，因为这些错误与主进程隔离。
    备注
提前终止调优
可以通过逼迫终止此进程来提前终止调优，只要在日志文件中为每个任务获得至少一个有效的调度，就能够举行编译（下面的部门）。
  编译及评估​

主动调优后，用找到的最佳调度来编译网络。在主动调优期间，所有测试记录都被转储到日志文件中，可以读取日志文件加载最佳调度。

1. # 用历史最佳编译
2. print("Compile...")
3. with auto_scheduler.ApplyHistoryBest(log_file):
4. with tvm.transform.PassContext(opt_level=3, config={"relay.backend.use_auto_scheduler": True}):
5.         lib = relay.build(mod, target=target, params=params)
7. # 创建图执行器
8. dev = tvm.device(str(target), 0)
9. module = graph_executor.GraphModule(lib["default"](dev))
10. data_tvm = tvm.nd.array((np.random.uniform(size=input_shape)).astype(dtype))
11. module.set_input("data", data_tvm)
13. # 评估
14. print("Evaluate inference time cost...")
15. print(module.benchmark(dev, repeat=3, min_repeat_ms=500))

复制代码

输出结果：

1. Compile...
2. /workspace/python/tvm/driver/build_module.py:268: UserWarning: target_host parameter is going to be deprecated. Please pass in tvm.target.Target(target, host=target_host) instead.
3. "target_host parameter is going to be deprecated. "
4. Evaluate inference time cost...
5. Execution time summary:
6. mean (ms)   median (ms)    max (ms)     min (ms)     std (ms)
7. 10.0003 9.9944 10.0327 9.9738 0.0244

复制代码

其他本事​

• 在调优过程中，auto-scheduler 需要编译很多步调，并从中提取特征。这部门会占用大量 CPU 资源，所以推荐使用多核的高性能 CPU，加快搜刮速度。
  
• 可以用 python3 -m tvm.auto_scheduler.measure_record --mode distill -i log.json 提取大日志文件，并仅生存最有效的记录。
  
• 可以从以前的日志文件规复搜刮，只需要在函数 run_tuning 中创建任务调度步调时添加一个新参数 load_log_file。好比，tuner = auto_scheduler.TaskScheduler(tasks, task_weights, load_log_file=log_file)
  
• 若有多个 target CPU，则可以将所有这些 CPU 用于并行化测试。查看这 部门 了解如何使用 RPC 跟踪器和 RPC 服务器。要在 auto-scheduler 中使用 RPC 跟踪器，请将 TuningOptions 中的 runner 更换为 auto_scheduler.RPCRunner。
  

下载 Python 源代码：tune_network_cuda.py
下载 Jupyter Notebook：tune_network_cuda.ipynb

免责声明：如果侵犯了您的权益，请联系站长，我们会及时删除侵权内容，谢谢合作！更多信息从访问主页：qidao123.com:ToB企服之家，中国第一个企服评测及商务社交产业平台。