pytest分布式执行插件 pytest-xdist 的高级用法

络腮胡菲菲  金牌会员 | 2022-9-16 17:18:16 | 显示全部楼层 | 阅读模式
打印 上一主题 下一主题
楼主

主题 922|帖子 922|积分 2766

想要使用多个CPU核心来进行测试，可以使用 -n 参数( 或者 --numprocesses)
(使用8个核心来跑测试用例)
  1. pytest -n 8
复制代码
使用 -n auto 参数可以利用电脑的所有核心来跑测试用例
测试时使用的算法可以根据--dist命令参数定制：

  • --dist load(默认选项)：给每个CPU核心随机分配用例，不保证执行顺序。
  • --dist loadscope：对于测试函数，测试按模块分组，对于测试方法，测试按类分组。每组作为一个整体分配给可用的worker。这保证了组中的所有测试都在同一进程中运行。如果的模块级或类级fixtures，这将非常有用。按类分组优先于按模块分组。
  • --dist loadfile: 测试用例按其所在文件分组。每组作为一个整体分配给可用的worker。这保证了文件中的所有测试都在同一个辅助进程中运行。
  • --dist loadgroup: 测试按xdist_group标记分组。每组作为一个整体分配给可用的执行器。这保证了具有相同xdist_ group名称的所有测试都在同一个worker中运行。
  1. @pytest.mark.xdist_group(name="group1")
  2. def test1():
  3.     pass
  5. class TestA:
  6.     @pytest.mark.xdist_group("group1")
  7.     def test2():
  8.         pass-
复制代码
这将确保test1和TestA:：test2将在同一个worker中运行。没有xdist_ group标记的测试在--dist=load模式下正常运行。

  • --dist no：正常的pytest执行模式，一次运行一个测试（完全没有分发）。
例子：
项目目录结构

xdist_test.py
  1. import logging
  3. import pytest
  6. class TestXdist(object):
  8.     @pytest.mark.xdist_group("group1")
  9.     def test_one(self):
  10.         logging.info("1")
  11.         assert True
  13.     @pytest.mark.xdist_group("group1")
  14.     def test_two(self):
  15.         logging.info("2")
  16.         assert True
  18.     @pytest.mark.xdist_group("group2")
  19.     def test_three(self):
  20.         logging.info("3")
  21.         assert True
  23.     @pytest.mark.xdist_group("group2")
  24.     def test_four(self):
  25.         logging.info("4")
  26.         assert True
复制代码
xdist_dummy_test.py
  1. import logging
  3. import pytest
  6. class TestXdist(object):
  7.     @pytest.mark.run(order=1)
  8.     @pytest.mark.xdist_group("group1")
  9.     def test_dummy_one(self):
  10.         logging.info("d1")
  11.         assert True
  13.     @pytest.mark.xdist_group("group1")
  14.     @pytest.mark.run(order=2)
  15.     def test_dummy_two(self):
  16.         logging.info("d2")
  17.         assert True
  19.     @pytest.mark.run(order=3)
  20.     @pytest.mark.xdist_group("group2")
  21.     def test_dummy_three(self):
  22.         logging.info("d3")
  23.         assert True
  25.     @pytest.mark.xdist_group("group2")
  26.     @pytest.mark.run(order=4)
  27.     def test_dummy_four(self):
  28.         logging.info("d4")
  29.         assert True
复制代码
  1. (venv) ➜  pytest pytest -n auto --dist loadscope
  2. =========================================================================== test session starts ===========================================================================
  3. platform darwin -- Python 3.8.9, pytest-7.1.2, pluggy-1.0.0
  4. rootdir: /Users/spock/PycharmProjects/pytest, configfile: pytest.ini
  5. plugins: xdist-2.5.0, forked-1.4.0, ordering-0.6
  6. [gw0] Python 3.8.9 (default, May 17 2022, 12:55:41)  -- [Clang 13.1.6 (clang-1316.0.21.2.5)]
  7. [gw1] Python 3.8.9 (default, May 17 2022, 12:55:41)  -- [Clang 13.1.6 (clang-1316.0.21.2.5)]
  8. [gw2] Python 3.8.9 (default, May 17 2022, 12:55:41)  -- [Clang 13.1.6 (clang-1316.0.21.2.5)]
  9. [gw3] Python 3.8.9 (default, May 17 2022, 12:55:41)  -- [Clang 13.1.6 (clang-1316.0.21.2.5)]
  10. [gw4] Python 3.8.9 (default, May 17 2022, 12:55:41)  -- [Clang 13.1.6 (clang-1316.0.21.2.5)]
  11. [gw5] Python 3.8.9 (default, May 17 2022, 12:55:41)  -- [Clang 13.1.6 (clang-1316.0.21.2.5)]
  12. [gw6] Python 3.8.9 (default, May 17 2022, 12:55:41)  -- [Clang 13.1.6 (clang-1316.0.21.2.5)]
  13. [gw7] Python 3.8.9 (default, May 17 2022, 12:55:41)  -- [Clang 13.1.6 (clang-1316.0.21.2.5)]
  14. gw0 [8] / gw1 [8] / gw2 [8] / gw3 [8] / gw4 [8] / gw5 [8] / gw6 [8] / gw7 [8]
  15. scheduling tests via LoadScopeScheduling
  17. test2/xdist_dummy_test.py::TestXdist::test_dummy_one
  18. test/xdist_test.py::TestXdist::test_one
  19. [gw0] [ 12%] PASSED test2/xdist_dummy_test.py::TestXdist::test_dummy_one
  20. [gw1] [ 25%] PASSED test/xdist_test.py::TestXdist::test_one
  21. test2/xdist_dummy_test.py::TestXdist::test_dummy_two
  22. test/xdist_test.py::TestXdist::test_two
  23. [gw1] [ 37%] PASSED test/xdist_test.py::TestXdist::test_two
  24. test/xdist_test.py::TestXdist::test_three
  25. [gw0] [ 50%] PASSED test2/xdist_dummy_test.py::TestXdist::test_dummy_two
  26. test2/xdist_dummy_test.py::TestXdist::test_dummy_three
  27. [gw1] [ 62%] PASSED test/xdist_test.py::TestXdist::test_three
  28. test/xdist_test.py::TestXdist::test_four
  29. [gw0] [ 75%] PASSED test2/xdist_dummy_test.py::TestXdist::test_dummy_three
  30. test2/xdist_dummy_test.py::TestXdist::test_dummy_four
  31. [gw1] [ 87%] PASSED test/xdist_test.py::TestXdist::test_four
  32. [gw0] [100%] PASSED test2/xdist_dummy_test.py::TestXdist::test_dummy_four
  33. ============================================================================ 8 passed in 0.40s ============================================================================
复制代码
  1. (venv) ➜  pytest pytest -n auto --dist loadfile
  2. =========================================================================== test session starts ===========================================================================
  3. platform darwin -- Python 3.8.9, pytest-7.1.2, pluggy-1.0.0
  4. rootdir: /Users/spock/PycharmProjects/pytest, configfile: pytest.ini
  5. plugins: xdist-2.5.0, forked-1.4.0, ordering-0.6
  6. [gw0] Python 3.8.9 (default, May 17 2022, 12:55:41)  -- [Clang 13.1.6 (clang-1316.0.21.2.5)]
  7. [gw1] Python 3.8.9 (default, May 17 2022, 12:55:41)  -- [Clang 13.1.6 (clang-1316.0.21.2.5)]
  8. [gw2] Python 3.8.9 (default, May 17 2022, 12:55:41)  -- [Clang 13.1.6 (clang-1316.0.21.2.5)]
  9. [gw3] Python 3.8.9 (default, May 17 2022, 12:55:41)  -- [Clang 13.1.6 (clang-1316.0.21.2.5)]
  10. [gw4] Python 3.8.9 (default, May 17 2022, 12:55:41)  -- [Clang 13.1.6 (clang-1316.0.21.2.5)]
  11. [gw5] Python 3.8.9 (default, May 17 2022, 12:55:41)  -- [Clang 13.1.6 (clang-1316.0.21.2.5)]
  12. [gw6] Python 3.8.9 (default, May 17 2022, 12:55:41)  -- [Clang 13.1.6 (clang-1316.0.21.2.5)]
  13. [gw7] Python 3.8.9 (default, May 17 2022, 12:55:41)  -- [Clang 13.1.6 (clang-1316.0.21.2.5)]
  14. gw0 [8] / gw1 [8] / gw2 [8] / gw3 [8] / gw4 [8] / gw5 [8] / gw6 [8] / gw7 [8]
  15. scheduling tests via LoadFileScheduling
  17. test/xdist_test.py::TestXdist::test_one
  18. test2/xdist_dummy_test.py::TestXdist::test_dummy_one
  19. [gw1] [ 12%] PASSED test/xdist_test.py::TestXdist::test_one
  20. [gw0] [ 25%] PASSED test2/xdist_dummy_test.py::TestXdist::test_dummy_one
  21. test/xdist_test.py::TestXdist::test_two
  22. test2/xdist_dummy_test.py::TestXdist::test_dummy_two
  23. [gw1] [ 37%] PASSED test/xdist_test.py::TestXdist::test_two
  24. [gw0] [ 50%] PASSED test2/xdist_dummy_test.py::TestXdist::test_dummy_two
  25. test2/xdist_dummy_test.py::TestXdist::test_dummy_three
  26. test/xdist_test.py::TestXdist::test_three
  27. [gw1] [ 62%] PASSED test/xdist_test.py::TestXdist::test_three
  28. [gw0] [ 75%] PASSED test2/xdist_dummy_test.py::TestXdist::test_dummy_three
  29. test2/xdist_dummy_test.py::TestXdist::test_dummy_four
  30. test/xdist_test.py::TestXdist::test_four
  31. [gw1] [ 87%] PASSED test/xdist_test.py::TestXdist::test_four
  32. [gw0] [100%] PASSED test2/xdist_dummy_test.py::TestXdist::test_dummy_four
  34. ============================================================================ 8 passed in 0.38s ============================================================================
复制代码
  1. (venv) ➜  pytest pytest -n auto --dist loadgroup
  2. =========================================================================== test session starts ===========================================================================
  3. platform darwin -- Python 3.8.9, pytest-7.1.2, pluggy-1.0.0
  4. rootdir: /Users/spock/PycharmProjects/pytest, configfile: pytest.ini
  5. plugins: xdist-2.5.0, forked-1.4.0, ordering-0.6
  6. [gw0] Python 3.8.9 (default, May 17 2022, 12:55:41)  -- [Clang 13.1.6 (clang-1316.0.21.2.5)]
  7. [gw1] Python 3.8.9 (default, May 17 2022, 12:55:41)  -- [Clang 13.1.6 (clang-1316.0.21.2.5)]
  8. [gw2] Python 3.8.9 (default, May 17 2022, 12:55:41)  -- [Clang 13.1.6 (clang-1316.0.21.2.5)]
  9. [gw3] Python 3.8.9 (default, May 17 2022, 12:55:41)  -- [Clang 13.1.6 (clang-1316.0.21.2.5)]
  10. [gw4] Python 3.8.9 (default, May 17 2022, 12:55:41)  -- [Clang 13.1.6 (clang-1316.0.21.2.5)]
  11. [gw5] Python 3.8.9 (default, May 17 2022, 12:55:41)  -- [Clang 13.1.6 (clang-1316.0.21.2.5)]
  12. [gw6] Python 3.8.9 (default, May 17 2022, 12:55:41)  -- [Clang 13.1.6 (clang-1316.0.21.2.5)]
  13. [gw7] Python 3.8.9 (default, May 17 2022, 12:55:41)  -- [Clang 13.1.6 (clang-1316.0.21.2.5)]
  14. gw0 [8] / gw1 [8] / gw2 [8] / gw3 [8] / gw4 [8] / gw5 [8] / gw6 [8] / gw7 [8]
  15. scheduling tests via LoadGroupScheduling
  17. test2/xdist_dummy_test.py::TestXdist::test_dummy_three@group2
  18. test2/xdist_dummy_test.py::TestXdist::test_dummy_one@group1
  19. [gw1] [ 12%] PASSED test2/xdist_dummy_test.py::TestXdist::test_dummy_three@group2
  20. test2/xdist_dummy_test.py::TestXdist::test_dummy_four@group2
  21. [gw0] [ 25%] PASSED test2/xdist_dummy_test.py::TestXdist::test_dummy_one@group1
  22. [gw1] [ 37%] PASSED test2/xdist_dummy_test.py::TestXdist::test_dummy_four@group2
  23. test2/xdist_dummy_test.py::TestXdist::test_dummy_two@group1
  24. [gw0] [ 50%] PASSED test2/xdist_dummy_test.py::TestXdist::test_dummy_two@group1
  25. test/xdist_test.py::TestXdist::test_three@group2
  26. [gw1] [ 62%] PASSED test/xdist_test.py::TestXdist::test_three@group2
  27. test/xdist_test.py::TestXdist::test_four@group2
  28. [gw1] [ 75%] PASSED test/xdist_test.py::TestXdist::test_four@group2
  29. test/xdist_test.py::TestXdist::test_one@group1
  30. [gw0] [ 87%] PASSED test/xdist_test.py::TestXdist::test_one@group1
  31. test/xdist_test.py::TestXdist::test_two@group1
  32. [gw0] [100%] PASSED test/xdist_test.py::TestXdist::test_two@group1
  34. ============================================================================ 8 passed in 0.40s ============================================================================
复制代码
免责声明：如果侵犯了您的权益，请联系站长，我们会及时删除侵权内容，谢谢合作！

本帖子中包含更多资源

您需要 登录 才可以下载或查看，没有账号？立即注册

x
回复

使用道具 举报

0 个回复

正序浏览
返回列表

快速回复

高级模式
B Color Image Link Quote Code Smilies
您需要登录后才可以回帖 登录 or 立即注册

本版积分规则

络腮胡菲菲

金牌会员
这个人很懒什么都没写!

楼主热帖

标签云

存储 服务器
微信订阅号
微信服务号
微信客服
小程序
H5
关于我们 商务合作 网站地图
©Copyright 2022-2024 杭州祥升科技有限公司 版权所有 浙ICP备20004199号
快速回复 返回顶部 返回列表