Mercurial > public > mercurial-scm > hg-stable

annotate contrib/automation/hgautomation/cli.py @ 44753:9d441f820c8b stable

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
automation: support building Windows wheels for Python 3.7 and 3.8 The time has come to support Python 3 on Windows. Let's teach our automation code to produce Windows wheels for Python 3.7 and 3.8. We could theoretically support 3.5 and 3.6. But I don't think it is worth it. People on Windows generally use the Mercurial installers, not wheels. And I'd prefer we limit variability and not have to worry about supporting earlier Python versions if it can be helped. As part of this, we change the invocation of pip to `python.exe -m pip`, as this is what is being recommended in Python docs these days. And it seemed to be required to avoid a weird build error. Why, I'm not sure. But it looks like pip was having trouble finding a Visual Studio files when invoked as `pip.exe` but not when using `python.exe -m pip`. Who knows. Differential Revision: https://phab.mercurial-scm.org/D8478
author Gregory Szorc <gregory.szorc@gmail.com>
date Thu, 23 Apr 2020 17:24:37 -0700
parents 2372284d9457
children 802ee93c205d
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
42024
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
1 # cli.py - Command line interface for automation
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
2 #
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
3 # Copyright 2019 Gregory Szorc <gregory.szorc@gmail.com>
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
4 #
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
5 # This software may be used and distributed according to the terms of the
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
6 # GNU General Public License version 2 or any later version.
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
7
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
8 # no-check-code because Python 3 native.
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
9
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
10 import argparse
42312
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42310
diff changeset
11 import concurrent.futures as futures
42024
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
12 import os
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
13 import pathlib
42312
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42310
diff changeset
14 import time
42024
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
15
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
16 from . import (
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
17 aws,
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
18 HGAutomation,
42312
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42310
diff changeset
19 linux,
43057
c5c502bd1f70 automation: add a command to submit to a Try server
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43055
diff changeset
20 try_server,
42024
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
21 windows,
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
22 )
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
23
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
24
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
25 SOURCE_ROOT = pathlib.Path(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
26 os.path.abspath(__file__)
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
27 ).parent.parent.parent.parent
42024
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
28 DIST_PATH = SOURCE_ROOT / 'dist'
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
29
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
30
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
31 def bootstrap_linux_dev(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
32 hga: HGAutomation, aws_region, distros=None, parallel=False
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
33 ):
42312
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42310
diff changeset
34 c = hga.aws_connection(aws_region)
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42310
diff changeset
35
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42310
diff changeset
36 if distros:
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42310
diff changeset
37 distros = distros.split(',')
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42310
diff changeset
38 else:
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42310
diff changeset
39 distros = sorted(linux.DISTROS)
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42310
diff changeset
40
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42310
diff changeset
41 # TODO There is a wonky interaction involving KeyboardInterrupt whereby
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42310
diff changeset
42 # the context manager that is supposed to terminate the temporary EC2
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42310
diff changeset
43 # instance doesn't run. Until we fix this, make parallel building opt-in
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42310
diff changeset
44 # so we don't orphan instances.
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42310
diff changeset
45 if parallel:
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42310
diff changeset
46 fs = []
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42310
diff changeset
47
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42310
diff changeset
48 with futures.ThreadPoolExecutor(len(distros)) as e:
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42310
diff changeset
49 for distro in distros:
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42310
diff changeset
50 fs.append(e.submit(aws.ensure_linux_dev_ami, c, distro=distro))
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42310
diff changeset
51
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42310
diff changeset
52 for f in fs:
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42310
diff changeset
53 f.result()
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42310
diff changeset
54 else:
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42310
diff changeset
55 for distro in distros:
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42310
diff changeset
56 aws.ensure_linux_dev_ami(c, distro=distro)
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42310
diff changeset
57
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42310
diff changeset
58
42648
d80edcb0b30c automation: make Windows base image name configurable
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42312
diff changeset
59 def bootstrap_windows_dev(hga: HGAutomation, aws_region, base_image_name):
42024
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
60 c = hga.aws_connection(aws_region)
42648
d80edcb0b30c automation: make Windows base image name configurable
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42312
diff changeset
61 image = aws.ensure_windows_dev_ami(c, base_image_name=base_image_name)
42024
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
62 print('Windows development AMI available as %s' % image.id)
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
63
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
64
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
65 def build_inno(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
66 hga: HGAutomation, aws_region, arch, revision, version, base_image_name
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
67 ):
42024
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
68 c = hga.aws_connection(aws_region)
42648
d80edcb0b30c automation: make Windows base image name configurable
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42312
diff changeset
69 image = aws.ensure_windows_dev_ami(c, base_image_name=base_image_name)
42024
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
70 DIST_PATH.mkdir(exist_ok=True)
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
71
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
72 with aws.temporary_windows_dev_instances(c, image, 't3.medium') as insts:
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
73 instance = insts[0]
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
74
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
75 windows.synchronize_hg(SOURCE_ROOT, revision, instance)
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
76
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
77 for a in arch:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
78 windows.build_inno_installer(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
79 instance.winrm_client, a, DIST_PATH, version=version
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
80 )
42024
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
81
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
82
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
83 def build_wix(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
84 hga: HGAutomation, aws_region, arch, revision, version, base_image_name
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
85 ):
42024
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
86 c = hga.aws_connection(aws_region)
42648
d80edcb0b30c automation: make Windows base image name configurable
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42312
diff changeset
87 image = aws.ensure_windows_dev_ami(c, base_image_name=base_image_name)
42024
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
88 DIST_PATH.mkdir(exist_ok=True)
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
89
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
90 with aws.temporary_windows_dev_instances(c, image, 't3.medium') as insts:
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
91 instance = insts[0]
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
92
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
93 windows.synchronize_hg(SOURCE_ROOT, revision, instance)
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
94
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
95 for a in arch:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
96 windows.build_wix_installer(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
97 instance.winrm_client, a, DIST_PATH, version=version
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
98 )
42024
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
99
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
100
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
101 def build_windows_wheel(
44753
9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43076
diff changeset
102 hga: HGAutomation,
9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43076
diff changeset
103 aws_region,
9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43076
diff changeset
104 python_version,
9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43076
diff changeset
105 arch,
9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43076
diff changeset
106 revision,
9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43076
diff changeset
107 base_image_name,
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
108 ):
42024
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
109 c = hga.aws_connection(aws_region)
42648
d80edcb0b30c automation: make Windows base image name configurable
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42312
diff changeset
110 image = aws.ensure_windows_dev_ami(c, base_image_name=base_image_name)
42024
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
111 DIST_PATH.mkdir(exist_ok=True)
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
112
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
113 with aws.temporary_windows_dev_instances(c, image, 't3.medium') as insts:
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
114 instance = insts[0]
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
115
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
116 windows.synchronize_hg(SOURCE_ROOT, revision, instance)
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
117
44753
9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43076
diff changeset
118 for py_version in python_version:
9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43076
diff changeset
119 for a in arch:
9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43076
diff changeset
120 windows.build_wheel(
9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43076
diff changeset
121 instance.winrm_client, py_version, a, DIST_PATH
9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43076
diff changeset
122 )
42024
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
123
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
124
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
125 def build_all_windows_packages(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
126 hga: HGAutomation, aws_region, revision, version, base_image_name
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
127 ):
42024
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
128 c = hga.aws_connection(aws_region)
42648
d80edcb0b30c automation: make Windows base image name configurable
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42312
diff changeset
129 image = aws.ensure_windows_dev_ami(c, base_image_name=base_image_name)
42024
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
130 DIST_PATH.mkdir(exist_ok=True)
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
131
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
132 with aws.temporary_windows_dev_instances(c, image, 't3.medium') as insts:
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
133 instance = insts[0]
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
134
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
135 winrm_client = instance.winrm_client
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
136
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
137 windows.synchronize_hg(SOURCE_ROOT, revision, instance)
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
138
44753
9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43076
diff changeset
139 for py_version in ("2.7", "3.7", "3.8"):
9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43076
diff changeset
140 for arch in ("x86", "x64"):
9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43076
diff changeset
141 windows.purge_hg(winrm_client)
9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43076
diff changeset
142 windows.build_wheel(
9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43076
diff changeset
143 winrm_client,
9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43076
diff changeset
144 python_version=py_version,
9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43076
diff changeset
145 arch=arch,
9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43076
diff changeset
146 dest_path=DIST_PATH,
9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43076
diff changeset
147 )
9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43076
diff changeset
148
42024
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
149 for arch in ('x86', 'x64'):
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
150 windows.purge_hg(winrm_client)
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
151 windows.build_inno_installer(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
152 winrm_client, arch, DIST_PATH, version=version
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
153 )
42024
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
154 windows.purge_hg(winrm_client)
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
155 windows.build_wix_installer(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
156 winrm_client, arch, DIST_PATH, version=version
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
157 )
42024
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
158
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
159
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
160 def terminate_ec2_instances(hga: HGAutomation, aws_region):
42304
dd6a9723ae2b automation: don't create resources when deleting things
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42024
diff changeset
161 c = hga.aws_connection(aws_region, ensure_ec2_state=False)
42024
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
162 aws.terminate_ec2_instances(c.ec2resource)
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
163
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
164
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
165 def purge_ec2_resources(hga: HGAutomation, aws_region):
42304
dd6a9723ae2b automation: don't create resources when deleting things
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42024
diff changeset
166 c = hga.aws_connection(aws_region, ensure_ec2_state=False)
42024
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
167 aws.remove_resources(c)
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
168
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
169
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
170 def run_tests_linux(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
171 hga: HGAutomation,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
172 aws_region,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
173 instance_type,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
174 python_version,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
175 test_flags,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
176 distro,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
177 filesystem,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
178 ):
42312
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42310
diff changeset
179 c = hga.aws_connection(aws_region)
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42310
diff changeset
180 image = aws.ensure_linux_dev_ami(c, distro=distro)
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42310
diff changeset
181
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42310
diff changeset
182 t_start = time.time()
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42310
diff changeset
183
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42310
diff changeset
184 ensure_extra_volume = filesystem not in ('default', 'tmpfs')
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42310
diff changeset
185
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42310
diff changeset
186 with aws.temporary_linux_dev_instances(
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
187 c, image, instance_type, ensure_extra_volume=ensure_extra_volume
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
188 ) as insts:
42312
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42310
diff changeset
189
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42310
diff changeset
190 instance = insts[0]
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42310
diff changeset
191
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
192 linux.prepare_exec_environment(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
193 instance.ssh_client, filesystem=filesystem
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
194 )
42312
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42310
diff changeset
195 linux.synchronize_hg(SOURCE_ROOT, instance, '.')
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42310
diff changeset
196 t_prepared = time.time()
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
197 linux.run_tests(instance.ssh_client, python_version, test_flags)
42312
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42310
diff changeset
198 t_done = time.time()
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42310
diff changeset
199
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42310
diff changeset
200 t_setup = t_prepared - t_start
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42310
diff changeset
201 t_all = t_done - t_start
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42310
diff changeset
202
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42310
diff changeset
203 print(
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42310
diff changeset
204 'total time: %.1fs; setup: %.1fs; tests: %.1fs; setup overhead: %.1f%%'
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
205 % (t_all, t_setup, t_done - t_prepared, t_setup / t_all * 100.0)
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
206 )
42312
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42310
diff changeset
207
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42310
diff changeset
208
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
209 def run_tests_windows(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
210 hga: HGAutomation,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
211 aws_region,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
212 instance_type,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
213 python_version,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
214 arch,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
215 test_flags,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
216 base_image_name,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
217 ):
42024
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
218 c = hga.aws_connection(aws_region)
42648
d80edcb0b30c automation: make Windows base image name configurable
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42312
diff changeset
219 image = aws.ensure_windows_dev_ami(c, base_image_name=base_image_name)
42024
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
220
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
221 with aws.temporary_windows_dev_instances(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
222 c, image, instance_type, disable_antivirus=True
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
223 ) as insts:
42024
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
224 instance = insts[0]
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
225
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
226 windows.synchronize_hg(SOURCE_ROOT, '.', instance)
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
227 windows.run_tests(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
228 instance.winrm_client, python_version, arch, test_flags
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
229 )
42024
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
230
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
231
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
232 def publish_windows_artifacts(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
233 hg: HGAutomation,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
234 aws_region,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
235 version: str,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
236 pypi: bool,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
237 mercurial_scm_org: bool,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
238 ssh_username: str,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
239 ):
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
240 windows.publish_artifacts(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
241 DIST_PATH,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
242 version,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
243 pypi=pypi,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
244 mercurial_scm_org=mercurial_scm_org,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
245 ssh_username=ssh_username,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
246 )
42913
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42648
diff changeset
247
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42648
diff changeset
248
43057
c5c502bd1f70 automation: add a command to submit to a Try server
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43055
diff changeset
249 def run_try(hga: HGAutomation, aws_region: str, rev: str):
c5c502bd1f70 automation: add a command to submit to a Try server
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43055
diff changeset
250 c = hga.aws_connection(aws_region, ensure_ec2_state=False)
c5c502bd1f70 automation: add a command to submit to a Try server
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43055
diff changeset
251 try_server.trigger_try(c, rev=rev)
c5c502bd1f70 automation: add a command to submit to a Try server
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43055
diff changeset
252
c5c502bd1f70 automation: add a command to submit to a Try server
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43055
diff changeset
253
42024
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
254 def get_parser():
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
255 parser = argparse.ArgumentParser()
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
256
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
257 parser.add_argument(
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
258 '--state-path',
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
259 default='~/.hgautomation',
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
260 help='Path for local state files',
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
261 )
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
262 parser.add_argument(
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
263 '--aws-region', help='AWS region to use', default='us-west-2',
42024
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
264 )
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
265
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
266 subparsers = parser.add_subparsers()
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
267
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
268 sp = subparsers.add_parser(
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
269 'bootstrap-linux-dev', help='Bootstrap Linux development environments',
42312
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42310
diff changeset
270 )
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42310
diff changeset
271 sp.add_argument(
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
272 '--distros', help='Comma delimited list of distros to bootstrap',
42312
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42310
diff changeset
273 )
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42310
diff changeset
274 sp.add_argument(
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42310
diff changeset
275 '--parallel',
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42310
diff changeset
276 action='store_true',
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
277 help='Generate AMIs in parallel (not CTRL-c safe)',
42312
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42310
diff changeset
278 )
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42310
diff changeset
279 sp.set_defaults(func=bootstrap_linux_dev)
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42310
diff changeset
280
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42310
diff changeset
281 sp = subparsers.add_parser(
42024
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
282 'bootstrap-windows-dev',
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
283 help='Bootstrap the Windows development environment',
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
284 )
42648
d80edcb0b30c automation: make Windows base image name configurable
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42312
diff changeset
285 sp.add_argument(
d80edcb0b30c automation: make Windows base image name configurable
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42312
diff changeset
286 '--base-image-name',
d80edcb0b30c automation: make Windows base image name configurable
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42312
diff changeset
287 help='AMI name of base image',
d80edcb0b30c automation: make Windows base image name configurable
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42312
diff changeset
288 default=aws.WINDOWS_BASE_IMAGE_NAME,
d80edcb0b30c automation: make Windows base image name configurable
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42312
diff changeset
289 )
42024
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
290 sp.set_defaults(func=bootstrap_windows_dev)
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
291
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
292 sp = subparsers.add_parser(
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
293 'build-all-windows-packages', help='Build all Windows packages',
42024
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
294 )
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
295 sp.add_argument(
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
296 '--revision', help='Mercurial revision to build', default='.',
42024
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
297 )
42310
d137a3d5ad41 automation: add --version argument to build-all-windows-packages
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42304
diff changeset
298 sp.add_argument(
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
299 '--version', help='Mercurial version string to use',
42310
d137a3d5ad41 automation: add --version argument to build-all-windows-packages
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42304
diff changeset
300 )
42648
d80edcb0b30c automation: make Windows base image name configurable
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42312
diff changeset
301 sp.add_argument(
d80edcb0b30c automation: make Windows base image name configurable
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42312
diff changeset
302 '--base-image-name',
d80edcb0b30c automation: make Windows base image name configurable
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42312
diff changeset
303 help='AMI name of base image',
d80edcb0b30c automation: make Windows base image name configurable
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42312
diff changeset
304 default=aws.WINDOWS_BASE_IMAGE_NAME,
d80edcb0b30c automation: make Windows base image name configurable
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42312
diff changeset
305 )
42024
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
306 sp.set_defaults(func=build_all_windows_packages)
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
307
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
308 sp = subparsers.add_parser(
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
309 'build-inno', help='Build Inno Setup installer(s)',
42024
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
310 )
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
311 sp.add_argument(
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
312 '--arch',
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
313 help='Architecture to build for',
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
314 choices={'x86', 'x64'},
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
315 nargs='*',
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
316 default=['x64'],
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
317 )
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
318 sp.add_argument(
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
319 '--revision', help='Mercurial revision to build', default='.',
42024
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
320 )
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
321 sp.add_argument(
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
322 '--version', help='Mercurial version string to use in installer',
42024
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
323 )
42648
d80edcb0b30c automation: make Windows base image name configurable
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42312
diff changeset
324 sp.add_argument(
d80edcb0b30c automation: make Windows base image name configurable
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42312
diff changeset
325 '--base-image-name',
d80edcb0b30c automation: make Windows base image name configurable
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42312
diff changeset
326 help='AMI name of base image',
d80edcb0b30c automation: make Windows base image name configurable
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42312
diff changeset
327 default=aws.WINDOWS_BASE_IMAGE_NAME,
d80edcb0b30c automation: make Windows base image name configurable
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42312
diff changeset
328 )
42024
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
329 sp.set_defaults(func=build_inno)
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
330
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
331 sp = subparsers.add_parser(
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
332 'build-windows-wheel', help='Build Windows wheel(s)',
42024
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
333 )
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
334 sp.add_argument(
44753
9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43076
diff changeset
335 '--python-version',
9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43076
diff changeset
336 help='Python version to build for',
9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43076
diff changeset
337 choices={'2.7', '3.7', '3.8'},
9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43076
diff changeset
338 nargs='*',
9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43076
diff changeset
339 default=['3.8'],
9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43076
diff changeset
340 )
9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43076
diff changeset
341 sp.add_argument(
42024
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
342 '--arch',
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
343 help='Architecture to build for',
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
344 choices={'x86', 'x64'},
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
345 nargs='*',
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
346 default=['x64'],
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
347 )
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
348 sp.add_argument(
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
349 '--revision', help='Mercurial revision to build', default='.',
42024
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
350 )
42648
d80edcb0b30c automation: make Windows base image name configurable
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42312
diff changeset
351 sp.add_argument(
d80edcb0b30c automation: make Windows base image name configurable
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42312
diff changeset
352 '--base-image-name',
d80edcb0b30c automation: make Windows base image name configurable
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42312
diff changeset
353 help='AMI name of base image',
d80edcb0b30c automation: make Windows base image name configurable
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42312
diff changeset
354 default=aws.WINDOWS_BASE_IMAGE_NAME,
d80edcb0b30c automation: make Windows base image name configurable
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42312
diff changeset
355 )
42024
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
356 sp.set_defaults(func=build_windows_wheel)
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
357
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
358 sp = subparsers.add_parser('build-wix', help='Build WiX installer(s)')
42024
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
359 sp.add_argument(
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
360 '--arch',
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
361 help='Architecture to build for',
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
362 choices={'x86', 'x64'},
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
363 nargs='*',
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
364 default=['x64'],
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
365 )
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
366 sp.add_argument(
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
367 '--revision', help='Mercurial revision to build', default='.',
42024
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
368 )
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
369 sp.add_argument(
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
370 '--version', help='Mercurial version string to use in installer',
42024
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
371 )
42648
d80edcb0b30c automation: make Windows base image name configurable
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42312
diff changeset
372 sp.add_argument(
d80edcb0b30c automation: make Windows base image name configurable
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42312
diff changeset
373 '--base-image-name',
d80edcb0b30c automation: make Windows base image name configurable
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42312
diff changeset
374 help='AMI name of base image',
d80edcb0b30c automation: make Windows base image name configurable
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42312
diff changeset
375 default=aws.WINDOWS_BASE_IMAGE_NAME,
d80edcb0b30c automation: make Windows base image name configurable
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42312
diff changeset
376 )
42024
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
377 sp.set_defaults(func=build_wix)
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
378
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
379 sp = subparsers.add_parser(
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
380 'terminate-ec2-instances',
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
381 help='Terminate all active EC2 instances managed by us',
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
382 )
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
383 sp.set_defaults(func=terminate_ec2_instances)
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
384
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
385 sp = subparsers.add_parser(
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
386 'purge-ec2-resources', help='Purge all EC2 resources managed by us',
42024
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
387 )
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
388 sp.set_defaults(func=purge_ec2_resources)
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
389
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
390 sp = subparsers.add_parser('run-tests-linux', help='Run tests on Linux',)
42312
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42310
diff changeset
391 sp.add_argument(
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42310
diff changeset
392 '--distro',
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42310
diff changeset
393 help='Linux distribution to run tests on',
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42310
diff changeset
394 choices=linux.DISTROS,
43020
d1d919f679f7 automation: support and use Debian Buster by default
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42913
diff changeset
395 default='debian10',
42312
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42310
diff changeset
396 )
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42310
diff changeset
397 sp.add_argument(
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42310
diff changeset
398 '--filesystem',
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42310
diff changeset
399 help='Filesystem type to use',
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42310
diff changeset
400 choices={'btrfs', 'default', 'ext3', 'ext4', 'jfs', 'tmpfs', 'xfs'},
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42310
diff changeset
401 default='default',
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42310
diff changeset
402 )
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42310
diff changeset
403 sp.add_argument(
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42310
diff changeset
404 '--instance-type',
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42310
diff changeset
405 help='EC2 instance type to use',
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42310
diff changeset
406 default='c5.9xlarge',
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42310
diff changeset
407 )
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42310
diff changeset
408 sp.add_argument(
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42310
diff changeset
409 '--python-version',
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42310
diff changeset
410 help='Python version to use',
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
411 choices={
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
412 'system2',
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
413 'system3',
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
414 '2.7',
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
415 '3.5',
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
416 '3.6',
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
417 '3.7',
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
418 '3.8',
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
419 'pypy',
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
420 'pypy3.5',
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
421 'pypy3.6',
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
422 },
42312
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42310
diff changeset
423 default='system2',
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42310
diff changeset
424 )
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42310
diff changeset
425 sp.add_argument(
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42310
diff changeset
426 'test_flags',
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42310
diff changeset
427 help='Extra command line flags to pass to run-tests.py',
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42310
diff changeset
428 nargs='*',
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42310
diff changeset
429 )
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42310
diff changeset
430 sp.set_defaults(func=run_tests_linux)
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42310
diff changeset
431
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42310
diff changeset
432 sp = subparsers.add_parser(
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
433 'run-tests-windows', help='Run tests on Windows',
42024
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
434 )
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
435 sp.add_argument(
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
436 '--instance-type', help='EC2 instance type to use', default='t3.medium',
42024
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
437 )
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
438 sp.add_argument(
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
439 '--python-version',
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
440 help='Python version to use',
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
441 choices={'2.7', '3.5', '3.6', '3.7', '3.8'},
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
442 default='2.7',
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
443 )
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
444 sp.add_argument(
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
445 '--arch',
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
446 help='Architecture to test',
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
447 choices={'x86', 'x64'},
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
448 default='x64',
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
449 )
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
450 sp.add_argument(
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
451 '--test-flags', help='Extra command line flags to pass to run-tests.py',
42024
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
452 )
42648
d80edcb0b30c automation: make Windows base image name configurable
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42312
diff changeset
453 sp.add_argument(
d80edcb0b30c automation: make Windows base image name configurable
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42312
diff changeset
454 '--base-image-name',
d80edcb0b30c automation: make Windows base image name configurable
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42312
diff changeset
455 help='AMI name of base image',
d80edcb0b30c automation: make Windows base image name configurable
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42312
diff changeset
456 default=aws.WINDOWS_BASE_IMAGE_NAME,
d80edcb0b30c automation: make Windows base image name configurable
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42312
diff changeset
457 )
42024
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
458 sp.set_defaults(func=run_tests_windows)
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
459
42913
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42648
diff changeset
460 sp = subparsers.add_parser(
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42648
diff changeset
461 'publish-windows-artifacts',
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
462 help='Publish built Windows artifacts (wheels, installers, etc)',
42913
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42648
diff changeset
463 )
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42648
diff changeset
464 sp.add_argument(
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42648
diff changeset
465 '--no-pypi',
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42648
diff changeset
466 dest='pypi',
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42648
diff changeset
467 action='store_false',
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42648
diff changeset
468 default=True,
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42648
diff changeset
469 help='Skip uploading to PyPI',
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42648
diff changeset
470 )
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42648
diff changeset
471 sp.add_argument(
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42648
diff changeset
472 '--no-mercurial-scm-org',
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42648
diff changeset
473 dest='mercurial_scm_org',
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42648
diff changeset
474 action='store_false',
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42648
diff changeset
475 default=True,
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42648
diff changeset
476 help='Skip uploading to www.mercurial-scm.org',
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42648
diff changeset
477 )
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42648
diff changeset
478 sp.add_argument(
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
479 '--ssh-username', help='SSH username for mercurial-scm.org',
42913
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42648
diff changeset
480 )
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42648
diff changeset
481 sp.add_argument(
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
482 'version', help='Mercurial version string to locate local packages',
42913
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42648
diff changeset
483 )
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42648
diff changeset
484 sp.set_defaults(func=publish_windows_artifacts)
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42648
diff changeset
485
43057
c5c502bd1f70 automation: add a command to submit to a Try server
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43055
diff changeset
486 sp = subparsers.add_parser(
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
487 'try', help='Run CI automation against a custom changeset'
43057
c5c502bd1f70 automation: add a command to submit to a Try server
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43055
diff changeset
488 )
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
489 sp.add_argument('-r', '--rev', default='.', help='Revision to run CI on')
43057
c5c502bd1f70 automation: add a command to submit to a Try server
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43055
diff changeset
490 sp.set_defaults(func=run_try)
c5c502bd1f70 automation: add a command to submit to a Try server
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43055
diff changeset
491
42024
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
492 return parser
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
493
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
494
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
495 def main():
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
496 parser = get_parser()
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
497 args = parser.parse_args()
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
498
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
499 local_state_path = pathlib.Path(os.path.expanduser(args.state_path))
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
500 automation = HGAutomation(local_state_path)
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
501
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
502 if not hasattr(args, 'func'):
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
503 parser.print_help()
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
504 return
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
505
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
506 kwargs = dict(vars(args))
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
507 del kwargs['func']
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
508 del kwargs['state_path']
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
509
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
510 args.func(automation, **kwargs)